JP3469231B2 - Animation search device - Google Patents

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to an animation search device.

[0002]

2. Description of the Related Art There are the following documents regarding an animation creating / reproducing apparatus.

Japanese Unexamined Patent Publication No. 5-290144 discloses a device that includes a device for storing basic shape data of character data and a device for selecting basic shape data, and executes an animation in accordance with a description in a predetermined action description language. Has been done.

As a device for outputting an animation by switching a pattern, Japanese Patent Laid-Open No. 6-203139 discloses a device for outputting an animation by switching a pattern over time.

Further, as one using communication, Japanese Patent Laid-Open No. 6-
Japanese Patent No. 203120 discloses an apparatus that performs animation processing such as changing the facial expression of a character on the screen in response to a reply such as "Yes" or "No" to the other party.

[0006] As means for transmitting animation,
Files using compressed moving image files such as QuickTime (Apple Inc.) and MPEG are widely used.

[0007] Regarding facial expressions in animation, there are many themes currently under study, for example, the Institute of Electronics and Communication Engineers, April 1995, pp. Research is being conducted in fields such as realistic communication, intelligent communication, and multi-modal interaction represented by 512-522 "Study on CG Animation of Eyes and Perception of Gaze".
Several achievements have been announced.

On the other hand, as a means for creating a facial expression by CG, a method of adjusting a parameter along the facial expression muscle representing the facial expression has already been established.

[0009]

[Patent Document 1] Japanese Patent Application Laid-Open No. 5-290144 (publication date: November 05, 2005)

[0010]

[Patent Document 2] Japanese Unexamined Patent Publication No. 6-203139 (publication date: July 22, 2006)

[0011]

[Patent Document 3] Japanese Unexamined Patent Publication No. 6-203120 (publication date: July 22, 2006)

[0012]

However, in Japanese Patent Laid-Open No. 5-290144, what kind of data is stored as basic shape data, the display position of each data and its designation, control method for selection and switching, and operation. The specific structure of the description language is not described, and the animation to be realized is unknown.

Further, the apparatus disclosed in Japanese Patent Laid-Open No. 6-203139 switches a predetermined pattern with the passage of time and outputs a predetermined animation, and the user cannot make an animation. Absent. The device described in Japanese Patent Laid-Open No. 6-203120 has a predetermined animation for the answer, and the user can only select the prepared answer pattern.

As described above, these conventional devices lack customizability. When animation is used for communication, it is necessary to reflect the intention of the user, and these methods in which the user cannot create the animation themselves are insufficient.

As a method of transmitting a free animation, there is a method of transmitting a moving image file as described above. These methods have few restrictions on the animation that can be transmitted, but the amount of data is very large and the current transmission is impossible. The method takes a very long time to send, and is not suitable for real-time communication such as chat.

When expressing the contents of a conversation by inputting characters like a conventional device, it is necessary to consider a sentence because the contents to be conveyed by a picture are described in words. Also,
When an animation is used to represent the movement of a person and the action is used to express the content of the conversation, it is necessary to convey it in the same way even if the emotion can be expressed.

The above-mentioned QuickTime and MPE
When an animation is transmitted by a compressed moving image file such as G, the amount of bitmap data to be transmitted / received is large, and it is necessary to use a dedicated high-speed communication line for real-time communication.

In all of the researches on the facial expressions of animation, the focus is on anthropomorphic machines such as the computer automatically detecting the facial expression shown by the user and the computer contacting the user with the facial expression as a reaction to the user. However, it is not intended for the user to easily create an arbitrary facial expression.

The method of creating facial expressions along the facial muscles is used by a professional animator when creating a CG work. To create facial expression animation by this method, a specialized method is required. Knowledge and sense are required.

Further, it is difficult to search the facial expression expressed by animation with the conventional technology / device, and therefore, when communicating, an animation corresponding to information such as emotions intended by the caller is added. It is difficult to communicate by doing.

In order to use an animation representing information such as emotions as one of communication means,
There is a need for a database equipped with various animations that express emotions and a search means that can easily extract animations that accurately express the emotions to be conveyed from the database.

An object of the present invention is to solve these problems, and to provide an animation search device capable of easily extracting an animation that accurately represents a user's intended emotions. Is to provide.

[0023]

[0024]

An animation retrieval device of the present invention comprises emotion parameter input means for inputting a parameter indicating the type of emotion the user wants to express.
A database that stores the data of each part pattern that constitutes a face animation for each kind of emotion in association with the parameters, and the data of each part pattern that corresponds to the parameters input from the emotion parameter input means from the database. The emotion parameter input means includes a control means for searching and outputting the emotion type.
Allocation to one direction of each coordinate axis and emotional strength level
The parameter coordinates corresponding to the absolute value of the coordinate are displayed.
The user can enter the above-mentioned axes and absolute values on the axes.
Enter the type and level of emotions by pressing
It is characterized by being configured as possible .

[0025]

[0026]

The animation search device of the present invention is
Parameter for selecting the type of emotion that the user wants to express
Parameter selection section and the parameters selected in the parameter selection section above.
Required for creating and compositing animations based on data
A data control unit that creates simple script data
Creates and replays the animation depending on the crypto data.
The animation creation / playback section that performs live and the created animation
Animation display part that displays animation on the screen
And the animation creation / playback unit has eyes,
Means for creating a montage consisting of a combination of a plurality of parts patterns such as mouth, hands, feet, etc., means for switching each part pattern of the montage, and a user using a predetermined command to select and switch the part patterns. An input means for inputting with a script consisting of corresponding parameters, an execution means for translating and executing the input script, and a registration means for the user to register a combination of scripts as a macro. Is characterized by.

The animation search device of the present invention is
Parameter for selecting the type of emotion that the user wants to express
Parameter selection section and the parameters selected in the parameter selection section above.
Required for creating and compositing animations based on data
A data control unit that creates simple script data
Creates and replays the animation depending on the crypto data.
The animation creation / playback section that performs live and the created animation
Animation display part that displays animation on the screen
The data control unit corresponds to the parameter input from the parameter input unit, and a database that stores the data of each part pattern that forms a face animation for each emotion type in association with the parameter. It is characterized by comprising control means for retrieving and outputting data of each part pattern from the database.

In the animation retrieval device of the present invention, animation data is retrieved and output according to the parameter indicating the kind of emotion input by the user. Therefore, in conventional communication using electronic networks such as e-mail, not only linguistic information such as text but also an animation that makes it easy to visually understand emotions can be added. It becomes possible to convey the intention of the sender by supplementing the language in expressions such as (when it is angry in words, but when the intention is happy).

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of an animation creating / reproducing apparatus of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an embodiment of an animation creating / reproducing apparatus of the present invention.

When communication is performed by the animation created by the device of this embodiment, the device A having the same configuration as the device A shown in FIG. 1 and the device B having the same configuration are connected via the RS-232C means 2.

In FIG. 1, a processing unit 1 including a CPU and the like.
Is a script data editing means 1a for the user to generate and edit the script, and a script data translating means 1b for separating the script into instructions and parameters,
The script data execution means 1c that actually performs processing according to the contents of the script and the pattern data are read,
The animation display means 1d is displayed at a specified position on the screen of the display device 7 such as a CRT connected to the processing unit.

The part pattern data and the positioning coordinates of each part, which are necessary when displaying a character as shown in FIG. 2, are the part pattern recording means 5 connected to the animation display means 1d and the parts positioning coordinates. It is managed by the recording means 6. R for these
OM or the like is used.

In the script data editing means 1a, a macro recording means 3 for recording macros (list of script commands) created by the user or prepared from the beginning.
Are connected. A backup memory or a flash memory is used for this. The script data execution means 1c has a timer 4 for determining the script execution interval.
The repetitive data recording means 8 for recording repetitive data of the script and the character name registration means 9 for registering the name of the character formed by combining a plurality of part patterns as shown in FIG. 2 are connected. Has been done.

The RS-232C means 2 is used for transmitting / receiving a script to / from another connected device, and transfers the created animation from the machine A to the machine B or vice versa.

The script translation / execution processing of the apparatus of this embodiment will be described below.

The script used in this embodiment is
The commands are described in the format shown in FIG. 4, and if the types are roughly divided, there are three commands: commands to specify / create / delete characters, commands to switch parts, and commands to control animation such as repetition / execution intervals. Is. Each command is divided into a command part (command) and a parameter part, which are separated by the script data translating means 1b, and the corresponding processing is performed by the script data executing means 1c.

The function of each command will be described below.

The sel command is an instruction to register the character name in the character name registration means 9. Once registered, the same sel command can be used to specify the character for pattern switching. This allows control of multiple characters. The registered character is e
It is valid until erased using the rc command.

The set command is an instruction to change the pattern data of the specified character part. The pattern data of each part is as shown in FIG.
Uniquely determined by the part classification code and the part code. For example, the pattern data of the thin body is designated by the part code 01 and the part classification code 00. The animation display means 1d reads the pattern data from the part pattern recording means 5 and uses the coordinates read from the part positioning coordinate recording means 6 to form a pattern at a position corresponding to the selected character as shown in FIG. Is displayed. The priorities at this time are as shown in FIG.

The hed command is an instruction for determining the start of repetition, and the script data execution means 1c uses the h command.
When the ed command is received, the script data is repeatedly stored in the data recording means 8 until the rep command is received.

The rep command is a command for designating the repetition, and the parameter of the repetition number is read, and the script data recorded in the repetition data recording means 8 is executed by the repetition number.

Wit is an instruction for changing the interrupt interval of the timer 4. Since the command is executed at every timer interrupt, the command execution interval can be controlled by this.

%%% is a script terminator, and when it is read, execution is terminated.

By creating a combination of these commands, animation display can be performed by pattern switching.

FIG. 5 is a flow chart of script translation and execution.

First, the script reading start position is set to the head (S1), and the script is read from the head. This script can be user created and local,
It may be received by communication or read by a disk or the like.

Since the command names of the script are all three characters, the command names are read (S2), and the processing is divided according to the conditions (S4 to S10). Until the terminal symbol (%%%) of the script is read, the read start position is moved to the next command start position (S11) and the same processing is repeated.

FIG. 7 is a diagram for explaining a macro used in this embodiment. As shown in the figure, a combination of the above scripts, "Takeshi appeared" and "V sign"
By registering with the macro, it is possible to execute these series of operations by designating one macro. Furthermore, it is also possible to combine a plurality of macros into one macro, such as a macro "Takeshi V sign" that is a combination of "Takeshi appeared" and "V sign". By using such a macro, the time required for the user to input can be shortened, and it becomes possible to respond to the communication partner in real time.

Next, transmission / reception of animation between two devices (machine A and machine B) using the communication function will be described.

When moving a character on the machine B from the machine A, the sel command is first sent to the machine B by RS-2.
32C means 2 is used for transmission. Machine B receives the signal and performs script translation / execution according to the flowchart shown in FIG. In this case, when the character designated by the sel command in the machine B is not registered in the character name registration means 9, registration is performed, and when registered, the control target is changed.

After that, the script sent from the machines A to B is executed for the designated character. As a result, as shown in FIG. 8, a plurality of characters are created on the machine B by the sel command, and the characters are controlled while switching the control target. As shown in FIG. 9, one common character from the plurality of machines is used. It is also possible to control. The erc command is used to delete the registered character.

As with the character data described above, for background data as well, the bitmap data is not directly transmitted / received, but the type, position, size of the part data,
And a command to specify the number.

As shown in FIG. 10, using the communication function,
When the background part C is sent from the machine A to the machine B, a command for selecting the part C is sent to the machine B. At this time, data on the position, size (reduction rate), and number of the parts C are also transmitted as signals at the same time. Machine B receives these signals and translates and executes the signals. As a result, a plurality of background parts can be created on the machine B by the selection command, and the parts can be changed or deleted according to the scene. You can also create a common background from multiple machines.

An embodiment of the animation retrieval device of the present invention will be described below with reference to the drawings.

FIG. 11 is a block diagram showing the configuration of the embodiment of the animation search device of the present invention.

The animation search device according to the present embodiment is necessary for creating / synthesizing an animation based on the parameter selection unit 101 for selecting the emotion type and emotion level that the user wants to convey, and the parameter relating to the selected emotion. Data control unit 102 for creating various script data, and animation creation / playback unit 10 for creating / playing animation according to the script data.
3, an animation display unit 104 for displaying the created animation on the screen, a determination result input unit 105 for the user to input whether or not the displayed animation is intended, and the animation is output as data. And a data output unit 106.

As the animation creating / reproducing unit 104, a device having the same configuration as the device described in the above-described embodiment of the animation creating / reproducing device can be applied, and in the following description, it is assumed that the device has the same configuration. To do.

Next, referring to FIGS. 11 and 12,
The operation of the animation search device according to the present embodiment will be described.

First, the user selects the parameter selection unit 10
The emotion to be expressed and the emotion level are selected and input via 1 (S101).

Based on the input parameters, the data control unit 102 outputs the data necessary for creating and synthesizing the animation in the form of script data (S1).
02).

Upon receiving the script data, the animation creation / playback unit 104 creates the animation
An animation is created by the procedure described in the embodiment of the playback device (S103).

The created animation is displayed on the screen by the animation display unit 104 (S10).
4).

The user looks at the animation on the screen and inputs via the determination result input unit 105 whether or not it is the facial expression intended by the user (S105).

If the result is OK, the script data created by the data control unit 102 is output from the data output unit 106 to an e-mail editor or the like (S106).

The user makes a No determination in S105
If is input, the mode is again set to accept the parameter input and the process returns to S101.

The data control unit of this embodiment will be described below.

The data control section of the present embodiment comprises a database section for storing part pattern data of facial expressions corresponding to emotion types and levels, and a control section for performing processing according to input parameters. It is configured.

As shown in FIG. 13, the database unit classifies the facial expression pattern data according to the type and level of emotion for each part of eyebrows, eyes, nose, upper cheeks, mouth and lower cheeks. And store it. The data of each part is registered as script data (part code) that creates an animation.

When a plurality of emotion types are selected, the control unit carries out a process as to how to process the data of each part. For example, when “joy” and “sadness” are selected, processing is performed on how to handle the “eyebrow” data. The following three can be considered as the processing in this case.

(1) Either emotion is given priority. If there is a difference in level, the one with the higher level is given priority, and if the levels are the same, the preset priority order is followed.

(2) Use intermediate data.

(3) The processing method is stored for each case, and the processing is performed based on the stored information.

The parameter selection unit of this embodiment will be described below with reference to FIG.

“Pleasure”, “Sadness”, which are assigned to the predetermined direction of each coordinate, respectively, the kind of emotion to be expressed,
One or more are selected from "surprise" and "anger", and each emotion level is indicated by a broken line in the figure.
Select from among the levels of stages.

When selecting, the axial points of the coordinate axes shown on the screen are designated via an input device such as a mouse or arrow keys. The designated area is marked so that it can be easily visually recognized, as indicated by an X in the figure.If the search result is No, the marking area is fed back to the selection section until a new designation is made. Keep it. When the search result is OK by the user, the marking display is erased at that point and the original state is restored.

The above description is based on the embodiment of the animation search device of the present invention, but various methods can be used for setting the emotion type and level parameters to be selected depending on the purpose of communication, the user or the terminal device. Can be considered. For example, when the search device of the present invention is applied to a simple terminal, it is possible to omit the emotion level and select and input only the type.

Regarding the pattern data registered in the database shown in FIG. 13, not only the above-mentioned six parts from the eyebrows to the lower cheeks, depending on the purpose, user and terminal,
It is possible to increase or decrease the number of patterns.

[0080]

According to the animation retrieval apparatus of the present invention, it is possible to retrieve an animation corresponding to an emotion to be expressed, and in addition to a language message such as text or voice, an animation expressing an emotion can be used. It becomes possible to send. This makes it possible to “visualize emotions and communicate”, which was difficult with conventional telephones, letters, and emails. As a result, it is possible to convey emotions by animation without considering difficult language expressions that convey emotions during transmission, and the information intended by the sender is transmitted to the receiver without misunderstanding. This is effective when communicating in a language other than the language normally used by the sender.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an animation creating / reproducing apparatus of the present invention.

FIG. 2 is a diagram showing a configuration example of character data including a plurality of part patterns.

FIG. 3 is a diagram showing priorities when displaying a part pattern.

FIG. 4 is a diagram showing a format of a script.

FIG. 5 is a flowchart of script translation / execution.

FIG. 6 is a diagram showing a data structure of a part pattern.

FIG. 7 is a diagram showing a relationship between a script and a macro.

FIG. 8 is an explanatory diagram of an example of controlling a plurality of characters by communication.

FIG. 9 is an explanatory diagram of an example of controlling one character from a plurality of machines.

FIG. 10 is an explanatory diagram of an example of transmitting background data.

FIG. 11 is a diagram showing a configuration of an animation search device of the present invention.

FIG. 12 is a flowchart illustrating an animation search operation.

FIG. 13 is a diagram showing a structure of a database.

FIG. 14 is a diagram showing a configuration of a parameter selection unit.

[Explanation of symbols]

1 processing unit (CPU) 1a Script data editing means 1b Script data translation means 1c Script data execution means 1d animation display means 2 RS-232C means 2a Script data transmission means 2b Script data receiving means 3 Macro recording means 4 timer 5 Parts pattern recording means 6 Parts positioning coordinate recording means 7 Display means (CRT) 8 Repeated data recording means 9 Character name registration means 101 Parameter selection section 102 data control unit 103 Animation Creation / Playback Section 104 Animation display section 105 Judgment result input section 106 data output section

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Natsuko Ikeda 22-22 Nagaikecho, Abeno-ku, Osaka City, Osaka Prefecture Sharp Corporation (56) Reference JP-A-6-203127 (JP, A) JP-A-3- 156573 (JP, A) JP-A-2-216588 (JP, A) JP-A-7-210652 (JP, A) JP-A-4-287176 (JP, A) JP-A-8-30803 (JP, A) JP-A-5-233779 (JP, A) Ryumi Sakaguchi, 3 others, "Scenario description tool for creating facial animation and real-time moving image display", IEICE technical research report (HC91-54 ~ 60 Human Communication), The Institute of Electronics, Information and Communication Engineers, March 10, 1992, Vol. 91, No. 508, p. 23-30 Tomoo Inoue and 2 others, "Non-verbal emotional expressions for emotional communication", IPSJ Research Report (94-DPS-63 94-GW-5), Information Processing Society of Japan, January 28, 1994, Vol. 94, No. 12, p. 33-40 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 13/00 G06T 11/80 G06F 17/30 G06T 15/70 CSDB (Japan Patent Office)

Claims (3)

(57) [Claims] 1. An emotion parameter input means for inputting a parameter indicating a type of emotion desired to be expressed by a user, and data of each part pattern forming an animation of a face for each type of emotion is stored in association with the parameter. And a control means for searching and outputting the data of each part pattern corresponding to the parameter input from the emotion parameter input means from the database, and the emotion parameter input means determines the emotion type for each coordinate axis.
Is assigned to one direction and the level of emotion
Display the parameter coordinates corresponding to the absolute value, and
Input the above-mentioned coordinate axis and the absolute value on the coordinate axis
Allows input of the type and level of emotion
An animation search device characterized by being used. 2. The type of emotion the user wants to express is selected.
Based on the parameters selected by the parameter selection section and the above parameter selection section
Script for creating and compositing animation
The data control unit that creates data and the creation of animation according to the script data
An animation creation / playback unit that performs playback and playback, and an animation that displays the created animation on the screen.
The animation creation / playback unit is a combination of a plurality of parts patterns such as eyes, mouth, hands, and feet.
Means for creating a montage consisting of, and means for switching each part pattern of the montage
And the user has to select, switch, etc. the part pattern.
Command consists of a predetermined command and corresponding parameters.
Input means for inputting by crypto, execution means for translating and executing the input script, and user registering a combination of scripts as a macro
That it comprises a registration unit, wherein the to luer <br/> animation search device for. 3. A parameter selection unit for selecting the kind of emotion the user wants to express, and a data control unit for creating script data necessary for creating and synthesizing an animation based on the parameters selected by the parameter selection unit. When the animation creation and playback unit that performs animation creation and playback in accordance with the script data, and a animation display unit for displaying an animation that is created on the screen, the data control unit, for each type of emotion Facial animation
The data of each part pattern that makes up the
Data stored in association with the parameters and the parameters input from the parameter input means.
Data of each corresponding part pattern is the above database
An animation retrieval device comprising a control means for retrieving and outputting from an animation.