JP3437686B2 - Display device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device for displaying a screen in which a character string and an image are used together, and more particularly to a display device suitable for dialogue involving images.

[0002]

2. Description of the Related Art Conventionally, a personal communication system has been constructed in which terminal devices such as personal computers are connected to each other via a communication line, and those personal computers are operated to send a message to the other party to carry out a dialogue. ing. FIG. 18 is a block diagram showing an example of a conventional terminal device used in a personal communication system.

Conventionally, a dialogue system mainly using characters has been adopted. This is also the case when searching the database in the center of the personal communication system on the terminal side,
Online between unspecified personal communication system members
The same applies when a chat (immediate character dialogue) is performed.
When the user inputs a character string from the keyboard 11 of the terminal, the input character string is first input to the display mechanism 13 that controls the screen displayed on the display 14, and the input character string is displayed on the display 14 under the control of the display mechanism 13. Is displayed. When the user confirms the input character string by operating the keyboard 11, the confirmed character string is encoded by the encoder / decoder 12 and passed through a communication device such as the modem 15 or a public line network. It is sent to the center or the other party. The partner terminal is also configured in the same manner as the terminal shown in FIG. 18, and the character string transmitted to the partner via the public line network is decoded by the partner encoder / decoder. The converted character string is input to the display mechanism, and the transmitted character string is displayed on the display of the other terminal. In this way, mutual or unspecified number of users repeatedly perform the same operation to have a dialogue.

Since the above-mentioned dialogue method uses only character strings, it has a drawback that emotional expressions associated with the dialogue cannot be performed well and it is difficult to conduct a flexible and delicate dialogue. As an improvement, a dialogue communication method that involves not only character strings but also images such as animation has been proposed ("Lucas Film's Habitat Lesson" Cyberspace,
NTT Publishing (1994) pp282-307, "CY
BERSPACE: First Steps ”, edi
ted by Michael Benedikt,
See The MIT Press (1991)).

FIG. 19 is a block diagram showing an example of a terminal device used in the interactive communication system proposed therein.
In the interactive communication system proposed here, each user has an animation person image, and each animation person can change a facial expression and a posture according to the instruction of the user. Therefore, compared to a simple dialogue using only character strings, a dialogue expressing emotions can be performed.

[0006]

However, in this method, for the operation of the movement of an image (animation) and the expression of emotion, a menu is displayed on the display 14 and a desired selection technique is performed by the pointing device of the mouse 16. An operation method of selection is adopted, and the user needs to perform both interactive character input from the keyboard 11 and instructions for image movement and emotion expression with the mouse 16 or the like. Only the character string is first transmitted and displayed on the other party's display,
Then, using the mouse 16 or the like, it is preferable that the emotional expression is instructed and the animation on the display finally changes to the facial expression corresponding to the displayed character string, so that it is preferable that the character strings are displayed simultaneously at the same time. There is a problem in that there is a time lag between the emotional expression and the emotional expression, and it is still difficult to convey the natural emotional expression.

In view of the above circumstances, the present invention provides a display device capable of continuously performing an image operation and a character input operation and capable of displaying a screen in which a character string and an image are used in combination. With the goal.

[0008]

The first display device of the display devices of the present invention which achieves the above object is a display device which displays a screen in which a character string and an image are used in combination (1) Character string input operator for operating and inputting the character string of (2) The input character string input by operating the character string input operator is converted into an interactive character string displayed as a character string and an image. A character string classifying unit that classifies the control character string into an image corresponding to the control character string, and an interactive character string that is classified by the character string classifying unit.
It is characterized by comprising a screen display means for displaying a screen composed of an image obtained by converting the control character strings classified by the character string classifying means by the image converting means.

Here, the first display device of the present invention is
Furthermore, transmission means for transmitting a character string, is the ash and a receiving means for receiving the character string. In this case, in response to the fact that the upper Symbol receiving means predetermined identification string is received, be provided with response means for transmitting by said transmission means a predetermined identification string corresponding to the identification string, good <br/> not better. Further, the first display device of the present invention further causes the transmission means to transmit a predetermined identification character string, and within a predetermined time from the transmission, the predetermined identification character string corresponding to the transmitted identification character string. Is provided by the receiving means, and the transmitting means determines whether or not the transmitting means confirms that the predetermined identification character string is received within a predetermined time. , the input string containing both the interactive character string and the control character string, or it shall be the feature is to send a conversation string classified by the character string classifying means.

Here, the first display device of the present invention has a control character string dictionary in which control character strings are registered, and the character string classification means refers to the control character string dictionary to input characters. It is preferable to classify the string into an interactive character string and a control character string. In that case, the control character string dictionary provides information for classifying each control character string into a first type that is used only as a control character string and a second type that is used as a control character string and is also used as an interactive character string. It is also a preferable aspect that the character string classifying means classifies the input character string corresponding to the second type control character string into both a control character string and an interactive character string.

The second of the above-mentioned display devices of the present invention .
The display device of (1) is similar to the above-mentioned first display device.
(4) further comprises a pattern database in which image pattern data representing an image corresponding to the control character string is registered, and the image conversion means refers to the pattern database and classifies by the character string classification means. An image corresponding to the generated control character string is generated . In this case, the pattern database has a plurality of image pattern data registered corresponding to at least one control character string, and the image conversion means has a control of a plurality of image pattern data registered. When converting the character string into an image, a plurality of images corresponding to the plurality of image pattern data may be sequentially generated.

The second display device of the present invention further comprises:
The input character string includes a deformation instruction parameter that is attached to the control character string and that instructs deformation of the image, and the image conversion means converts the control character string with the deformation instruction parameter into an image into a pattern database. registered, the image pattern data corresponding to the control string, Ru der to produce new images by deforming based on the deformation instruction parameters. In that case, the pattern database is formed by registering a plurality of image pattern data corresponding to at least one control character string to which a deformation instruction parameter can be added, and the image conversion means includes a plurality of image pattern data. When generating an image corresponding to a control character string in which is registered, a plurality of images corresponding to the control character string registered in the pattern database are generated.
Image pattern data based on its deformation instruction parameters
A new image may be generated by transforming it.

Furthermore, the third of the display devices of the present invention is used.
The display device is the same as the first display device of the present invention.
It is equipped with (1) to (4) and corresponds to the control character string.
The image pattern data representing the image to be registered is registered.
It has a pattern database and the image conversion means
By referring to the turn database, the character string classification means
Generates an image corresponding to the classified control string.
In addition, the input character string further includes a plurality of control character strings that are combined with each other, and the image conversion means converts the plurality of control character strings that are combined with each other into an image. Ru der to produce new image by combining a plurality of image pattern data corresponding to a plurality of control strings. The problem with the conventional technology is that when a character string is input, a keyboard or the like is used, and when an animation operation is performed, a mouse or the like is used. Characters and images are input and operated using separate operation means. There is a root cause in doing. The present invention is to define a control character string for image operation and sequentially input both the interactive character string and the control character string by using only a keyboard, for example. As a result, both the interactive character string and the control character string can be sequentially input in succession, and the interactive character string and the emotion desired to be expressed together with the interactive character string can be displayed almost at the same time. It is possible to realize dialogue with expressions.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below. FIG. 1 is a block diagram showing a schematic configuration of an embodiment of a display device of the present invention. Referring to this FIG.
First, the outline of this embodiment will be described. The components corresponding to the components of the conventional terminal shown in FIGS. 18 to 19 described above are designated by the same reference numerals as those shown in FIGS. 18 and 19, and the differences will be described.

An input character string input from the keyboard 11 is input to the character string separating mechanism 21 and the display 1
4 is classified into an interactive character string displayed as a character string and a control character string for controlling an animation image, that is, a control character string converted into an animation image and displayed on the display 14. The interactive character string is input to the display mechanism 13 as it is, and the control character string is input to the pattern selection mechanism 22. Pattern selection mechanism 2
2 stores a plurality of predetermined animation images corresponding to a plurality of control character strings, and when the control character string is input to the pattern selection mechanism 22, the input control character string is It is converted into an animation image and the animation image is sent to the display mechanism 13. The display mechanism 13 displays characters and animation images at predetermined positions on the display 14, respectively.

The input character string from the keyboard 11 is sent to the encoder / decoder 12 as it is and encoded while including both the interactive character string and the control character string.
It is transmitted to the other party through the communication system such as the modem 15 and the public line network. When a character string is input from the other party of the conversation via the public line network, the character string is decoded by the encoder / decoder 12 and then input to the character string separating mechanism 21. It is handled in the same way as the input character string from the keyboard 11. If the terminal on the other end of the dialogue is also provided with the character string separating mechanism and the pattern selecting mechanism similar to those shown in FIG. 1, by communicating the input character strings input from the keyboards of the two with each other, Dialogue with both sides of the animation image is performed.

FIG. 2 is a diagram showing an example of a character string input from the keyboard 11, and FIG. 3 is a diagram showing changes in the screen of the display 14 due to the input character string of FIG. As illustrated in FIG. 2, the input character string from the keyboard 11 is made up of an interactive character string indicating normal interactive content and a control character string inserted between the interactive character strings.

[0018] In the example of FIG. 2, following the normal dialogue sentence "Hello" (interactive character string), the symbol column (control strings) that * hello showing the expression of the time followed, further "Today good It's weather, "followed by a sequence of symbols:-). Where * he
llo represents an action in which a person image on the screen greets, and :-) represents an expression that the person image laughs, and both are registered as rules in advance. That is, as shown in FIG. 3, the character string of FIG. 2, initially displays sentences on the display as "Hello", and immediately thereafter, the person image displayed on the most delay rather, the same screen hand Next, a display operation is performed in which the sentence "Today's weather is nice" is displayed on the display and the person's image changes to a smile.

FIG. 4 is a block diagram showing an embodiment of the display device of the present invention, and FIG. 5 is a schematic diagram showing each part of the control character string dictionary and pattern database shown in each block in FIG. For example, it is assumed that the character string in FIG. 2 is input from the keyboard 111. This input character string is temporarily stored in the character string buffer 211, and the sequencer 212 sequentially searches the stored character string from the beginning to cut out a certain group of character strings. For example, if the rule that the character string used for normal dialogue is always enclosed in "", the character string in the part not enclosed in "" is regarded as the control character string, and only that part can be cut out. is there. The same applies when other rules for distinguishing the dialogue character string and the control character string are determined. In the case of the example in FIG. 2, the part of * hello not enclosed by "" is the sequencer 212.
Cut out by.

Next, in order to confirm whether the character string * hello cut out by the sequencer 212 is registered as a control character string, the character string * hello is checked.
Is sent to the character string comparator 213. Character string comparator 213
Compares the character string “* hello” sent from the sequencer 212 with the key of the control character string dictionary 1 created in advance, searches for a match, and if there is one, Output the animation data number corresponding to the key. A configuration example of the control character string dictionary 1 and the pattern database 2 is as shown in FIG. A character string “* hello” is registered as a key in the control character shown in FIG. 5, and the animation data number pattern-1 corresponding to the character string is output. If the character string corresponding to the control character string dictionary 1 is not registered as a key, the character string comparator 213 returns No output to the sequencer 212. When the sequencer 212 receives the output of No, the character string that has just been cut out is not surrounded by "", but is regarded as it is as a character string for dialogue and is sent to the display mechanism 113.

When the character string comparator 213 searches for the animation data number registered in the control character string dictionary, the pattern selector 221 searches for the actual animation data (image) based on the animation data number. . In the present embodiment, a plurality of animation images are stored in the pattern database 2 corresponding to one animation data number, and the plurality of animation images are sequentially sent to the display mechanism, and the animation is displayed on the display 141. The image is displayed.

In the example of FIG. 2, the animation (pattern-1) corresponding to * hello is composed of three images. The first image shows the state before raising the hand, the second image shows the state where the right hand is raised, and the third image shows the state where the hand is lowered again. Relative time is written under each image, and indicates approximately at what time interval the image is transferred to the display mechanism. The example of pattern-1 indicates that the animation is performed at a time interval of updating the system time by one. :-) corresponding to the facial expression of pattern-2 is set so that the third image is executed at time 8, which is to restore the original facial expression after a considerable time interval from the second image. Is shown. By writing the relative time in each image of such an animation, the update rate of the animation is controlled.

The character string regarded as a normal dialogue sentence by the sequencer 212 (the portion surrounded by "") is not transferred to the character string comparator 213 but is sent to the display system as it is and displayed as a normal character string. Is done. Display 141
, The normal dialogue character string sent from the sequencer 212 and the animation image sent from the pattern selector 221 are simultaneously displayed, and the animation image changes with time. The character string is usually displayed only within a predetermined time.

The control character string dictionary shown in FIG. 5 is provided with a display flag column. When "0" is entered here, the registered control character string is used only for animation operation. Not displayed as a character string, and when "1" is entered here, it means that the control character string is displayed not only as an animation operation but also as an interactive character string at the same time.

In the case of FIG. 5, the registered control character string "* h"
Since the display flag “0” is set for “ello”,
It only displays the animation pattern-1. Since the display flag 1 is set for the control character string ":-)", at the same time as displaying the animation pattern-2, the dialog character string "today is good weather" is followed by ":-)". Display as a string. In other words, "It's a nice day today :-)". Further, as will be described later, in the embodiment shown in FIG. 4, a mode in which an input character string input from the keyboard 111 is transmitted including both an interactive character string and a control character string, and the input character string The mode can be switched to the mode in which only the selected dialogue character string is sent. However, even when only the dialogue character string is sent, the ":-)" part is sent to the other party in the same way as a normal dialogue character string. , Even if the communication system on the other side has only the function of displaying the transmitted character string as it is, the expression symbol “:
-) "Is transferred and displayed together, so that the dialogue can be given an expression.

Further, in the embodiment shown in FIG. 4, the information necessary for converting the control character string into the animation image is the control character string dictionary 1 and the pattern database 2.
Therefore, the control character string and the animation image are easily associated with each other. Moreover, here, the control character string dictionary 1 and the pattern database 2 can be added and rewritten by the operation of the user's keyboard or the like, the user can register any character string as a control character string, and can also use any animation image. Can be registered in association with the control character string.

In the embodiment shown in FIG. 4, a mode in which the character string input from the keyboard 111 is transmitted as it is, that is, a mode in which both the interactive character string and the control character string are included and a mode in which only the interactive character string is transmitted are transmitted. Switched to. When transmitting a character string including both an interactive character string and a control character string, as shown in FIG. 4, the character string buffer 211 directly transmits the transmission character string to the encoder / decoder 121. In this case, if the other party has the same character string separating mechanism as in FIG. 4, the same dialogue sentence and animation operation as in FIG. 4 can be executed. Also, if the other side does not have the same mechanism and has only the function of displaying the character string as it is, the control character string is also displayed as a character string as an interactive sentence, but at this time the control character string is also normal. Since it is composed of only the character code used for (for example, ASCII character code), it does not matter even if it is displayed as it is on the other party's terminal.

On the other hand, after the character string is transferred from the character string buffer 211 to the sequencer 212 and the character string is separated into the interactive character string and the control character string, only the interactive character string is encoded / decoderd from the sequencer 212. 121 can be transferred. In this case, only the dialogue character string not including the control character string is transmitted to the other party. The other side has a normal dialogue using only the character string.

As described above, the character string buffer 211
Can directly send the transmission character string to the encoder / decoder 121 or send only the dialogue character string without the control character string from the sequencer 212 to the encoder / decoder 121. . It is easy for the user to instruct this from the outside to switch it, but in the present embodiment, which transmission mode is used is automatically switched as described below.

FIG. 6 is a flow chart showing the procedure for selecting the transmission mode. First, it is confirmed whether or not the other terminal has a similar character string separation mechanism or animation display mechanism by exchanging a predetermined identification character string with each other.
The confirmation method is as follows. Monitoring unit 122 shown in FIG.
Then, as shown in FIG. 6, first, a predetermined identification character string (for example, "OK") is transmitted to the other party, and when the other party can accept it, the identification character string "OK" is returned as it is. To do. The identification character string for transmission and the identification character string for reception may be different. After transmitting the identification character string, the monitoring unit 122 monitors whether the identification character string is returned from the other party. If the other party does not return the identification string within the specified time after sending the identification string, it means that the other party does not have the same mechanism. In this case, only the dialogue string is sent. The character string sent from the other party is also displayed on the display 141 as it is. On the other hand, when the identification character string is returned within a predetermined time, both the self side and the other side have the same mechanism, and therefore the character strings including the control character string are transmitted to each other. On the other hand, when the identification string is sent first from the other party,
The response unit 123 recognizes that the identification character string has been sent, immediately returns the identification character string to the other party who sent the identification character string, and a character string including the control character string with the other party. Send each other.

FIG. 7 is a block diagram showing a schematic configuration of another embodiment of the display device of the present invention, FIG. 8 is a diagram showing an example of an input character string, and FIG. 9 is a diagram showing the input character string of FIG. ,
FIG. 6 is a diagram showing a change in screen of the display 14. Figure 1
Differences from the embodiment shown in FIG. 3 will be described. In the present embodiment, as shown in FIG. 8, the input character string includes a dialogue character string, a control character string, and a transformation instruction parameter associated with the control character string. In the example shown in FIG. 8, the modification instruction parameters (0.8) and (0.9) are attached to the control character string * hello, :-), respectively. The transformation instruction parameter is a parameter that indicates the influence of the corresponding control character string on the animation image, that is, the degree of transformation of the animation image.

If the correspondence between the control character string and the motion or facial expression of the animation image is always fixed, the flexibility of the facial expression or motion is reduced. It enables the display of animated images with finely detailed facial expressions and movements, and enables dialogue that includes richer emotional expressions.

The animation image selected by the pattern selection mechanism 22 shown in FIG. 7 is sent to the pattern automatic deformation mechanism 31 together with the deformation instruction parameter. The pattern automatic transformation mechanism 31 transforms the sent animation image based on the transformation instruction parameter, and sends the transformed animation image to the display mechanism 13. The display mechanism 13 displays characters and animation images at predetermined positions on the display 14, respectively.

Further, the input character string from the keyboard 11 includes both the interactive character string and the control character string, and further the encoder / decoder 12 while including the modification instruction parameter attached to the control character string. Sent to and encoded by the modem 15
Etc. is transmitted to the other party via a communication system such as a public network. When a character string is input from the conversation partner side via the public line network, the character string is decoded by the encoder / decoder 12 and then input to the character string separation mechanism 21. The character string is handled in the same manner as the input character string from the keyboard 11.

As shown in FIG. 2, the character string in this embodiment is composed of an interactive character string, a control character string, and a modification instruction parameter attached to the control character string. In the example of FIG. 8, "Hello" that followed the normal dialogue (dialogue string), the symbol column (control strings) that * hello showing the expression of the time followed, further of the control string * hello The deformation instruction parameter (0.8) follows. The transformation instruction parameter (0.8) indicates that the operation pattern corresponding to the control character string * hello, in which the right hand is raised and greeted, is displayed at a degree of 80%, and the right hand is completely upward. Can be raised to 80% without being raised.

In the example of FIG. 8, the dialogue character string "Today is good weather" is followed by the control character string indicating a smile:-), and the transformation parameter (0.9) is further added. in the process of. In this case, the pattern selection mechanism 22 shown in FIG. 7 selects two types of animation patterns, a non-expression pattern and a smile pattern, and sends them to the pattern automatic transformation mechanism 31. From the facial expression animation pattern and the smile animation pattern, by interpolation calculation,
An animation pattern that is 90% closer to a smile is generated.

FIG. 10 is a block diagram showing a more detailed embodiment of the display device of the present invention based on the idea according to the embodiment shown in FIG. 7, and FIG. 11 is a control shown by each block in FIG. It is a schematic diagram which shows each part of a character string dictionary and a pattern database. Differences from the embodiment shown in FIGS. 4 and 5 will be described. As shown in FIG.
In the control character string dictionary, control character strings and animation data numbers are registered in association with each other. Also,
Two animation pattern data of a minimum model and a maximum model are registered in the pattern database corresponding to each animation data number.

In the sequencer 212 shown in FIG. 10, for example, the control character string :-) and the transformation instruction parameter (0.9) in the character string shown in FIG. 8 are cut out, and the control character string :-) is compared with the character string. To the deformation unit 213, and the deformation instruction parameter (0.
9) is sent to the segment interpolator 311. The character string comparator 213 compares the control character string :-) sent from the sequencer 212 with the control character string registered in the control character string dictionary 1 in advance to search for a match. If there is a match, the animation data number (animation data number pattern-2 in this example) corresponding to the control character string is output.

The pattern selector 221 receives the animation data number pattern-2 and receives the animation data number pat from the pattern database.
An animation image corresponding to tern-2 (both the minimum model and the maximum model) is read and sent to the segment interpolator 311. The segment interpolator 311 deforms the sent animation image based on the transformation instruction parameter as described below, and sends the transformed animation image to the display mechanism 113. The display mechanism 113 displays the interactive character string sent from the sequencer 212 and the animation image sent from the segment interpolator 311 at each predetermined position on the display 141.

FIG. 12 is a schematic diagram for explaining an interpolation calculation method executed by the segment interpolator 311.
FIG. 6 is a flowchart showing the interpolation calculation method. 12
As shown in (A), the minimum model and the maximum model respectively represent an expressionless face and a smile of 100%, and the segments constituting these expressions are S1 to S, respectively.
5, L1 to L5.

FIG. 12B shows the interpolation of the constituent segment representing the shape of the mouth out of the respective five constituent segments. By the method shown in FIG. 13, the smallest model constituent segment S (t) is shown. And maximum model configuration segment L
The constituent segment P (t) of the interpolated image is generated from (t). Here, t (t = 0 to 1) is the constituent segment S
(T) and L (t) represent an array of pixels, and S (0)
Point corresponds to L (0), and point S (1) corresponds to point L (1). The transformation instruction parameter is k (k = 0 to 0
1), the interpolation element P (t) is represented by P (t) = (1-k) .S (t) + k.L (t) (1). As shown in FIG. 13, the above formula (1) is calculated while sequentially changing t from 0 to 1 for each constituent segment, and this is repeated for all constituent segments to generate an interpolated image. .

Here, if the above equation (1) is modified, it can be expressed as P (t) = S (t) + k.multidot. (L (t) -S (t)) (2). That is, the equation (2) is obtained by multiplying the difference image (L (t) -S (t)) between the maximum model L (t) and the minimum model S (t) by the deformation instruction parameter k, and the minimum model. It represents addition with S (t). Therefore, instead of registering the minimum model and the maximum model in the pattern database shown in FIG. 11, the minimum model S (t) and the difference image (L (t) -S) are stored.
(T)) may be registered and the interpolated image P (t) may be obtained by the calculation of the equation (2).

The above is an example in which the degree of smile is changed for one facial expression, for example, a smile. However, in actual humans, a plurality of facial expressions such as a smile and a crying face or an angry face are mixed intricately. There is something to show. Hereinafter, a method of generating such a complicated facial expression will be described. FIG. 14 is a diagram showing a configuration example of a character string when combining a plurality of facial expressions, and FIG. 15 is a flowchart showing a method of combining the plurality of facial expressions.

Here, there is a case where only a plurality of control character strings are continuous without interposing an interactive character string, or a case where a plurality of control character strings are continuous with only a deformation instruction parameter interposed therebetween. When the control character string of is a control character string belonging to the same genre such as facial expression,
The control strings are considered to be combined with each other.

In the flow of FIG. 15, the portion surrounded by the alternate long and short dash line is the same as that of the flow of FIG. 13 and its explanation is omitted. Here, Q (t) is first initialized, and the calculation for one facial expression is overlaid on Q (t). This superposition is performed by the same interpolation processing as that in the alternate long and short dash line.

The number of interpolation images generated is not limited to one, and a method of continuously transforming an animation image from one image to another may be adopted. 16
50% smile :-) (0.5) to 80% angry face:-
It is the figure which showed the character string which instruct | indicates to transform continuously to <(0.8). 50% smile :-) (0.5)
And 80% of the angry faces:-<(0.8), the continuous deformation indicator → is inserted, and this continuous deformation indicator → is the facial expression immediately before (50% smile) to the facial expression immediately after. It is instructed to continuously transform it into (80% angry face).

FIG. 17 is a flow chart showing a method for realizing this continuous deformation. First, the deformation calculation is performed on the first image (here, a smile) according to the deformation instruction parameter (0.5) (see FIG. 13). The 50% smile generated by this is set as the pattern P1 (t). Similarly, the deformation calculation is performed on the second image (here, angry face) according to the deformation instruction parameter (0.8). Let that pattern be P2 (t).

Next, these two patterns P1
Two patterns P1 for each value δ while sequentially changing the value of the variable δ (δ = 0 to 1) corresponding to the deformation parameter when (t) and P2 (t) are used as reference patterns.
An interpolated image between (t) and P2 (t) is obtained. In this way, an animation image sequentially transformed from a smile of 50% to an angry face of 80% is generated and displayed on the display 141.
Is displayed in.

[0049]

As described above, according to the present invention,
The dialogue character string and the image corresponding to the dialogue character string can be displayed without a large time delay between them, and the dialogue in which the emotional expression is expressed more appropriately can be performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of a display device of the present invention.

FIG. 2 is a diagram illustrating an example of a character string input from a keyboard.

FIG. 3 is a diagram showing changes in the screen of the display according to the input character string in FIG.

FIG. 4 is a block diagram showing an embodiment of a display device of the present invention.

FIG. 5 is a schematic diagram showing each part of a control character string dictionary and a pattern database shown in each block in FIG.

FIG. 6 is a flowchart showing a procedure for selecting a transmission mode.

FIG. 7 is a block diagram showing a schematic configuration of another embodiment of the display device of the present invention.

FIG. 8 is a diagram showing an example of an input character string.

FIG. 9 is a diagram showing changes in the screen of the display according to the input character string shown in FIG.

FIG. 10 is a block diagram showing a detailed embodiment of the display device of the present invention.

FIG. 11 is a schematic diagram showing each part of the control character string dictionary and the pattern database shown in each block in FIG.

FIG. 12 is a schematic diagram for explaining an interpolation calculation method executed by a segment interpolator.

FIG. 13 is a flowchart showing an interpolation calculation method.

FIG. 14 is a diagram showing a configuration example of a character string when a plurality of facial expressions are combined.

FIG. 15 is a flowchart showing a method of combining the plurality of facial expressions.

FIG. 16 is a diagram showing a character string instructing continuous transformation from 50% smile :-) (0.5) to 80% angry face:-<(0.8).

FIG. 17 is a flowchart showing a method for realizing this continuous deformation.

FIG. 18 is a block diagram showing an example of a conventional terminal device.

FIG. 19 is a block diagram showing an example of a conventional terminal device.

[Explanation of symbols]

1 Control string dictionary 2 pattern database 11 keyboard 13 Display mechanism 14 display 15 modem 16 mice 21 Character string separation mechanism 22 Pattern selection mechanism 31 Automatic pattern deformation mechanism 111 keyboard 121 Encoder / Complexer 122 Monitor 123 Response part 131 display mechanism 141 display 211 buffer 212 Sequencer 213 Character comparator 221 pattern selector 311 segment interpolator

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Takashi Ohno 1015 Kamiodanaka, Nakahara-ku, Kawasaki City, Kanagawa Prefecture, Fujitsu Limited (56) Reference JP-A-4-261274 (JP, A) ▲ Masanori Sakaki, PC -9801 Series B Introduction to ASIC, ASCII Publishing, Japan, September 15, 1985, first edition, p. 30, 42-44, 72, 246-247 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 3/00 651 G09G 5/00 510

Claims (5)

(57) [Claims] 1. A display device for displaying a screen in which a character string and an image are used in combination, a character string input operator for operating and inputting an arbitrary character string, and an input by operating the character string input operator. A character string classification means for classifying the input character string into an interactive character string displayed as a character string and a control character string converted into an image; converting the control character string into an image corresponding to the control character string And a screen including an interactive character string classified by the character string classifying unit and an image obtained by converting the control character string classified by the character string classifying unit by the image converting unit. Screen display means, transmission means for transmitting character strings, character strings
Predetermined by the receiving means for receiving
The identification character string of is transmitted and within a predetermined time from the transmission.
A predetermined identification character string corresponding to the identification character string
Equipped with monitoring means to monitor whether or not it is received by the receiving means.
If the transmitting means is within a predetermined time by the monitoring means,
A specific identification character string is confirmed to be received.
Input sentence containing both control string and dialogue string
Character strings, or classified by the character string classification means
A display device characterized by transmitting an interactive character string . 2. A control statement in which the control character string is registered
A character string dictionary, wherein the character string classification means
Referring to the dictionary, the input character string is converted into an interactive character string and a control statement.
The character string is classified into a character string.
Display device described. 3. A screen displaying a combination of a character string and an image is displayed.
In the display apparatus, is input by the character string input operator for operating input an arbitrary character string, said character string input operator is operated
The input character string that is displayed as an interactive character string
And the control string that is converted into an image
Similar means, converting the control character string into an image corresponding to the control character string
Image conversion means, and an interactive character string classified by the character string classification means,
The control character string classified by the character string classification means is
Display a screen consisting of the image converted by the image conversion means
Screen display means for displaying an image pattern data representing an image corresponding to the control character string.
Has a pattern database in which the
The image conversion means refers to the pattern database
To the control character string classified by the character string classifying means.
A corresponding image is generated, wherein the input character string is attached to the control character string,
It includes a transformation instruction parameter for instructing transformation, and
The conversion means is a control character with the transformation instruction parameter.
When converting a row into an image, the pattern database
Image pattern corresponding to the control character string registered in
The data can be transformed based on the transformation instruction parameter.
Viewing device you <br/> characterized in that to generate a new image by the. 4. The pattern database is a deformation instruction.
At least one control string to which parameters can be attached
A plurality of image pattern data is registered corresponding to
Wherein the image conversion means includes a plurality of image patterns.
Convert the control character string with registered data into an image
The pattern database registered in the pattern database
A plurality of image pattern data corresponding to the control character string
By transforming based on the shape designating parameter,
The display device according to claim 3, which is for generating an image . 5. A screen displaying a combination of a character string and an image is displayed.
In the display apparatus, is input by the character string input operator for operating input an arbitrary character string, said character string input operator is operated
The input character string that is displayed as an interactive character string
And the control string that is converted into an image
Similar means, converting the control character string into an image corresponding to the control character string
Image converting means that a dialogue text classified by and the string classifying means, before
The control character string classified by the character string classification means is
Display a screen consisting of the image converted by the image conversion means
And a pattern database in which image pattern data representing an image corresponding to the control character string is registered, and the image conversion means refers to the pattern database to perform the character string classification. all SANYO for generating an image corresponding to the classified control string by means of the plurality of control strings the input string are coupled together
A plurality of the image conversion means are coupled to each other.
When converting the character string to an image, the pattern data
Corresponding to the plurality of control character strings registered in the database
A display device, wherein a new image is generated by combining a plurality of image pattern data .