JP4465730B2 - Dialogue device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an interactive apparatus, and more particularly to an interactive apparatus in which a user and a computer interact through voice or facial expressions.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, an interactive device that controls a response on the computer side according to a user's emotion is known (Japanese Patent Laid-Open No. 8-339446). This conventional dialogue apparatus is composed of a plurality of input means corresponding to a user's action or action, a feature extraction means for extracting features of an input signal input from the input means, and a plurality of signal features extracted by the feature extraction means. It comprises emotion determination means for determining the user's emotion and response generation means for generating response contents of the computer based on the emotion determined by the emotion determination means.
[0003]
According to this conventional dialogue apparatus, since it can be controlled to respond from the computer side according to the user's emotion, a more familiar dialogue apparatus can be provided.
[0004]
[Problems to be solved by the invention]
However, in the above-described conventional interactive device, the computer only generates a response based only on the user's emotion determination based on the pre-stored feature data, and does not have a response feature reflecting the content of the dialog. In addition, the cause of emotion and the way of appropriate response are not constant depending on the user and the situation at the time. However, in the above-described conventional interactive device, the response is made only by the emotion determined by the characteristics of the superficial input signal. Therefore, it has not been able to respond appropriately to these situations.
[0005]
The present invention has been made in view of the above points. In addition to determining a user's emotion and generating a response, the present invention performs an appropriate response according to the personal information of each user, the situation at that time, and the content of the dialogue. An object is to provide an interactive device to obtain.
[0006]
[Means for Solving the Problems]
Since the present invention is to achieve the above object, the information about the voice generated by the User chromatography The, information about images captured user as input information at least one of the character information generated by a user operation input means for inputting a feature extraction means for extracting features of the input information, among the input information, an input sentence analyzing means for analyzing the input information includes language information, extracts a semantic structure, generally Knowledge information storage means for storing data relating to various knowledge and common sense, information retrieval / communication means for acquiring desired information by selecting and connecting an information server that provides desired latest information in a distributed environment, and a user and personal information storage unit that stores personal information the, the matter to humans causing any emotional inference rule to associate the emotion to occur in humans by the matter is inferred And emotional inference rule storing means that stores a response pattern storage means for holding a plurality of response patterns, the semantic structure of the input Chikarajo report analyzed by the input sentence analyzing unit, requesting explicitly Information If there is content, the response information for the request is obtained from the information server via the information search / communication means based on the requested information, the user's personal information, and general knowledge / common sense data. and, the user's emotions are expected to occur by the acquired reply information, and personal information of the reply information and the user, the response pattern was estimated based on the emotional inference rule, matching this estimated emotions selects from among the plurality of response patterns, using the selected response pattern, and interaction means for creating a semantic structure of a response sentence to the user, created by interaction means A response sentence generation means for generating an answering sentence based on taste structure, a response characteristic storage means for storing a plurality of response characteristics corresponding to a plurality of response patterns, corresponding to the response pattern selected by interaction means Response features are acquired from the response feature storage means , and based on the acquired response features and response sentence , response generation means for generating response contents, and output means for outputting the response contents generated by the response generation means It is characterized by having .
[0007]
In order to achieve the above object, the present invention provides at least one of information related to a user's voice, information related to a video image of a user, and character information generated by a user operation as input information. As input means, feature extraction means for extracting features of the input information, features extracted by the feature extraction means, and emotion determination means for determining a user's emotion from pre-stored emotion determination data , Input sentence analysis means for analyzing input information including linguistic information and extracting semantic structures, knowledge information storage means for storing data related to general knowledge and common sense, and distributed and desired information server to provide the latest information selection and connection to acquire desired information information retrieval and communication means in the environment, individuals containing information about the preference of the user Personal information storage means for storing information, emotion inference rule storage means for storing inference rules that associate a matter that causes human beings with arbitrary emotions, and emotions that are inferred to occur in humans due to such matters If the semantic structure of the input information analyzed by the response pattern storage means that holds a plurality of response patterns and the input sentence analysis means is content that does not explicitly request information , and information on at least preferences out, based on at least one of the information in the reply information acquired from the information server via the data and information retrieval and communication means on general knowledge and common sense, determined by emotion determination means The cause of the user's feelings is identified, and based on the identified cause, the emotions to be caused next by the user are estimated. A dialogue processing unit that selects a response pattern that matches the sent emotion from a plurality of response patterns and creates a semantic structure of a response sentence to the user using the selected response pattern; Response sentence generation means for generating a response sentence based on the created semantic structure, response feature storage means for storing a plurality of response features corresponding to a plurality of response patterns, and a response pattern selected by the dialogue processing means Is obtained from the response feature storage means, and based on the obtained response feature and response sentence, the response generation means for generating the response content, and the output for outputting the response content generated by the response generation means Means.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a block diagram of an embodiment of an interactive apparatus according to the present invention. As shown in the figure, this embodiment includes an input unit 1 corresponding to a plurality of inputs from a user such as a voice and a face image, a feature extraction unit 2 that extracts features from input signals of each input unit 1, Emotion feature storage means 3 for storing features corresponding to the emotion of the input signal, emotion determination means 4 for determining the user's emotion from the features of the signal obtained from the user's input based on the data stored in the emotion feature storage means 3 The response feature storage means 5 for storing output data for creating a response according to emotion, the response generation means 6 for generating response contents, and the output means 7 for outputting the generated data are the same as the conventional dialogue apparatus. It is set as the same structure.
[0009]
In the present embodiment, in addition to the configuration of these conventional devices, an input sentence analysis unit 8 that analyzes word information input from the input unit 1 and extracts a semantic structure, a personal information storage unit 9 that stores user personal information, The information server 10 and the information retrieval / communication means 11 for retrieving / acquiring information related to the topic, the information obtaining means 13 comprising the knowledge base 12, and the emotion for storing the inference rules for associating the objective matter with the user's emotion From the inference rule storage means 14, the response pattern storage means 15 that holds the index of the response pattern of the computer, the dialog history storage means 16 that stores the history of the dialog, the semantic structure of the input, the user's emotion, and the information acquisition means 13 An interactive processing method that selects the appropriate response pattern by referring to the acquired related information, personal information, and emotion inference rules, and creates the semantic structure of the response sentence. 17, with the addition of a structure consisting of the output sentence generating means 18 for generating a word output from the semantic structure.
[0010]
The personal information storage means 9 stores a personal profile 9a. As shown in another embodiment, the schedule information 9b can also be stored. Applications of the interactive apparatus shown in the present embodiment include those that play secretary roles such as schedule management, information retrieval / collection, and various information application apparatuses, but can also be used for other applications.
[0011]
The input unit 1 can include a plurality of input units corresponding to a plurality of inputs from the user such as voice, face image, sign language, and the like. The feature extraction unit 2 extracts a feature from the input signal of each input unit 1. Examples of this feature include features related to facial expression of facial images, voice pitch, and the like. The emotion feature storage means 3 stores a feature corresponding to the emotion of each input signal as data for determining the emotion. The emotion determination unit 4 determines the user's emotion from the features of the input signal obtained from the user's input based on the data of the emotion feature storage unit 3.
[0012]
The input sentence analysis means 8 analyzes information input as words, such as speech utterances, character input with a keyboard or pen, and sign language input, and creates an input semantic structure. The part omitted during the conversation is created by referring to the dialog history storage means 16 and supplementing from the context.
[0013]
The personal information storage means 9 stores personal information obtained in the course of dialogue for each user. The user profile is stored in the personal profile 9a. In addition to name, address, age, gender, family composition, affiliation, and status, information on hobbies and preferences is also accumulated through dialogue.
[0014]
The emotion inference rule storage means 14 stores inference rules for causal relations related to emotions. For example, as shown in FIG. 2, the inference rule is an emotion “joy” (pattern 1) inferred on the condition that a favorite object / person gains (success), and the favorite object / person suffers a loss ( Emotion “disappointment” inferred under the condition of “failure” (Pattern 2), disliked feelings “Anger” (pattern 3) inferred that the person gains profit (success) (pattern 3) There is an emotion “satisfaction” (pattern 4) inferred on the condition that a person suffers a loss (fails). The amplitude of emotion and the degree of preference vary among individuals, but these use information in the personal profile 9a. The degree to which a certain event meets a condition is evaluated with reference to information in the knowledge base 12.
[0015]
The knowledge base 12 is a database relating to general knowledge and common sense. The information server 10 is a server that provides information in a distributed environment. It is conceivable to use a plurality of information servers 10 depending on the purpose. Examples of the information server 10 include a database in a specific field, multimedia information such as music / video, and the latest information supply such as news / weather forecast. The information search / communication means 11 receives and receives a search request from the dialog processing means 17 to select / connect the information server 10 to search / acquire information.
[0016]
The response pattern storage means 15 stores the response pattern of the computer. As the response pattern, response contents, wording, and indexes of various response features can be selected. The dialogue history storage means 16 is a means for accumulating the history of dialogue between the user and the computer, and even if abbreviations are used in the spoken language of the user, an appropriate input is made with reference to the dialogue history to supplement the meaning. Required to create a semantic structure. The dialogue history storage means 16 is referred to by the dialogue processing means 17 in order to create the next response based on the flow of the dialogue.
[0017]
The dialog processing means 17 determines the semantic structure and response pattern of the response output based on the input semantic structure and emotion determination, and extracts information necessary for the response. If the input semantic structure explicitly requests information, the information is acquired by referring to the knowledge base 12, the information server 10, and the personal information storage means 9. The acquired information is applied to the emotion inference rule, and the user's emotion based on this information is estimated together with the personal preference in the personal profile 9a. An appropriate response pattern is selected based on this expected user emotion and the current user emotion, and an output semantic structure is created.
[0018]
If the input semantic structure is not an explicit information request, if the input semantic structure indicates the user's personal information, the dialogue processing means 17 stores the personal information in the personal information storage means 9. Based on the input semantic structure and emotion determination, the personal profile 9a and emotion inference rules are used to obtain relevant information from the knowledge base 12 or the information server 10 as necessary to estimate the cause of the user's emotion. The dialogue processing means 17 estimates the user's emotions based on the acquired information and the response inference rules from the personal preferences and emotion inference rules in the personal profile 9a, selects the output information and response pattern, and creates an output semantic structure.
[0019]
The output sentence generation unit 18 generates a response sentence based on the output semantic structure obtained from the dialogue processing unit 17. The response sentence is expressed by a method desired by the user among text, voice, sign language, and the like. The response feature storage means 5 stores response feature data corresponding to the response pattern. The characteristics of the response include the character's facial expression and gestures on the response screen, and the tone of the response voice. The response generation unit 6 extracts response feature data from the response feature storage unit 5 based on the designated response pattern, and generates a response output. The output means 7 outputs a response to a monitor screen / speaker or the like.
[0020]
Next, a simple example of response generation according to the embodiment of the present invention will be described with reference to professional baseball information. Actual emotion classifications and inference rules are more fragmented and complicated, but a simple example is given for explanation. The emotion inference rule storage means 14 stores the inference rules shown in FIG. It is assumed that the personal profile 9a of the user A stores the user's preferences (in this example, a favorite professional baseball team name and a favorite player name) as shown in FIG.
[0021]
The knowledge base 12 stores general knowledge about baseball and semi-fixed knowledge. The information server 10 is updated with the latest information such as the game results and highlight scenes of the day. Information such as player registration and game schedule is retrieved from the information server 10 as necessary, and the knowledge base 12 is updated.
[0022]
When the user A asks the dialog device “What is today's game result?”, The computer acquires the game result from the information server 10. When the result of “Yokohama 5-1 Giant” is obtained, it is determined that “the user's favorite team has been lost” and matches the inference pattern 2 shown in FIG. Expected to be discouraged (estimates the user's emotion), selects information and response pattern to be output according to the predicted emotion, creates an output semantic structure, and responds from the response feature storage means 5 based on the specified response pattern The characteristic data is taken out and a response output is generated by the response generation means 6, and the response is output to the monitor screen / speaker by the output means 7. As the response pattern, a pattern of unfortunately answering “Sorry I lost.” Is selected.
[0023]
On the other hand, when the user A “has lost again”, it is assumed that the user's emotion is determined to be discouraged from the voice / expression. In this case, when the game result is acquired from the information server 10 and the personal profile 9a is referred to, the cause of the discouragement of the user matches the inference pattern 2 because the favorite team has lost. The interaction processing means 17 of the computer searches the information that matches the inference pattern 1 that the user seems to be happy with as a response pattern, and causes the response generation means 6 to create a response. In this case, “But Matsui is the top player in the home run” is selected, and a pattern that answers brightly is selected.
[0024]
As described above, according to this embodiment, personal information of each user is accumulated, and from the accumulated information and information from the knowledge base 12 and the information server 10, each user's preference or An appropriate response can be created by estimating the cause of the emotion according to the situation and topic at that time and predicting the user's emotion to the response of the computer.
[0025]
Next, another embodiment of the present invention will be described. In another embodiment, the personal information storage means 9 stores schedule information 9b in addition to the personal profile 9a. The schedule information 9b stores the user's schedule. If the input semantic structure indicates the user's schedule in the dialog with the user, the information is stored in the schedule information 9b. When estimating the cause of the user's emotion by using the personal profile 9a and emotion inference rules based on the input semantic structure and emotion judgment, and extracting relevant information from the knowledge base 12 or the information server 10 as necessary. The user's schedule is also referred to from the information 9b, and the cause of the emotion is estimated. In this case, the cause of the emotion is estimated not only according to each user's preference, the situation and topic, but also the schedule, and the user's emotion to the computer response is predicted to create an appropriate response. Can do.
[0026]
【The invention's effect】
As described above, according to the present invention, the user's emotion is determined based on the input information from the input means, the personal information of each user is accumulated, the accumulated information, the information from the information acquisition means, From the above, use the reasoning rules to estimate the cause of emotion according to each user's preference, occasional situation and topic, predict the user's emotion to the response, and create and output an appropriate response As a result, not only the user's emotion but also the user's preference, the situation at the time, the topic, and the topic that are the cause of the emotion, the optimum dialogue can be adapted to each user, and the dialogue is more familiar than before. be able to.
[Brief description of the drawings]
FIG. 1 is a block diagram of an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of an inference rule.
FIG. 3 is a diagram showing an example of a personal profile.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Input means 2 Feature extraction means 3 Emotion feature storage means 4 Emotion determination means 5 Response feature storage means 6 Response generation means 7 Output means 8 Input sentence analysis means 9 Personal information storage means 9a Personal profile 9b Schedule information 10 Information server 11 Information retrieval Communication means 12 Knowledge base 13 Information acquisition means 14 Emotion reasoning rule storage means 15 Response pattern storage means 16 Dialog history storage means 17 Dialog processing means 18 Output sentence generation means

Claims (2)

An input means for inputting as input information at least one of information related to a voice uttered by the user, information related to a video obtained by photographing the user, and character information generated by a user operation;
Feature extraction means for extracting features of the input information;
Among the input information, input sentence analysis means for analyzing input information including language information and extracting a semantic structure;
Knowledge information storage means for storing data relating to general knowledge and common sense;
An information search / communication means for selecting / connecting an information server that provides the desired latest information in a distributed environment and acquiring the desired information; and
Personal information storage means for storing user personal information;
An emotion inference rule storage means for storing an inference rule that associates an event that causes a human being with any emotion and an emotion that is inferred to occur in the human by the event;
Response pattern storage means for holding a plurality of response patterns;
When the semantic structure of the input information analyzed by the input sentence analyzing means is the content explicitly requesting information,
Based on the requested information, the personal information of the user, and the data related to the general knowledge / common sense, obtain reply information to the request from the information server via the information search / communication means,
Estimating the user's emotions expected to occur with the acquired reply information based on the reply information, the user's personal information, and the emotion inference rules,
A dialogue processing means for selecting a response pattern matching the estimated emotion from the plurality of response patterns, and using the selected response pattern to create a semantic structure of a response sentence to the user;
Response sentence generation means for generating the response sentence based on the semantic structure created by the dialogue processing means;
Response feature storage means for storing a plurality of response features corresponding to the plurality of response patterns;
A response generation unit that acquires a response feature corresponding to the response pattern selected by the dialog processing unit from the response feature storage unit, and generates response content based on the acquired response feature and the response sentence;
And an output means for outputting the response content generated by the response generation means. An input means for inputting as input information at least one of information related to a voice uttered by the user, information related to a video obtained by photographing the user, and character information generated by a user operation;
Feature extraction means for extracting features of the input information;
Emotion determination means for determining the emotion of the user from the features extracted by the feature extraction means and pre-stored emotion determination data;
Among the input information, input sentence analysis means for analyzing input information including language information and extracting a semantic structure;
Knowledge information storage means for storing data relating to general knowledge and common sense;
An information search / communication means for selecting / connecting an information server that provides the desired latest information in a distributed environment and acquiring the desired information; and
Personal information storage means for storing personal information including information relating to user preferences ;
An emotion inference rule storage means for storing an inference rule that associates an event that causes a human being with any emotion and an emotion that is inferred to occur in the human by the event;
Response pattern storage means for holding a plurality of response patterns;
When the semantic structure of the input information analyzed by the input sentence analysis means is content that does not explicitly request information,
At least the information about the preference, at least one of the information in the reply information acquired from the information server via the data and the information retrieval and communication means for said common knowledge and common sense of the personal information of the user based on the bets, as well as identify the cause that the user has caused the determined emotion in the emotion determining means, based on the identified cause, then estimates the emotion to cause the user is the estimated A dialogue processing means for selecting a response pattern that matches an emotion from the plurality of response patterns, and using the selected response pattern to create a semantic structure of a response sentence to the user;
Response sentence generation means for generating the response sentence based on the semantic structure created by the dialogue processing means;
Response feature storage means for storing a plurality of response features corresponding to the plurality of response patterns;
A response generation unit that acquires a response feature corresponding to the response pattern selected by the dialog processing unit from the response feature storage unit, and generates response content based on the acquired response feature and the response sentence;
And an output means for outputting the response content generated by the response generation means.

Images