JP7365791B2 - Utterance generation device, utterance generation method, and utterance generation program - Google Patents

Description

The present invention relates to an apparatus, method, and program for generating a robot's utterances.

Conventionally, as a means for people to enjoy conversations with computers, for example, artificial brainless machines (chatbots, conversation bots) have been developed. Artificial incompetence employs a method of mechanically generating plausible response sentences from human utterances. Artificial brainlessness can be implemented using a dictionary type, in which utterances and responses are prepared as conversation rules, and a log type, in which a large amount of conversation logs is searched for the one closest to the user's utterance, and the response log is used. , a Markov sentence generation type that generates a sentence by creating a Markov chain from a large amount of sentences (for example, see Non-Patent Documents 1 and 2).

In recent years, communication robots with the ability to interact with humans have been developed by incorporating artificial brainlessness. A software program, or communication robot, incorporating an artificial brain generates plausible response sentences using words contained in the user's utterances.

Japanese Patent Application Publication No. 2018-185561 Japanese Patent Application Publication No. 2018-190077

Ryota Tomisaka, Takashi Suzuki, "Artificial brainlessness (conversation bot)", Journal of the Institute of Image Information and Media Engineers, Vol. 64, No. 1, pp. 64-66 (2010) Tomotoshi Akiyama, “Koisuru Program – Artificial Brain Created with Ruby”, Mainichi Communications, 2005

By the way, in daily conversations between humans, there are often contexts and topics that are mutually recognized and shared, although they are not explicitly stated in the utterances. Particularly while watching a television program, the conversation proceeds using the content of the television program as a common context, so the name of what is shown on the screen is often not explicitly included in the utterance.
However, the problem with general artificial brainlessness is that it does not take this kind of context into account. This causes responses that do not fit the conversation, and is one of the reasons why users lose interest in communication robots.

On the other hand, Patent Document 1 proposes a technique for generating a response sentence that understands the user's intention by classifying the user's utterances into topics stored in a storage unit. Further, Patent Document 2 proposes a technology that activates communication by extracting keywords from television programs and combining them with template sentences to make spontaneous utterances.
However, since the technique of Patent Document 1 compares the user's utterance with the topic stored in the storage unit, it cannot take into account the context that is not included in the user's utterance. Further, the technique disclosed in Patent Document 2 generates a spontaneous utterance from keywords extracted from a television program and a template sentence, and therefore cannot generate a response sentence to a user's utterance.

An object of the present invention is to provide an utterance generation device, an utterance generation method, and an utterance generation program that can estimate an unspecified context for a user's utterance and generate an utterance as a response.

The utterance generation device according to the present invention includes a keyword extraction unit that extracts keywords registered in a keyword database from an input sentence, stores the extracted keywords in a keyword memory, and stores the extracted keywords in a keyword memory, and a keyword management section that deletes a keyword from the keyword memory; a representative word search section that searches for a representative word included in a user's utterance; and a feature vector of the representative word and each of the keywords stored in the keyword memory. a target word estimating unit that determines the keyword to be a target word of the uttered sentence based on a distance from a feature vector; a uttered sentence generating unit that generates a response uttered sentence based on the target word and the representative word; Equipped with

The utterance generation unit may generate a response utterance by combining the target word with a template corresponding to the representative word.

The utterance generation unit may access the artificial incompetence application using parameters including the target word and the representative word, and obtain the response utterance.

The input sentence may be extracted from video or audio of a television program.

The input sentences are utterances of a plurality of users, and the target word estimation unit determines the target word from among keywords extracted from the utterances of users different from the user who extracted the representative word. It's okay.

The utterance generation method according to the present invention includes a keyword extraction step of extracting keywords registered in a keyword database from an input sentence, and storing the extracted keywords in a keyword memory. a keyword management step of deleting a keyword from the keyword memory; a representative word search step of searching for a representative word included in a user's utterance; and a feature vector of the representative word and each of the keywords stored in the keyword memory. a target word estimation step of determining the keyword to be the target word of the utterance based on the distance from the feature vector; a utterance generation step of generating a response utterance based on the target word and the representative word; is executed by the computer.

The utterance generation program according to the present invention is for causing a computer to function as the utterance generation device.

According to the present invention, an unspecified context can be estimated for a user's utterance, and an utterance can be generated as a response.

FIG. 2 is a block diagram showing the functional configuration of the speech generation device in the embodiment. It is a figure which illustrates the data stored in the keyword memory in embodiment. It is a figure which shows the example of registration of the representative word database in embodiment. It is a figure which shows the example of registration of the feature vector database in embodiment. It is a figure which shows the example of registration of the template database in embodiment. It is a flowchart which shows keyword processing in an embodiment. It is a figure which illustrates the operational image of the target word estimation part in embodiment. It is a flowchart which shows response generation processing in an embodiment. FIG. 7 is a diagram illustrating an example of generation of a response utterance using artificial brainlessness in the embodiment. FIG. 6 is a diagram illustrating an example of generation of response utterances in dialogue with multiple users in the embodiment.

A first embodiment of the present invention will be described below.
The speech generation device 1 in this embodiment generates phrases that include verbs expressing requests such as "I want to eat,""I want to talk," and "I want to go," or adjectives such as "beautiful,""tasty," and "big." Words and phrases containing these words are defined as representative words, and template sentences containing these representative words, feature vectors of the representative words, and feature vectors of words included in the keyword database are created in advance.

The utterance generation device 1 retains the keywords of the program being viewed for a certain period of time, and when a user utterance containing a representative word is input, it compares the feature vector of this representative word with the feature vector of the retained keywords. do. Then, the utterance generation device 1 generates a response utterance by combining keywords with similar feature vectors with the template sentence for the representative word.
Thereby, the utterance generation device 1 can generate an appropriate response utterance after estimating the target word, which is a context that is not specified in the user utterance. For example, if the keywords ``ramen'', ``donburi'', ``astragalus'', and ``hakata'' are stored, and the user says ``I want to eat this,'' then the representative word ``I want to eat this'' is used. "Ramen," which has a similar feature vector, is selected as the target word, and by combining it with the template sentence, a response sentence such as "I want to eat ramen, too" is generated.

[overall structure]
FIG. 1 is a block diagram showing the functional configuration of the speech generation device 1 in this embodiment.
The utterance generation device 1 is an information processing device equipped with various interfaces in addition to a control section and a storage section. Various functions are realized.

The storage unit of the utterance generation device 1 includes a keyword database 11 , a keyword memory 12 , a representative word database 13 , a feature vector database 14 , and a template database 15 .
The control unit of the utterance generation device 1 also includes a representative word search unit 21, a target word estimation unit 22, and an utterance generation unit 23 as a response generation processing block 20, and a keyword extraction unit as a keyword processing block 30. 31, and a keyword management section 32.

In this embodiment, the speech generation device 1 is installed inside a communication robot placed near a television.
Then, the user utterance U and the television output sentence T are input to the utterance generating device 1, and the response utterance R is output.

[User utterance U]
The user utterance U is a text obtained by extracting only the voice uttered by the user from the input voice Vi input to the voice input device 40. Note that the voice input device 40 is built into the communication robot and is configured by a microphone array including a plurality of microphones.

Since the communication robot is placed near the television, the input voice Vi input to the voice input device 40 is a synthesized sound from multiple sound sources including the user's uttered voice and the voice output from the television. is assumed.
Therefore, the direction of each input sound is detected and the synthesized sound is separated into sounds for each sound source direction by the sound source localization/separation unit 41. Subsequently, when only the user's uttered voice is extracted from the separated voices, the user's uttered sentence U converted into text is obtained by the voice recognition unit 42.

For the sound source localization/separation unit 41, for example, the method described in the following document A can be used, and for the speech recognition unit 42, for example, the method described in the following document B can be used, but the implementation method is not limited to these.
Literature A: Kazuhiro Nakadai, Hiroshi Okuno, Hiroshi Nakajima, Yuji Hasegawa, Hiroshi Tsujino, "Summary and evaluation of robot hearing open source software HARK", 26th Academic Conference of the Robotics Society of Japan, 2008
Literature B: Takahiro Shinozaki, “Construction of a speech recognition system using Kaldi toolkit”, IEICE Technical Report 116 (279), pp25-29, 2016

Note that in order to remove the audio output from the TV and extract only the user's uttered audio, the direction of the TV and the user is required. A method of acquiring an image from a computer and identifying the direction of the television and the user through image processing, a method of estimating the direction of the television and the user from the frequency of voice input, etc. can be used.

[TV output text T]
The television output sentence T may be a subtitle sentence (closed caption) added to a television broadcast program. Methods for extracting subtitles from broadcast programs include, for example, a robot working in conjunction with a television receiver to obtain subtitle information, or a broadcast-linked service that uses communication equipment such as the Internet to extract subtitle information. One possible method is to receive the information.
Furthermore, the television output sentences T are not limited to subtitle sentences; for example, sentences may be extracted from the television audio by voice recognition using a microphone or camera equipped on a robot, or the sentences may be extracted from the television video by voice recognition. Open captions may be extracted from the image by image processing. Alternatively, the name or explanatory text of the subject may be generated using existing image recognition technology.

[Response utterance R]
The response utterance R is converted into speech by the speech synthesis unit 51, and the speech is output from the speech output device 50 constituted by a speaker.
Although the method for realizing the speech synthesis unit 51 is not limited, for example, the method described in the following document C can be used.
Document C: Keiichiro Oura, "Japanese text-to-speech synthesis system Open JTalk", Acoustical Society of Japan Spring Proceedings, 2010 1, pp343-344, 2010

[Keyword database 11]
The keyword database 11 is a database in which keywords are registered. Here, the keywords are mainly proper or common nouns such as people's names, place names, and facilities. In this embodiment, it is assumed that words (keywords) that are desired to be extracted from the television output sentence T as target word (context) candidates are registered in the keyword database 11 in advance.

[Keyword memory 12]
The keyword memory 12 is a memory for holding keywords extracted from the television output sentence T for a certain period of time.

FIG. 2 is a diagram illustrating data stored in the keyword memory 12 in this embodiment.
In this example, the keyword memory 12 stores four keywords: "ramen,""donburi,""renge," and "hakata," and the date and time each keyword was stored is also associated with each keyword. It is preserved.

Keywords that have been stored in the keyword memory 12 for a certain period of time are deleted by the keyword management unit 32. The certain period here may be set to any desired period of time, but it is generally set to about 15 seconds, taking into account the time it takes for a human to react to a certain topic and the processing delay time.
Note that when the utterance generation device 1 starts operating, the keyword memory 12 is in a state where no keywords are stored.

[Representative word database 13]
In the representative word database 13, words including verbs and adjectives expressing emotions are registered as representative words indicating characteristics of utterances, and other expressions similar in meaning to these representative words are also registered.

FIG. 3 is a diagram showing an example of registration in the representative word database 13 in this embodiment.
In this example, in order to uniformly handle different expressions with the same or similar meaning, data in three fields: type, representative word, and other expressions are stored as a list.

The type represents the type (part of speech) of the representative word; for example, v represents a verb, and a represents an adjective.
Representative words are words that include verbs or adjectives that express emotions. For example, "I want to talk", "I want to hear", and "I want to eat" are registered as representative words that include verbs, and "I want to eat" are registered as representative words that include adjectives. "Beautiful", "Interesting", and "I like" are registered.

In the field of different expressions, different expressions having the same or similar meaning as the representative word are registered.
For example, "I want to talk" can be expressed as "I want to talk", "I want to talk", "I want to talk", "I want to say", "I want to say", and "I want to listen" can be expressed as "I want to talk", "I want to talk", "I want to talk", "I want to say", "I want to say", and "I want to listen" can be expressed as "I want to talk""I want to eat", "I want to hear", and "I want to listen" are registered as alternative expressions of "I want to eat", "I want to eat", "I want to eat", and "I want to eat".
Furthermore, "beautiful" is registered as an alternative expression for "pretty,""interesting" is an alternative expression for "interesting," and "I love" is an alternative expression for "like."

Here, in the representative word database 13, hiragana notation and kanji notation are registered as different words. For example, the notation used for subtitle sentences differs depending on the broadcasting station or program, but it can be matched with any notation.
In this embodiment, it is assumed that representative words and alternative expressions are registered in advance in the representative word database 13 as well as in the keyword database 11.

[Feature vector database 14]
In the feature vector database 14, words or phrases including representative words and other keywords, and feature vectors that are distributed expressions corresponding to each word are registered.

In this embodiment, a feature vector calculated by word2vec, which is a known method, is used as the feature vector. word2vec uses a three-layer neural network, and it is known empirically that the feature vectors calculated by inputting a large amount of separated sentences are vectors in which similar words are close in distance. There is.
Note that the feature vector calculation method is not limited to word2vec, and any calculation method may be used as long as the feature vectors of words that are semantically similar are distributed nearby on the vector space.

FIG. 4 is a diagram showing an example of registration in the feature vector database 14 in this embodiment.
In this example, vec1 is registered as the feature vector for "ramen", and hereafter, as the feature vectors for "donburi", "renge", "hakata", "want to eat", "want to listen", and "want to talk", respectively. vec2 to vec7 are registered, and a total of N feature vectors are registered.
The feature vector database 14 must register feature vectors learned in advance by inputting a large amount of sentences, and must include feature vectors of words included in the keyword database 11 and the representative word database 13.

[Template database 15]
In the template database 15, templates of utterances for representative words are registered. The representative word is the same as the word registered in the representative word field of the representative word database 13.

FIG. 5 is a diagram showing an example of registration in the template database 15 in this embodiment.
In this example, template sentences such as "I want to use %key" and "I will try using %key next time" are registered in the template database 15 for the representative word "I want to use". Similarly, template sentences corresponding to the representative words "want to drink,""want to eat,""beautiful,""interesting," and "like" are registered.
Here, %key indicates a location to be replaced with some keyword.
Note that the data in the template database 15 is prepared and registered in advance.

The utterance generation process by the utterance generation device 1 utilizes these various databases and is comprised of processing of each functional block, the response generation processing block 20 and the keyword processing block 30. These two processes may be performed in parallel.

[Keyword processing]
Keyword processing is executed by a keyword extraction section 31 and a keyword management section 32 that constitute the keyword processing block 30.

The keyword extracting unit 31 searches whether a keyword registered in the keyword database 11 is included in the television output sentence T (the input sentence to the speech generation device 1). If the keyword is not included, the keyword extraction unit 31 waits until the next television output sentence T is input. If a keyword is included, the keyword extraction unit 31 extracts the relevant keyword and inputs it to the keyword management unit 32.
Note that if the television output sentence T includes a plurality of keywords, all of these keywords may be input to the keyword management section 32.

The keyword management unit 32 manages keywords temporarily held in the keyword memory 12. Specifically, the keyword management section 32 stores the keyword input from the keyword extraction section 31 in the keyword memory 12 together with time information. Further, the keyword management unit 32 sets a keyword retention time in advance when the utterance generation device 1 starts operating, and deletes the keyword from the keyword memory 12 for which the keyword retention time has elapsed since the keyword was stored in the keyword memory 12.
As a result, the keyword memory 12 retains a keyword that is currently a hot topic in the TV program being viewed for a predetermined period of time (for example, 15 seconds).

FIG. 6 is a flowchart showing keyword processing in this embodiment.
In step S1, the keyword extraction unit 31 receives the television output sentence T as an input text to the utterance generation device 1.

In step S2, the keyword extraction unit 31 determines whether the received television output sentence T includes a keyword registered in the keyword database 11. If this determination is YES, the process moves to step S3, and if the determination is NO, the process ends.

In step S3, the keyword management unit 32 stores the keyword included in the television output sentence T in the keyword memory 12 together with time information.
Note that, apart from this process, the stored keywords are deleted from the keyword memory 12 after a predetermined keyword retention time has elapsed.

[Response generation processing]
The response generation process is executed by the representative word search unit 21 , target word estimation unit 22 , and uttered sentence generation unit 23 that constitute the response generation process block 20 .

The representative word search unit 21 searches whether the input user utterance U includes a representative word stored in the representative word database 13 or another expression thereof. When the representative word in the representative word database 13 and its alternative expression are included in the user utterance U, the representative word search unit 21 searches the user utterance U and the representative word contained therein in the target word estimation unit 22. Enter.
Note that if the user utterance sentence U includes a plurality of representative words, the representative word search unit 21 may input all of these to the target word estimation unit 22. If no representative word is included, there is no input to the target word estimating unit 22, and the representative word searching unit 21 waits until the next user utterance U is input.

The target word estimating unit 22 estimates a keyword to be the target word of the user's utterance from the keyword memory 12 based on the user's utterance U and the representative word.
Specifically, first, it is checked whether the user's utterance U includes the keyword stored in the keyword memory 12. When a keyword is included in the user's utterance U, it is clear that the target of the representative word included in the user's utterance U is the confirmed keyword, and there is no need to estimate it. For example, if the user utterance sentence U is "This ramen looks delicious," it is obvious that the target of the representative word "delicious" is the keyword "ramen."

Note that in this embodiment, the target word estimation unit 22 merely checks whether the user's uttered sentence U contains the keyword stored in the keyword memory 12; By using reception analysis technology, etc., target words can be identified with higher accuracy.

On the other hand, if the user's utterance sentence U does not include the keyword stored in the keyword memory 12, the target word estimating unit 22 estimates the keyword to be the target word. At this time, the target word estimating unit 22 retrieves the characteristic vectors of the representative words extracted by the representative word searching unit 21 and the keywords included in the keyword memory 12 from the feature vector database 14, and calculates the distance between the characteristic vectors. do.
Here, when the feature vector of the keyword k stored in the keyword memory 12 is veck, and the feature vector of the representative word t included in the user's utterance U is vect, the distance between veck and vect is, for example, The following cosine similarity cos(veck,vect) can be used.
cos(veck,vect)=(veck・vect)/(|veck||vect|)

Next, the target word estimation unit 22 ranks the cosine similarity calculation results between the feature vector of the keyword included in the keyword memory 12 and the feature vector of the representative word included in the user utterance U in descending order of the value ( sort). Then, the target word estimating unit 22 determines the keyword having the highest ranking and having a strong relation to the representative word included in the user's utterance sentence U from the keyword memory 12 as the estimation result. The keyword determined as the target word is input to the utterance generation unit 23.

Note that when a plurality of representative words are extracted by the representative word search unit 21, the target word estimation unit 22 calculates the cosine similarity of each of the plurality of representative words with the keywords included in the keyword memory 12, and then All calculation results are ranked (sorted) in descending order of values, and the pair of representative word and keyword with the highest cosine similarity is input to the utterance generation unit 23.

FIG. 7 is a diagram illustrating an operational image of the target word estimation unit 22 in this embodiment.
In this example, the keyword memory 12 stores four keywords: "ramen", "donburi", "renge", and "hakata", and their respective feature vectors are stored in the feature vector database 14 as vec1 and vec2. , vec3, and vec4. At this time, consider a case where the user's utterance U includes the representative word "I want to eat" (feature vector vec5).

In the feature vectors of this embodiment, feature vectors of words with similar tendencies are arranged close to each other, for example, "udon", "curry", "yakisoba", etc. are distributed near "ramen". Along with this, the feature vectors of related words also become closer. In other words, the feature vector of a food such as "Ramen" is distributed close to the feature vector of "I want to eat", while the feature vector of a place name such as "Hakata" is distributed close to the feature vector of "I want to eat". are distributed far away. As a result, among the keywords included in the keyword memory 12, the keyword "ramen" has the highest cosine similarity to the representative word "eatai", and the target word of the representative word "eatai" is the keyword "ramen". It is presumed to be "Ramen".

The utterance generation unit 23 randomly selects a template from among the templates registered in the template database 15, corresponding to the representative word included in the user utterance U. Then, the utterance generation unit 23 generates an utterance by replacing the “%key” part in the selected template with the keyword estimated as the target word by the target word estimation unit 22.

For example, assume that the representative word included in the user's utterance U is "I want to eat" and "Ramen" in the keyword memory 12 is estimated as the target word. In this case, when the first registered template for the representative word "I want to eat" is selected from the template database 15 in FIG. ”.
The utterance generation device 1 outputs the utterance generated in this way as a response utterance R.

In the present embodiment, the utterance generation unit 23 has been described as randomly selecting a plurality of registered templates for the representative word, but the present invention is not limited to this. For example, a specific template may be weighted to make it easier to select, giving a biased selection probability, or this selection probability may be updated as appropriate in relation to the program being viewed to give individuality to the utterance. It's okay.

FIG. 8 is a flowchart showing response generation processing in this embodiment.
In step S11, the representative word search unit 21 receives the user's utterance U.

In step S12, the representative word search unit 21 determines whether the user uttered sentence U includes the representative word stored in the representative word database 13 or another expression thereof. If this determination is YES, the process moves to step S13, and if the determination is NO, the process ends.

In step S13, the target word estimation unit 22 determines whether the user utterance U includes the keyword stored in the keyword memory 12. If this determination is YES, the keyword included in the user's utterance sentence U is set as the target word, and the process moves to step S15. On the other hand, if the determination is NO, the process moves to step S14.

In step S14, the target word estimation unit 22 calculates the distance between the feature vector of the representative word and the feature vector for each keyword stored in the keyword memory 12, and determines the keyword with the closest distance as the target word. .

In step S15, the utterance generation unit 23 selects a template corresponding to the representative word.

In step S16, the utterance generation unit 23 combines the template with the target word, generates and outputs a response utterance R.

According to the present embodiment, the utterance generation device 1 extracts a representative word from the user's uttered sentence U and a keyword from the TV output sentence T, and extracts keywords that are close to the feature vector of the representative word and used within a predetermined time. A response utterance R is generated by combining the keywords found as target words.
As a result, the utterance generation device 1 generates utterances uttered by the user while watching a TV program that express emotions and do not explicitly include the target context, such as utterances uttered in a scene where ramen noodles are served. In response to an utterance such as ``I want to eat this,'' an utterance such as ``I would like to try this ramen'' or ``I like miso ramen'' is uttered after inferring the unspecified context. Can generate sentences.

Therefore, by incorporating the utterance generation device 1 into a communication robot, a communication robot that can estimate a target word that is not explicitly stated in a user's utterance and interact with a person using a response sentence that includes the target word is realized. As a result, the speech generation device 1 can be expected to prevent the user from giving a response that does not fit the conversation, and to help the user develop an attachment to the communication robot.

Furthermore, the utterance generation device 1 can easily generate the response utterance R by combining the target word with the template corresponding to the representative word.

Further, by extracting the keyword from the video or audio of the television program, the utterance generating device 1 can generate an appropriate response utterance R in consideration of the context to the utterance of the user while watching the television.

[Second embodiment]
A second embodiment of the present invention will be described below.
In this embodiment, the function of the utterance generation unit 23 is different from that in the first embodiment, and the response utterance R is generated using external artificial incompetence.

The utterance generation unit 23 does not use the template database 15, but accesses an external artificial mind through the dialogue API calling unit 231.
Here, the artificial brain is composed of a program that automatically carries out conversations through text or voice. Many of these artificial mindless machines are able to gather information on the Internet and carry out conversations about current topics that are being talked about on social networking sites.
The artificial brain responds to sentences sent by the user. For example, when a user sends a message saying, ``I want to eat ramen,'' the artificial mindless brain is expected to respond with something like ``I like miso ramen,'' or ``It seems like ○○'s ramen is popular these days.'' be done.

FIG. 9 is a diagram showing an example of generation of a response utterance R using artificial brainlessness in this embodiment.
In this example, for the user's uttered sentence U, "I want to eat this," the representative word search unit 21 extracts "I want to eat" as the representative word, and the target word estimation unit 22 estimates "ramen" as the target word. The case is shown below.

The sentence ``I want to eat ramen'' can be created by combining the representative word ``I want to eat'' and the target word ``Ramen.'' This sentence is a sentence in which a target word that was not specified in the user's uttered sentence U is added. By sending this sentence to the artificial brain, the artificial brain will respond with a sentence such as ``I like miso ramen.'' The utterance generation unit 23 can realize utterance generation without using the template database 15 by outputting the response sentence from the artificial brain as the response utterance R.
Here, it is assumed that a sentence is input as a parameter to the artificial incompetence. For this reason, a template for creating a sentence from the representative word and the target word may be prepared in advance, or the template database 15 may be used. Alternatively, in the case of an artificial incompetence that can accept words, the representative word and the target word may be input as parameters as they are.

Note that when generating the response utterance R, it is also possible to use a method using the template database 15 and a method using artificial brainlessness in combination. The uttered sentence generation unit 23 may randomly output either of the sentences generated using these two methods.
It is also possible to use multiple artificial brains, but artificial brains use the first person (me, me, etc.) and word endings (desu, dayo, etc.) that match their personality. There are cases. In this case, it is preferable that the response utterance R is formed by formatting the sentence in accordance with the template sentence in the template database 15.

According to this embodiment, the utterance generation device 1 accesses the artificial incompetent person using parameters including the target word and the representative word, and obtains a response utterance.
Thereby, the utterance generation device 1 can output response utterances R with a wide variety without relying on a limited template. Furthermore, since the combination of the representative word and the target word is an interpretation of the content of the user's utterance, the utterance generating device 1 can respond to the user's utterance by further acquiring a response sentence. It is possible to realize a dialogue based on

Note that the artificial incompetence may be an application provided in an external server such as a cloud, but is not limited to this, and may be incorporated in the speech generation device 1.

[Third embodiment]
A third embodiment of the present invention will be described below.
The utterance generation device 1 estimates the keyword that is the target of the representative word included in the user utterance sentence U from among the keywords included in the keyword memory 12. The origin is not limited. That is, the source of the target word serving as the context of the dialogue can be various situations such as television, radio, humans, etc. Therefore, by changing the text input to the keyword processing block 30 and changing the data in the keyword memory 12, it is possible to generate utterances in different situations.

In this embodiment, the text input to the keyword processing block 30 is different from the first embodiment, and instead of the above-mentioned television output text T, the text input to the keyword processing block 30 is a text uttered by a plurality of users.
Thereby, the keywords issued by each of the plurality of users are stored in the keyword memory 12 in association with the user.
Then, the target word estimating unit 22 determines a target word from among the keywords extracted from the utterances of a user different from the user who extracted the representative word.

FIG. 10 is a diagram showing an example of generating a response utterance R in a dialogue with multiple users in this embodiment.
In this example, it is assumed that a communication robot in which the speech generation device 1 is installed is present near two users.

The speech generation device 1 uses the speech input device 40 and the sound source localization/separation unit 41 to classify and process the speeches of the two users. Thereafter, the user's utterance U converted into text by the voice recognition unit 42 is input to the representative word search unit 21 and at the same time, it is also input to the keyword extraction unit 31.
At this time, in addition to the keyword and storage date and time fields, the keyword memory 12 is provided with a new field for storing user information. This user information field stores the name or ID of the user who uttered the user utterance U from which the keyword is extracted.

When the representative word search unit 21 extracts a representative word from the user utterance U, the target word estimation unit 22 extracts from the keyword memory 12 a keyword uttered by a user different from the user who uttered the user utterance U. Once extracted, the cosine similarity between these keywords and the representative word is calculated. That is, the target word estimation unit 22 estimates the target word from the keywords uttered by user Y for the utterances of user X, and from the keywords uttered by user X for the utterances of user Y.

For example, in FIG. 10, "ramen" is a keyword uttered by user X, and "yakiniku" and "Shibuya" are stored as keywords uttered by user Y, in the keyword memory 12. At this time, if the user X utters, "Oh, I want to eat that. Let's go.", the representative word search unit 21 extracts the representative word "I want to eat." The target word estimation unit 22 calculates the cosine similarity between the keywords "Yakiniku" and "Shibuya" uttered by the user Y stored in the keyword memory 12 and the representative word "I want to eat," and ranks them (sorts). )I do. As a result, "Yakiniku" is selected as the target word, and by using the template database 15, a response utterance R of "I also want to eat Yakiniku" is generated.

Similarly, when there are three or more users, the target word estimation unit 22 estimates the target word from among the keywords uttered by users other than the user who uttered the utterance from which the representative word has been extracted, thereby generating the response utterance R. can be generated.

According to this embodiment, the utterance generation device 1 determines the target word from among the keywords extracted from the utterances of a user different from the user who uttered the utterance from which the representative word was extracted.
Thereby, the utterance generation device 1 can participate in the conversation of a plurality of users and respond to the utterances of any of the users while estimating the topic among the users.

Although the embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Further, the effects described in the above-described embodiments are only a list of the most preferable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the embodiments.

In the embodiment described above, the utterance generation device 1 generated the response utterance R as a response to the user's utterance, but it may also combine other utterance generation processes to increase the variety of dialogues. For example, the robot may be configured to speak even when the user does not speak, by combining the process of spontaneously speaking based on the TV output sentence T as disclosed in Patent Document 2. .

In the embodiment described above, various databases are provided in the speech generation device 1, but the present invention is not limited to this, and the databases may be placed in an external server such as a cloud.

In the present embodiment, the configuration and operation of the utterance generation device 1 have been mainly described, but the present invention is not limited to this, and may be configured as a method or program that includes each component and generates a robot's utterance. It's okay.

Furthermore, the functions of the utterance generation device 1 may be realized by recording a program on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium. good.

The "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems.

Furthermore, a ``computer-readable recording medium'' refers to a computer-readable storage medium that dynamically retains a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a device that stores a program at a fixed time, such as a volatile memory inside a computer system that is a server or client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system. .

R Response utterance T Television output sentence U User utterance 1 Utterance generation device 11 Keyword database 12 Keyword memory 13 Representative word database 14 Feature vector database 15 Template database 20 Response generation processing block 21 Representative word search unit 22 Target word estimation unit 23 Utterance Sentence generation unit 30 Keyword processing block 31 Keyword extraction unit 32 Keyword management unit 231 Dialogue API calling unit

Claims (7)

a keyword extraction unit that extracts keywords registered in the keyword database from the input sentence;
a keyword management unit that stores the extracted keyword in a keyword memory and deletes the keyword from the keyword memory after a predetermined period of time has elapsed since the keyword was stored;
a representative word search unit that searches for a pre-registered representative word included in a user's utterance;
a target word estimation unit that determines the keyword to be the target word of the uttered sentence based on the distance between the feature vector of the representative word and the feature vector of each of the keywords stored in the keyword memory;
an utterance generating unit that generates a sentence in which the target word is combined with an utterance template registered in advance corresponding to the representative word as a response utterance to the user's utterance ; . a keyword extraction unit that extracts keywords registered in the keyword database from the input sentence;
a keyword management unit that stores the extracted keyword in a keyword memory and deletes the keyword from the keyword memory after a predetermined period of time has elapsed since the keyword was stored;
a representative word search unit that searches for a pre-registered representative word included in a user's utterance;
a target word estimation unit that determines the keyword to be the target word of the uttered sentence based on the distance between the feature vector of the representative word and the feature vector of each of the keywords stored in the keyword memory;
Access an external application that generates a sentence that combines the target word with a pre-registered utterance template corresponding to the representative word, and automatically generates and replies to the received sentence . and an utterance generation unit that acquires a response sentence to the combined sentence as a response utterance to the user's utterance. The utterance generation device according to claim 1 or 2 , wherein the input sentence is extracted from video or audio of a television program. The input sentence is a sentence uttered by a plurality of users,
The utterance generation device according to claim 1 or 2 , wherein the target word estimation unit determines the target word from among keywords extracted from utterances of a user different from the user who extracted the representative word. a keyword extraction step of extracting keywords registered in the keyword database from the input sentence;
a keyword management step of storing the extracted keywords in a keyword memory, and deleting the keywords from the keyword memory after a predetermined period of time has passed since the keywords were stored;
a representative word search step of searching for a pre-registered representative word included in the user's utterance;
a target word estimation step of determining the keyword to be the target word of the uttered sentence based on the distance between the feature vector of the representative word and the feature vector of each of the keywords stored in the keyword memory;
A computer executes a utterance generation step of generating a sentence in which the target word is combined with a utterance template registered in advance corresponding to the representative word as a response utterance to the user's utterance. Utterance generation method. a keyword extraction step of extracting keywords registered in the keyword database from the input sentence;
a keyword management step of storing the extracted keywords in a keyword memory, and deleting the keywords from the keyword memory after a predetermined period of time has passed since the keywords were stored;
a representative word search step of searching for a pre-registered representative word included in the user's utterance;
a target word estimation step of determining the keyword to be the target word of the uttered sentence based on the distance between the feature vector of the representative word and the feature vector of each of the keywords stored in the keyword memory;
Access an external application that generates a sentence that combines the target word with a pre-registered utterance template corresponding to the representative word, and automatically generates and replies to the received sentence . and acquiring a response sentence to the combined sentence as a response sentence to the user's utterance. A speech generation program for causing a computer to function as the speech generation device according to any one of claims 1 to 4 .

Images