JP6654611B2 - Growth type dialogue device - Google Patents

Description

  The present invention relates to a growing dialogue apparatus, and more particularly to a growing dialogue apparatus suitable for performing speech recognition learning and intention understanding learning for automatically performing a voice dialogue.

  2. Description of the Related Art A system (hereinafter, referred to as a “dialogue system”) for performing a dialogue between a computer or a robot and a human user by voice has been developed. A robot equipped with this dialogue system or a device such as a smartphone or a tablet PC (hereinafter, referred to as a “dialogue device”) operates the dialogue system and implements a service by voice dialogue to a facing user.

  In this interactive system, first, a voice uttered by a user is collected by a microphone to perform voice recognition. Next, the transcribed text is subjected to a process called intent understanding to determine what the user wants the interactive system to do. Finally, based on the determination result, the interactive device performs an operation intended by the user, for example, an operation of reading out a response to a question or displaying information, or an operation of moving a robot.

  In this dialogue system, if the user asks a question for which the system does not know the answer, the dialogue system has previously provided an answer such as "I do not know". And the knowledge of the question answer can be updated with the result. Patent Document 1 discloses the configuration of such a dialog system. In the voice dialogue device of Patent Document 1, when the user is informed that he or she does not know, the user is asked for the answer, the content of the question and the answer are stored, and used for the next dialogue. As a result, the need to interrupt the conversation due to unknown conversation contents or change the topic presented by the user is reduced, and the learning is increased to increase new scenarios and vocabulary to improve the knowledge, and to improve the user And can be reflected in the dialogue.

  By the way, in the interactive system, speech recognition is performed for the utterance of the user, but if the accuracy of the speech recognition is not good, the intention understanding is not correctly performed as a result.

  Patent Literature 2 discloses an intention estimating method that uses a learning text including an error intentionally as a learning text for understanding an intention to enhance tolerance to an error in speech recognition and ensure the understanding of the intention. An apparatus is disclosed.

JP 2004-109323 A JP 2015-230384 A

  According to Patent Document 2 of the related art described above, the accuracy of intention estimation can be improved by including an erroneous text in a text obtained by speech recognition.

  However, Patent Literature 2 does not suggest the viewpoint of re-learning the speech recognition process and the intention understanding process based on the reaction of the user to improve the accuracy of the speech recognition process and the intention understanding process. In general, in a dialogue system, when the user's intention is different, the user learns speech recognition processing and intention understanding processing while interacting with the user, but performs speech recognition processing in order to understand the user's intention. Since the intention understanding process is performed afterwards, it is difficult to determine whether the error is caused at the time of the speech recognition process or the error is caused at the time of the intention understanding process. There is a problem that it is difficult to determine whether or not.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a growth type capable of appropriately performing optimal speech recognition processing and re-learning of intention understanding processing based on the correspondence of a user, and improving both correctness accuracy of the speech recognition processing and intention understanding processing. It is to provide an interactive device.

  The configuration of the growing dialogue apparatus of the present invention is preferably a growing dialogue apparatus that performs a voice dialogue with a user, analyzes and responds to the intention of the user, and the voice recognition learned from the learning data for voice recognition. Speech recognition unit that performs speech recognition processing on the acquired user's uttered speech by referring to the model data for speech recognition, and refers to the model data for intention understanding learned from the learning data for intention understanding to perform speech recognition. An intention understanding unit that performs an intention understanding process that analyzes the utterance intention from the text recognized by the speech recognition unit, and an extended intention understanding learning data in which different texts are associated with the same intention label in the intention understanding learning data. Referring to the learned model data for extended intention understanding, an intention understanding process for analyzing the utterance intention from the text recognized by the speech recognition unit is performed. An extended intention understanding unit and a user response identification unit that analyzes a user's voice response, a video to which the user responds, and determines whether the response to the uttered voice of the growing dialogue device is positive or negative, Based on the judgment result of the user reaction identification unit, whether there is an error in the speech recognition processing, whether there is an error in the intention label output by the intention understanding unit that referred to the model data for intention understanding, or refer to the model data for extended intention understanding. Error analysis unit that determines whether there is an error in the extended intention label output by the extended intention understanding unit, and speech recognition learning that updates the model data for speech recognition from the learning data for speech recognition according to the determination by the error cause analysis unit. Processing, updating intention understanding model data from intention understanding learning data Intention understanding learning processing, updating extended intention understanding model data from extended intention understanding learning data Either intention understanding learning process, or is obtained by so and a relearning determination unit for instructing the processing of the combination.

  ADVANTAGE OF THE INVENTION According to this invention, the optimal type | mold speech recognition process and the re-learning of the intention understanding process are appropriately performed based on the correspondence of the user, and the growth type which can improve the correct answer precision of both the speech recognition process and the intention understanding process. An interactive device can be provided.

FIG. 2 is a diagram illustrating a functional configuration of the growth-type interactive device according to the first embodiment. It is a figure showing an example of model data for intention understanding. It is a figure showing an example of learning data for intention understanding. It is a figure showing an example of learning data for extension intention understanding. FIG. 4 is a diagram showing a flow of processing of the growing interactive device according to the first embodiment. 5 is a flowchart illustrating an error cause analysis process according to the first embodiment. FIG. 9 is a diagram illustrating a functional configuration of a growth-type interactive device according to a second embodiment. FIG. 14 is a diagram showing a flow of processing of the growing interactive device according to the second embodiment. 10 is a flowchart illustrating an error cause analysis process according to the second embodiment (part 1). 10 is a flowchart illustrating an error cause analysis process according to the second embodiment (part 2). It is a figure showing an example of a transcription text input screen.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 10.

[Embodiment 1]
Hereinafter, Embodiment 1 of the present invention will be described with reference to FIGS.

First, the configuration of the growth-type interactive device according to the first embodiment will be described with reference to FIG.
As shown in FIG. 1, the growth-type interactive device 200 according to the present embodiment includes a processing unit 210, a storage unit 280, a voice input / output unit 230, and an image input unit 240. The growth-type interactive device 200 may be a general information processing device such as a PC or a smartphone, or may be a robot for performing a specific task.

  A microphone 231 and a speaker 230 are connected to the audio input / output unit 230 so that a user's voice is input from the microphone 231 and a voice is output from the speaker 230 to the user.

  A camera (imaging device) 242 is connected to the image input unit 240, and can capture moving images and still images taken from the camera.

The processing section 210 is a section that executes each processing of the growing interactive apparatus 200, and includes an utterance section detection section 211, a speech recognition section 212, an intention understanding section 213, an extended intention understanding section 214, an intention understanding model learning section 215, and a response. The sub-components include a generation unit 216, a voice synthesis unit 217, a voice emotion recognition unit 218, a face / expression motion recognition unit 219, a user reaction identification unit 220, an error cause analysis unit 260, and a relearning determination unit 270.
The functions of the sub-components of the processing unit 210 are executed when a central processing unit (CPU) as hardware executes a program loaded on a main storage device.

  The utterance section detection unit 211 is a functional unit that detects a silent section from a user voice and detects an utterance section. The voice recognition unit 212 is a functional unit that recognizes a user voice and converts it into text. The intention understanding unit 213 is a functional unit that, based on the model data for intention understanding 110, means the meaning of the user's utterance from the text recognized by speech. The extended intention understanding unit 214 is a functional unit that, based on the extended intention understanding model data 111, means the user's utterance intention from the text recognized by speech. The intention understanding model learning unit 215 is a functional unit that generates intention understanding model data (described later) from intention understanding learning data (described later). The response generation unit 216 is a functional unit that generates data for a response of the growing interactive device 200 based on the processing of the intention understanding unit 213. The voice synthesizer 217 is a functional unit that generates voice data for the growing interactive device 200 to respond to the user. The voice emotion recognition unit 218 is a functional unit that recognizes voice input from the user and determines whether the user's emotion is positive or negative. The face / expression motion recognition unit 219 recognizes the user's face, facial expression and the motion based on the image of the user input from the image input unit 240, and determines whether the user's emotion is positive or negative. It is. The error cause analysis unit 260 is a functional unit that determines whether the cause of the error in the user's understanding of the intention as a result of the user's response is due to the speech recognition processing or the intention understanding processing. The re-learning processing unit 270 is a functional unit that determines whether to re-learn speech recognition or re-learn intention understanding based on the analysis result of the error cause analysis unit 260.

  The storage unit 280 is a unit that stores data necessary for executing each process of the growing dialogue apparatus 200, and includes the voice recognition model data 108, the voice recognition learning data 150, the intention understanding model data 110, and the extended intention. Various data of the model data for understanding 111, the learning data for intention understanding 120, and the learning data for extended intention understanding 121 are stored.

  The model data for speech recognition 108 is reference data for speech recognition including probability values for converting acoustic data into phonemes and identifying word connections. The speech recognition learning data 150 is a speech corpus in which speech data and text used for creating the speech recognition model data 108 are paired. The intention understanding model data 110 is data generated based on the intention understanding learning data 120, and is reference data for extracting an intention from the recognized text. The extended intention understanding model data 111 is data generated based on the extended intention understanding learning data 121, and is reference data for extracting the intention from the recognized text. The intention understanding learning data 120 is learning data for linking the recognized text with the user's utterance intention. The extended intention understanding learning data 121 is data obtained by adding a label indicating the intention to the text when the recognized text has an error, in addition to the intention understanding learning data 120.

The intention understanding learning data of the present embodiment includes two types of a normal version of the intention understanding learning data 120 and the extended intention understanding learning data 121 obtained by expanding the normal version of the learning data. Note that certain intention understanding model data 110 and extended intention understanding model data 111 are generated.
The details of each data will be described later.

Next, main data structures used in the growth-type interactive apparatus according to the present embodiment will be described with reference to FIGS.
The intention understanding model data 110 is reference data for extracting an intention from the recognized text, and includes fields of a text 110a, an intention label 110b, and a certainty factor 110c, as shown in FIG.

  The text 110a is a field for storing a word string of a text obtained from the intention understanding learning data. The intention label 110b is a field for storing a label indicating an intention read from a user for operating the growth-type interactive device. The certainty factor 110c indicates a probability between 0 and 1 that the growth-type interactive device 200 reads and responds to the intention of the intention label 110b from the user when a word of the text 110a appears in the conversation with the user. This is a field for storing a numerical probability.

  For example, in the example shown in the first record in FIG. 2, when the word “Takaoyama” appears in the dialogue with the user, the intention of the user to read “the intention of climbing Takao” is 0.3. In the example shown in the second record, when the words “Mt. Takao” and “Climbing” appear in the dialogue with the user, the intention to read from the user is “0.8. It indicates that there is.

  In the example shown in the seventh record of FIG. 2, when the word "Takaoyama" appears in a dialogue with the user, "What is the height of Mt. Takao?" In the example shown in the eighth record, when the words “Takaoyama” and “height” appear in the dialogue with the user, “what is the height of Mt. Takao?” , 0.9.

The growth-type dialogue apparatus 200 recognizes the voice from the user, refers to the model data for intent understanding 110, analyzes the words contained therein, and obtains the highest confidence in the corresponding intention label. , It is possible to estimate the intention of the user most likely.
The data structure of the extended intention understanding model data 111 is the same as that of the intention understanding model data 110.

  The learning data for intention understanding 120 is the original data for creating the model data for intention understanding 110, and includes fields of a text 120a and an intention label 120b as shown in FIG.

  The text 120a is a field for storing text transcribed by speech recognition of a user's conversation. The intention label 120b is a field for storing an intention label for the growth-type interactive device corresponding to the text 120a to read the user's intention. In the example illustrated in FIG. 3, when the user reads “Climb Mt. Takao” as the intention label 120b of the user, the corresponding text is “I want to climb Mt. Takao”, “I want to climb Mt. I have never climbed. "

  The extended intention understanding learning data 121 is the original data for creating the extended intention understanding model data 111, and includes fields of a text 121a and an intention label 121b as shown in FIG. The extended intention understanding learning data 121 has the same data structure as the intention understanding learning data 120, and the meanings of the text 121a and the intention label 121b are the same as those of the text 120a and the intention label 120b of the intention understanding learning data 120, respectively. is there.

  However, as shown in FIG. E, new data (sixth record to eighth record) is added to the learning data 121 for understanding an extended intention. These records are texts when speech recognition of the text 121a is erroneously recognized, and correspond to the intention label 121b. For example, when the user's utterance is “I want to climb Mt.

  By creating model data by learning based on the extended intention understanding learning data 121, the growth type interactive device 200 is robust (Robust) against erroneous recognition of speech recognition processing, and performs speech recognition processing. It can be expected that even if there is an erroneous recognition of, the user will be able to reach the intention. However, regarding the sixth record, if the text of the user's voice recognition result is “Look at Mt. Takao” and the user's intention is “I want to Mt. Takao” (view Mt. It is important to keep in mind that you may be guided by your intentions.

Next, the processing of the growing interactive device according to the first embodiment will be described with reference to FIGS.
First, a flow of a voice interaction process through a growing dialogue device, particularly a flow of a series of processes until a growing dialogue device utters an answer voice in response to a question voice spoken by a user will be described with reference to FIG. I do.

  The growing interactive device always keeps collecting sound with the microphone 231 shown in FIG. In this situation, the user uttered voice 101 uttered by the user is subjected to voice input processing 102, converted into a digital signal by a microphone, and then only the user's voice part is cut out by the utterance section detection processing 103. Next, voice recognition processing is performed by the voice recognition processing 104, and the text is transcribed into text corresponding to the voice. Here, the voice recognition processing 104 uses the voice recognition model data 108.

  Next, an extension intention understanding process 115 is performed based on the model data 111 for extension intention understanding. In the present embodiment, performing the intention understanding process based on the model data for intention understanding 110 is referred to as an intention understanding process 105 (described later), and the intention understanding process based on the model data for extended intention understanding 111 is referred to as extended intention understanding. The processing 115 is distinguished. The extended intention understanding process 115 is a process of determining an intention label of a user question (the intention label output at this time is referred to as an “extended intention label”) based on the word information included in the speech recognition text. .

  Then, upon receiving the output of the intention label, a response generation process 106 is performed, the answer text is converted into a synthesized voice by a voice synthesis process 107, a voice output process 112 is performed from the interactive device through the speaker 232, and the system voice 113 is uttered. .

  On the other hand, the growth-type interactive device 200 captures a response to the system utterance voice 113 by the camera 242, captures an image by the image input processing 130, and captures the captured human image or face by the person / face extraction processing 131. And image recognition is performed by the face / expression action recognition processing 132.

  In addition, the growth type interactive device 200 captures the user uttered voice 101 by the voice input process 102 as a response to the system uttered voice 113, and performs the emotion recognition process on the user through the voice emotion recognition process 133.

  Then, the output of the face / expression motion recognition process 132 and the output of the voice emotion recognition process 133 are analyzed by the user response identification process 134 to determine whether the response of the target user is positive (Positive). , Is determined to be negative, and the result is input to the error cause analysis processing 116.

  In the above process, the extended intention understanding process 115 is performed based on the extended intention understanding model data 111. At the same time, the intention understanding process 105 is performed based on the intention understanding model data 110 to obtain an intention label (at this time, Is referred to as a “normal intention label” in order to distinguish the intention label from the intention label “extended intention label” output by the extended intention understanding process 115).

The extended intention label and the normal intention label are also input to the error cause analysis processing 116.
Then, based on the result of the error cause analysis processing 116, the re-learning determination processing 117 is operated, and 1) the voice recognition learning processing 141 is performed using the voice recognition learning data 150 to update the voice recognition model data 108. Or 2) performing intention understanding learning processing 140 using the intention understanding learning data 120 to update the intention understanding model data 110, or 3) using the extended intention understanding learning data 121 to perform intention understanding learning. The processing 118 is performed to update the extended intention understanding model data 111. 4) The weight of the extended intention understanding learning data 121 is corrected by the weight adjustment learning processing 119 to update the extended intention understanding model data 111. In addition, according to the result of the error cause analysis processing 116, the response generation processing 106 Combining process 107, performs voice output processing 112, performs a speech processing system speech 113, also performs watching the reaction of the user (the image and audio inputs, the path leading to the user reaction identification process 134).

Next, the error cause analysis processing 116 will be described in detail with reference to FIG.
First, the growth-type interactive device 200 receives the result of the user response identification processing 134, and determines whether the user response of the system utterance voice 113 is positive or negative based on the extended intention label (S100). .
When the user response is positive (S100: YES), it is next determined whether or not the normal intention label and the extended intention label match (S101).

  When the normal intention label and the extended intention label match (S101: YES), it is estimated that the speech recognition processing is correct, the normal intention label is correct, and the extended intention label is correct. Therefore, at this time, it is determined that there is no need for re-learning of speech recognition processing and intention understanding, and re-learning is not performed (S200).

  When the normal intention label and the extended intention label do not match (S101: NO), it is estimated that the speech recognition processing is erroneous, the normal intention label is erroneous, and the extended intention label is correct. In this case, it is considered that the extended part of the extended intention understanding learning data 121 has derived a correct intention label, and it is considered that there is an error in the speech recognition processing. Therefore, at this time, re-learning of an error in the speech recognition processing is performed (S201).

  Further, in the process of S100, when the user response is negative (S100: NO), it is next determined whether or not the normal intention label and the extended intention label match (S102).

  When the normal intention label matches the extended intention label (S102: YES), it is presumed that the speech recognition processing is erroneous, the normal intention label is erroneous, and the extended intention label is erroneous. In this case, it is assumed that when the pronunciation is correct, a correct intention label is derived by the intention understanding processing 105. At this time, re-learning of an error in the speech recognition processing, updating of the intent understanding model data 110 by the intent understanding learning processing 140 using the intent understanding learning data 120, and intent understanding using the extended intent understanding learning data 121 The extended intention understanding model data 111 is updated by the learning process 118 (S202).

  When the normal intention label does not match the extended intention label (S102: NO), it is estimated that the speech recognition processing is correct or incorrect, the normal intention label is correct or incorrect, and the extended intention label is incorrect.

  In this case, the response generating process 106, the voice synthesizing process 107, and the voice output process 112 are performed based on the normal intention label, the system utterance voice 113 is uttered, the user's response is observed, and the user response identifying process 134 is performed. It is determined whether or not the user's response is positive (S103).

  In the process of S103, when the user's response is negative (S103: NO), it is estimated that the speech recognition process is erroneous, the normal intention label is erroneous, and the extended intention label is erroneous. Therefore, re-learning of an error in the speech recognition processing, updating of the intention understanding model data 110 by the intention understanding learning processing 140 using the intention understanding learning data 120, and intention understanding learning using the extended intention understanding learning data 121 The extended intention understanding model data 111 is updated by the process 118 (S202).

  In the process of S103, when the user's response is positive (S103: YES), it is estimated that the speech recognition process is correct, the normal intention label is correct, and the extended intention label is incorrect. At this time, it is conceivable that the expanded portion of the expanded intention understanding learning data 121 has an adverse effect. Therefore, by the weight adjustment learning process 119, a process of reducing the weight of the extended part and reducing its influence is performed (S203).

  The weight of the extension part is, specifically, the number of records of the extension part. As the number of the records increases, the confidence level 110C of the model data for intent understanding shown in FIG. Has been.

  In the above description, the re-learning process of the speech recognition error, the intention understanding learning process 118 using the extended intention understanding learning data 121, and the intention understanding learning process 140 using the intention understanding learning data 120 are automatically performed. Although it is described that the processing is performed, a dialog window may be output to a display device, an inquiry may be made to the operator, and the processing may be performed when the operator approves.

  According to the present embodiment, the content of the normal intention label and the content of the extended intention label are compared based on whether the user is affirmative with respect to the response of the growing interactive device 200 based on the extended intention label, By performing appropriate speech recognition learning and intention understanding learning, the accuracy of speech recognition processing and the precision of intention understanding processing can be improved.

[Embodiment 2]
The second embodiment will be described below with reference to FIGS.
In the first embodiment, the correspondence to the intention understanding process based on the extended intention understanding model data 111 constructed by the extended intention understanding learning data 121 and the contents of the normal intention label and the extended intention label are compared, so that the appropriate The speech recognition learning and the intention understanding learning are performed, and how the learning is performed by the processing of FIG. 6 has been described.

  However, the determination of the correctness of the speech recognition process at this time involves estimation and is not complete. For example, when the user response of the system utterance speech 113 based on the extended intention label is positive (S100: YES) and the normal intention label matches the extended intention label (S101: NO), the determination in FIG. Then, it is estimated that the speech recognition is correct, but even in this case, the speech recognition processing may be incorrect. Similarly, for the other combinations of YES and NO in S100 and the combinations of YES and NO in S101 and S102, there is a case where the speech recognition process is estimated to be correct, a case where the speech recognition process is incorrect, and a case where the speech recognition process is incorrect. Even when it is estimated that it is correct, there may be times when it is correct. In such a case, it is difficult for the logic of FIG. 6 to determine whether the speech recognition processing is correct or incorrect.

  The growth-type interactive device of the present embodiment aims to further improve the accuracy of voice recognition by allowing the growth-type interactive device to re-learn a text transcribed by listening to the speech of the user by the operator. is there.

In the present embodiment, a description will be given focusing on a point different from the second embodiment.
First, the configuration of the growth-type interactive device according to the second embodiment will be described with reference to FIG.
The growing interactive device 200 according to the second embodiment includes a data input / output unit 250 in addition to the structure of the growing interactive device 200 according to the first embodiment.

  The data input / output unit 250 is a functional unit for an operator to input data and display data, and is connected to input devices such as a display device 251, a mouse 252, and a keyboard 253.

  Further, a text transcription interface unit 280 is added to the processing unit 210. The text transcription interface unit 280 is a functional unit that allows the operator to listen to the user's voice, transcribe and input as text.

  The display device 251 is a device that displays information such as a liquid crystal display (LCD). Input devices such as a mouse 252 and a keyboard 253 are devices for inputting data and commands from an operator.

The storage unit 280 holds the voice data 170 and the transcribed text 190 in addition to the growth-type interactive device 200 of the first embodiment.
The voice data 170 is data obtained by recording a voice of a user who has interacted with the growing interactive device 200.
The transcribed text 190 is data in which the voice transcribed by the user by the operator is transcribed as text.

Next, the processing of the growth-type interactive device according to the second embodiment will be described with reference to FIGS.
The processing flow of the growing dialogue apparatus 200 according to the second embodiment is the same as the processing flow of the growing dialogue apparatus 200 according to the first embodiment shown in FIG. In addition, a transcript acquisition process 160 is added. The other processing flow is the same as the processing flow of the growth-type interactive device 200 according to the first embodiment illustrated in FIG.

  In the transcribed text acquisition process 160, based on the determination of the error cause analysis process 116, the operator hears the voice spoken by the user and transcribes it as text. When necessary, the speech and the text are stored in the speech recognition learning data 150 and re-learned, thereby learning a speech-text pair recognized by a human, thereby improving the accuracy of the speech recognition process. Can be done.

Next, an error cause analysis process according to the second embodiment will be described in detail with reference to FIGS. 9A and 9B.
The error cause analysis processing according to the second embodiment is almost the same as that of the first embodiment shown in FIG. 6, but as shown in FIG. 9A, S101: NO (* A), S102: YES and S103 in FIG. : The processing in the case of NO (* B) is different.

  When the user's reaction of the system utterance voice 113 based on the extended intention label is positive (S100: YES) and the normal intention label and the extended intention label do not match (S101: NO), in the present embodiment, the operator determines the user's The utterance is heard, transcribed, and a text is input (S2010 in FIG. 9B).

Then, the growth-type interactive device 200 determines whether or not the result of the speech recognition matches the transcribed text (S2011). Is correct, and the extended intention label is presumed to be correct.
At this time, re-learning of an error in the speech recognition processing is performed (S2012).

The growth-type interactive device 200 determines whether or not the result of the speech recognition matches the transcribed text (S2011). If the result matches (S2011: YES), the speech recognition process is correct and the normal intention label is incorrect. , The extended intention label is presumed to be correct.
At this time, it is considered that the intention by the extended intention label was affirmative because it was determined that the extended intention label was correct despite the fact that the normal intention label was incorrect. Therefore, re-learning is performed using the normal intention understanding learning data 120 to update the intention understanding model data 110 (S2013).

  Next, when the user reaction of the system utterance speech 113 based on the extended intention label is negative (S100: NO) and the normal intention label matches the extended intention label (S101: YES), or In a case where the extended intention labels do not match (S101: NO) and the user's reaction due to the normal intention label is negative (S103: NO), in this embodiment, the operator listens to the user's uttered voice and transcribes it. Input a text (S2012).

  Then, the growth-type interactive device 200 determines whether or not the speech recognition result matches the transcribed text (S2021). If not (S2021: NO), the speech recognition process is erroneous, and the normal intention label is used. Is incorrect and the extended intention label is estimated to be incorrect. Therefore, at this time, re-learning of an error in the speech recognition processing, updating of the intention understanding model data 110 by the intention understanding learning processing 140 using the intention understanding learning data 120, and use of the extended intention understanding learning data 121 are performed. The extended intention understanding model data 111 is updated by the intention understanding learning process 118 (S2022).

  The growth-type interactive device 200 determines whether or not the result of the speech recognition matches the transcribed text (S2021). If the result matches (S2021: YES), the speech recognition processing is correct, and the normal intention label is determined. In error, it is presumed that the extended intention label is incorrect. Therefore, at this time, the intent understanding model data 110 is updated by the intent understanding learning process 140 using the intent understanding learning data 120, and the extended intent understanding process 118 is performed by the intent understanding learning process 118 using the extended intent understanding learning data 121. The model data 111 is updated (S2023).

Next, an example of a transcription text input screen output by the text transcription interface unit will be described with reference to FIG.
The transcribed text input screen 400 is a screen in which the operator listens to the uttered voice of the user and inputs the transcribed text, and is output to the display device 251 in FIG.

  The transcription text input screen 400 includes a speech recognition result display field 401, a normal intention label display field 402, an extended intention label display field 403, a transcription text input field 404, a voice output button 410, an OK button 411, and a cancel button 412. .

  The speech recognition result display column 401 is a column for displaying text as a result of the speech recognition processing in units of phrases. The normal intention label display field 402 and the extended intention label display field 403 are fields for displaying a normal intention label and an extended intention label, respectively.

  The transcribed text input field 404 is a field where the operator listens to the user's uttered voice and inputs a text that can be heard. The input of the text is performed by, for example, the keyboard 253.

  The voice output button 410 is an operation button for outputting a user's uttered voice. The OK button 411 is a button for confirming the text input in the transcript text input field 404 and inputting the text to the growing interactive device 200. The cancel button 412 is an operation button for canceling the processing and closing the transcribed text input screen 400. The button operation can be performed with a pointing device such as the mouse 252.

  Instead of using such a user interface, the speech recognition result and the transcribed text may be input as plain text such as a CSV (Comma-Separated Value) format.

  Further, the text that transcribes the user's uttered voice may be limited to those whose confidence level of the voice recognition process is equal to or less than a predetermined threshold. In addition, a certain section such as a phrase unit of audio data may be limited to a section that is equal to or less than a predetermined threshold.

  In the present embodiment, the operator listens to the uttered voice of the user, determines whether re-learning of voice recognition is possible, and learns with the transcribed text, so that it takes time and effort, but improves the accuracy of the correct answer of voice recognition. You can expect that.

101: User uttered voice 102: Voice input process 103: Utterance section detection process 104: Voice recognition process 105: Intention understanding process 106: Response generation process 107: Voice synthesis process 108: Voice recognition model data 110: Intention understanding model data 111 ... Extended intention understanding model data 112 ... Speech output processing 113 ... System utterance speech 115 ... Extended intention understanding processing 116 ... Error cause analysis processing 117 ... Re-learning decision processing 118 ... Intention understanding learning processing 119 ... Weight adjustment learning processing 120 ... Intention understanding learning data 121 ... Extended intention understanding learning data 130 ... Image input processing 131 ... Person / face extraction processing 132 ... Face / expression motion recognition processing 133 ... Speech emotion recognition processing 140 ... Intention understanding learning processing 141 ... Speech recognition learning Process 150: Learning data for speech recognition 160: Transcription Processing

Claims (12)

A growth-type interactive device that performs a voice dialogue with a user, analyzes and responds to the intention of the user,
A speech recognition unit that performs speech recognition processing from the acquired user's uttered speech and converts it into text by referring to the speech recognition model data learned from the speech recognition learning data;
With reference to the model data for intention understanding learned from the learning data for intention understanding, an intention understanding unit that performs an intention understanding process of analyzing an utterance intention from the text recognized by the speech recognition unit,
With reference to the extended intention understanding model data learned from the extended intention understanding learning data in which different texts are associated with the same intention label of the intention understanding learning data, speech recognition is performed by the speech recognition unit. An extended intent understanding unit that performs intent understanding processing for analyzing speech intention from the text
A user response identification unit that analyzes a user's voice response, a video to which the user responds, and determines whether the response to the utterance voice of the growing dialogue device is positive or negative,
Based on the determination result of the user response identification unit, whether there is an error in the speech recognition process, whether there is an error in the intention label output by the intention understanding unit with reference to the model data for intention understanding, whether the model for extended intention understanding An error cause analysis unit that determines whether there is an error in the extension intention label output by the extension intention understanding unit that refers to the data,
Speech recognition learning processing for updating the speech recognition model data from the speech recognition learning data according to the determination of the error cause analysis unit, and intention understanding learning processing for updating the intention understanding model data from the intention understanding learning data A re-learning decision unit for instructing any one of intention understanding learning processes for updating the model data for extended intention understanding from the learning data for extended intention understanding, or a combination thereof. Interactive device. Whether the user response identification unit is affirmative or negative with respect to the utterance response of the growing-type dialogue device by the extended intention label output by the extended intention understanding unit with reference to the extended intention understanding model data Determine the first user reaction of
The error cause analysis unit includes an intention label output by an intention understanding unit that refers to the intention understanding model data with respect to an uttered voice of the same user as the first user reaction determined by the user reaction identification unit. 2. The growth-type dialogue apparatus according to claim 1, wherein the determination is made based on a match between the extended intention label output from the extended intention understanding unit with reference to the extended intention understanding model data. The first user reaction determined by the user reaction identification unit is positive,
When the intention label output by the intention understanding unit with reference to the model data for intention understanding and the extended intention label output by the extended intention understanding unit with reference to the model data for extended intention understanding do not match,
The growth type interactive apparatus according to claim 2, wherein a speech recognition learning process for updating the speech recognition model data from the speech recognition learning data is performed according to the determination by the error cause analysis unit. The first user reaction determined by the user reaction identification unit is negative,
When the intention label output by the intention understanding unit with reference to the model data for intention understanding and the extended intention label output by the extended intention understanding unit with reference to the model data for extended intention understanding match,
In accordance with the determination of the error cause analysis unit, a speech recognition learning process for updating the speech recognition model data from the speech recognition learning data is performed, and the intention understanding updating the intention understanding model data from the intention understanding learning data is performed. 3. The growth type interactive apparatus according to claim 2, wherein a learning process is performed, and an intention understanding learning process of updating the extended intention understanding model data from the extended intention understanding learning data is performed. The first user reaction determined by the user reaction identification unit is negative,
When the intention label output by the intention understanding unit with reference to the model data for intention understanding and the extended intention label output by the extended intention understanding unit with reference to the model data for extended intention understanding do not match,
The user response identification unit is an affirmative or negative answer to the utterance response of the growing dialogue device by the extended intention label output by the intention understanding unit with reference to the intention understanding model data. Judge the second user reaction,
When the second user reaction determined by the user reaction identification unit is negative,
In accordance with the determination of the error cause analysis unit, a speech recognition learning process for updating the speech recognition model data from the speech recognition learning data is performed, and the intention understanding updating the intention understanding model data from the intention understanding learning data is performed. Performing a learning process, performing an intention understanding learning process of updating the extended intention understanding model data from the extended intention understanding learning data,
When the second user reaction determined by the user reaction identification unit is positive,
3. The growth type interactive apparatus according to claim 2, wherein a process of adjusting a weight of the learning data for extended intention understanding, which has an influence on generation of the model data for extended intention understanding, is performed.   A speech recognition learning process for updating the model data for speech recognition from the learning data for speech recognition; an intention understanding learning process for updating the model data for intention understanding from the learning data for intention understanding; The growth-type interactive device according to claim 1, wherein any one of the intention understanding learning processes for updating the extended intention understanding model data, or a process of a combination thereof, is performed when an operator instructs. Further, a text transcript interface unit for listening to the user's uttered voice and inputting the transcribed text by the operator is provided,
The growth-type interactive apparatus according to claim 1, wherein it is determined whether or not the text output by the voice recognition processing matches the text transcribed by the operator with respect to the uttered voice of the same user. Whether the user response identification unit is affirmative or negative with respect to the utterance response of the growing-type dialogue device by the extended intention label output by the extended intention understanding unit with reference to the extended intention understanding model data Determine the first user reaction of
The error cause analysis unit includes an intention label output by an intention understanding unit that refers to the intention understanding model data with respect to an uttered voice of the same user as the first user reaction determined by the user reaction identification unit. The growth type interactive apparatus according to claim 7, wherein the determination is made based on a match between the extended intention label output from the extended intention understanding unit with reference to the extended intention understanding model data. The first user reaction determined by the user reaction identification unit is positive,
When the intention label output by the intention understanding unit with reference to the model data for intention understanding and the extended intention label output by the extended intention understanding unit with reference to the model data for extended intention understanding do not match,
For the uttered voice of the same user, determine whether the text output by the voice recognition process and the text transcribed by the operator match,
When the text output by the voice recognition process does not match the text transcribed by the operator for the uttered voice of the same user,
According to the determination of the error cause analysis unit, perform a voice recognition learning process of updating the voice recognition model data from the voice recognition learning data,
When the text output by the voice recognition process and the text transcribed by the operator match for the uttered voice of the same user,
9. The growth type interactive apparatus according to claim 8, wherein an intention understanding learning process for updating the intention understanding model data from the intention understanding learning data is performed. The first user reaction determined by the user reaction identification unit is negative,
When the intention label output by the intention understanding unit with reference to the model data for intention understanding and the extended intention label output by the extended intention understanding unit with reference to the model data for extended intention understanding match,
For the uttered voice of the same user, determine whether the text output by the voice recognition process and the text transcribed by the operator match,
When the text output by the voice recognition process does not match the text transcribed by the operator for the uttered voice of the same user,
In accordance with the determination of the error cause analysis unit, a speech recognition learning process for updating the speech recognition model data from the speech recognition learning data is performed, and the intention understanding updating the intention understanding model data from the intention understanding learning data is performed. Performing a learning process, performing an intention understanding learning process of updating the extended intention understanding model data from the extended intention understanding learning data,
When the text output by the voice recognition process and the text transcribed by the operator match for the uttered voice of the same user,
In accordance with the determination of the error cause analysis unit, an intention understanding learning process for updating the intention understanding model data from the intention understanding learning data is performed, and the extended intention understanding model data is updated from the extended intention understanding learning data. The growth type interactive device according to claim 8, wherein an intention understanding learning process is performed. The first user reaction determined by the user reaction identification unit is negative,
When the intention label output by the intention understanding unit with reference to the model data for intention understanding and the extended intention label output by the extended intention understanding unit with reference to the model data for extended intention understanding do not match,
The user response identification unit is an affirmative or negative answer to the utterance response of the growing dialogue device by the extended intention label output by the intention understanding unit with reference to the intention understanding model data. Judge the second user reaction,
When the second user reaction determined by the user reaction identification unit is negative,
For the uttered voice of the same user, determine whether the text output by the voice recognition process and the text transcribed by the operator match,
When the text output by the voice recognition process does not match the text transcribed by the operator for the uttered voice of the same user,
In accordance with the determination of the error cause analysis unit, a speech recognition learning process for updating the speech recognition model data from the speech recognition learning data is performed, and the intention understanding updating the intention understanding model data from the intention understanding learning data is performed. Performing a learning process, performing an intention understanding learning process of updating the extended intention understanding model data from the extended intention understanding learning data,
When the text output by the voice recognition process and the text transcribed by the operator match for the uttered voice of the same user,
According to the determination of the error cause analysis unit, an intention understanding learning process for updating the intention understanding model data from the intention understanding learning data is performed, and the extended intention understanding model data is updated from the extended intention understanding learning data. The growth type interactive device according to claim 8, wherein an intention understanding learning process is performed.   5. The method according to claim 4, wherein only the transcribed text of the text whose confidence factor of the voice recognition processing is equal to or less than a predetermined threshold value or the text in which a certain section such as a phrase unit of the voice data is equal to or less than the predetermined threshold value is input. The growth-type interactive device according to claim 7.

Images