JP3499658B2 - Dialogue support device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a dialogue support device for supporting conversation between humans using an electronic computer.

[0002]

2. Description of the Related Art In recent years, with the progress of hardware technology, particularly semiconductor integration technology, portable dialogue support devices have been put to practical use. These dialogue support devices can be used in various situations,
For example, an expression that is considered to be used in a hotel or an airport is stored together with the translation result, searched and displayed as needed, and used for conversation. In addition, some of them have a voice output function and can output a stored sentence as a voice.

The dialogue support device is used when the user has very little knowledge of the language (eg, English) supported by the device and cannot express his or her intention at all. It is effective when no response is required, that is, when the other party only needs to understand his / her intention.

However, in a normal dialogue, it is usual to confirm the intention of the speaker or to ask a question that the speaker does not say, and if such a response is not appropriately handled, Dialogue cannot be established. However, the above-described dialogue support device does not have a function of translating and displaying the utterance content of the other party, or searching and displaying an expression having the same content as the utterance of the other party. In addition, there is a problem in that the user himself / herself needs to search for expressions that are useful when responding to the utterance of the other party, and the effectiveness in actual conversation is low.

In addition to such a dialogue support device, research on a translation device for recognizing an input voice, translating the obtained linguistic expression, and further outputting with a voice of a partner language is under way. Due to the problems of translation speed and accuracy, it is the current situation that sufficient dialogue support has not been provided.

[0006]

As described above, the conventional dialogue support device is used for the purpose of understanding the utterance of the other party of the dialogue and retrieving the expression necessary for further responding to the utterance. Was poorly functioning.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a dialogue support device capable of supporting the understanding of the utterance content of the other party and the response to the utterance of the other party. And

[0008]

According to the present invention, a recognition means for recognizing a word included in an input voice, a control means for searching a word associated with the word recognized by the recognition means, and the control are provided. Display means for displaying the words retrieved by the means, and retrieval by the control means
When the displayed word is displayed by the display means, it is already displayed.
Since the display content is determined according to the displayed word, it is possible to refer to the words related to the words used in the conversation, and to utilize them appropriately to conduct the conversation.

Further, since the display means displays the word retrieved by the control means by adding a headline in a language different from the inputted voice, the display means has little knowledge about the language used in conversation. Even so, understanding of words is helped and conversation becomes possible.

[0010]

The present invention further comprises a recognizing means for recognizing a word included in the inputted voice, and a searching means for searching a word associated with the word recognized by the recognizing means.
Input means for designating a type of a word to be displayed among the words retrieved by the control means, and sorting means for sorting the word retrieved by the control means based on the type designated by the input means , And display means for displaying the words sorted by the sorting means,
It is possible to display the contents effective for the conversation support desired by the user.

According to the present invention, based on the recognition means for obtaining at least one recognition candidate for each word included in the input voice and the recognition candidate for each word obtained by the recognition means, For each combination of recognition candidates between words, the validity of the combination is evaluated based on the evaluation value, and a determination means for determining the order of the combination based on the evaluation result, and the recognition candidate determined by the determination means. For each combination of, the search means for searching for a word associated with each recognition candidate, the recognition candidate for the combination determined by the determining means, and the display means for displaying the word searched by the searching means. Therefore, even if there is ambiguity in the result of the voice recognition, the contents effective for supporting the conversation are displayed.

According to such a configuration, not only the words used by the speaker in the utterance but also the words related thereto (related words) are displayed together. Such related words not only help to understand the utterance of the other party, but can also be used when responding to the utterance of the other party and further supplementing the utterance of oneself. Therefore, it becomes possible to proceed with the conversation more smoothly.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a dialogue support device according to the present embodiment.
In the present embodiment, the description will be made assuming that the user whose native language is Japanese has the purpose of supporting a conversation in English.

First, the first embodiment will be described. As shown in FIG. 1, the dialogue support device according to the first exemplary embodiment includes a microphone 1, a recognition unit 2, a control unit 3, a display unit 4, an input unit 5, and a dictionary storage unit 6.

The microphone 1 is used to input the voice of conversation. The microphone 1 outputs the voice waveform data of the input voice to the recognition unit 2. The recognition unit 2 performs a voice recognition process for recognizing a voice based on the voice waveform data from the microphone 1. The recognition unit 2 recognizes a word included in the content uttered by the speaker, for example, by the voice recognition process.

The control unit 3 searches for an associative word for a word (recognition word) recognized by the recognition unit 2 using the knowledge information stored in the dictionary storage unit 6, and displays it on the display unit 4 from among the words. Select a related word to be used. The control unit 3 is provided with a result display list for storing information (recognition words, related words) to be displayed on the display unit 4.

The display unit 4 is used to display the search result obtained by the control unit 3. Generally, a liquid crystal display is used, and it can be realized by a CRT or the like.

The input unit 5 is for inputting various instructions from the user to the dialogue support device, for example, instructions for specifying the type of word to be displayed on the display unit 4. The dictionary storage unit 6 is realized by a magnetic disk device, an optical disk device, or the like, and stores knowledge information for determining the contents to be displayed on the display unit 4. The knowledge information stored in the dictionary storage unit 6 includes, for example, an associative dictionary 6a and a bilingual dictionary 6b. The associative dictionary 6a stores, for example, network information indicating an associative relationship between words stored in the dictionary storage unit 6. The bilingual dictionary 6b stores translated words for each of the words that the device can support.

FIG. 2 is an example of a screen configuration displayed on the display unit 4. A recognition word display area 10 and a related word display area 12 are provided on the display screen. The recognition word display area 10 is
Among the words recognized by the recognition unit 2, this is an area for displaying a search target word specified by, for example, a part of speech. In the first embodiment, among the words obtained as the recognition result in the recognition unit 2, only the word whose part of speech is a noun will be described. Related term display area 1
Reference numeral 2 is an area for displaying a recognized word, that is, a word related to the word displayed in the recognized word display area 10.

In addition, in the example shown in FIG. 2, the recognized word displayed in the recognized word display area 10 and the related word displayed in the related word display area 12 each include a translated word (heading in Japanese). it's shown.

FIG. 2 shows "I have a pai".
n in my stomach. An example of the display immediately after the utterance "" is made is shown. In the first embodiment, since the noun is the display target, "pain" and "sto" are displayed.
“Mach” and its translated word are displayed in the recognized word display area 10.

Further, in the related word display area 12, "sharp" and "dull" are displayed as related words of "pain" displayed in the recognized word display area 10, and "stom" is displayed.
As related words of "ach", "stomachache" and "vomit" are displayed.

Therefore, by referring to the display contents shown in FIG. 2, it is possible to grasp the gist of the utterance contents by the conversation partner, that is, the stomach pain ("pain""stomach"). Pain (“s
It is possible to easily make a response such as "harp""dull") or nausea ("vomit").

FIG. 3 illustrates a part of the contents of the network stored in the associative dictionary 6a in the dictionary storage unit 6. The network shown in FIG. 3 represents an associative relationship between words.

The method of expressing the associative relation between words as a network is, for example, "Kana-Kanji conversion method using the associative function of neural network", 40th Information Processing Society of Japan National Convention (19
90), etc. This discloses a method of expressing the association relation between words as a network and applying it to Kana-Kanji conversion to improve the translation accuracy of homonyms.

The network is composed of a plurality of nodes each having an active value and links connecting them. When configuring the network at the heading level,
Each node corresponds to a word (headline), and the links between the nodes indicate the relationships between the words. The link between nodes can be expressed as a bidirectional link having a weight to express the strength of connection between words (indicated by a numerical value in FIG. 3).

The process corresponding to the association (association process) can be performed by changing the active value of the node (details will be described later). For example, by giving a certain activity value to a node of a word used in a conversation and propagating that value to another node connected by a link according to the weight of the link, the activity value of the other node is changed. Change. Naturally, the activation value of each node is not determined by only one word, but the nodes corresponding to the words appearing in the conversation are sequentially activated, and at the same time, the activation value of each node is lowered over time. As a result, associative processing is performed according to the progress of the conversation, that is, according to the words included in the utterance content, so that it is possible to obtain an appropriate related word according to the context.

Note that various processes have been proposed, including the above-mentioned document, regarding the process corresponding to such an association, and the present invention does not have a feature in the association process itself, and therefore a detailed description thereof will be omitted.

Next, the operation of the dialogue support apparatus according to the first embodiment will be described with reference to the flowchart shown in FIG. First, when starting the processing, the control unit 3
Clears the result display list and the active values of all the nodes of the network stored in the associative dictionary 6a (step S1). The structure of the result display list will be described later.

After that, the voice of the conversation is input and a series of processes for supporting the dialogue is performed. First, the voice uttered by the speaker is input from the microphone 1, and the voice data is output to the recognition unit 2 (step S2). The recognition unit 2 performs a recognition process on the voice data from the microphone 1 to recognize a word included in the utterance content (step S3).

The recognition unit 2 recognizes the result obtained by the recognition process, that is, the English word, as shown in FIG. 5, with the headword (word) and its part-of-speech pair registered as one element, and the number of recognized words. It is passed to the control unit 3 as an array having elements of.

Incidentally, FIG. 5 shows "I have a pai".
n in my stomach. It shows the data of the array corresponding to the recognition result of the utterance content ".
The control unit 3 processes the words obtained by the recognition by the recognition unit 2 one by one. Therefore, the recognition unit 2 sets 1 to the variable i for managing the word to be processed.
Is set, and the process is started from the first word (step S4). The value of the variable i is sequentially updated in step S12 every time the processing for one word is completed.

The control unit 3 checks in step S5 whether or not the value of the variable i is less than or equal to the recognized number of words. If the value of the variable i is larger than the recognized number of words, the control unit 3 determines that all the words have been processed and returns to step S2.

The control unit 3 determines whether or not the word indicated by the variable i is a noun, from the part of speech corresponding to the same word (headline) registered in the array from the recognition unit 2 (step S).
6). If it is not a noun, the value of the variable i is incremented by 1 in step S12 in order to move to the processing of the next word, and the process returns to step S5.

On the other hand, when the part of speech of the target word is a noun, the control unit 3 updates the result display list (step S7). FIG. 6 shows an example of the data structure of the result display list held in the control unit 3.
have a pain in my stomac
h. It shows the contents of data registered when the processing for the utterance "" is performed. However, the data is the data at the time when the processing for all words is completed.

In this case, since the nouns among the words recognized during utterance are "pain" and "stomach",
These recognition words and the related words corresponding to each recognition word obtained by the associative processing described later are stored.

The number of words that can be displayed on the display unit 4 depends on the sizes of the recognition word display area 10 and the related word display area 12. If the number of rows on the screen is N and N pieces of data are already stored in the result display list, no more data can be displayed. (However, here, as shown in FIG. 2, the line displaying the English word and the line for the Japanese translation displayed under each word are combined into one line.) Therefore, the update in step S7 In the processing, the number of data stored in the result display list is checked, and if the number is the maximum displayable number N, an area for registering the word (recognition word) currently being processed and the related word. In order to ensure that, the first element of the list is deleted and the second to N-th elements are changed from the first to the (N-1) -th, respectively.

Next, the control section 3 carries out an associative process for the word (recognition word) indicated by the variable i which is the current processing target. In the associative process, a certain active value is given to a node on the network, which is stored in the associative dictionary 6a and corresponds to the currently focused i-th word, and the active value is given to another node connected by a link. This is done by propagating to a node and changing the active value of another node.

Specifically, the activation value Oj of each node is calculated as follows. Oj ← f (nj) nj ← (1-δ) nj + δ (ΣwijOi + Ij) f: monotonically increasing function δ: real number representing the inertia of the network (0 <δ <1) nj: input value to node j Ij: to node j External input wij: weight of link from node i to node j (however, wij = wji, wii = 0) For example, when associating the word “pain”, the active value of the node “pain” is set. At the time of calculation, the activity value is calculated by giving a value of 0 or more to Ij. This calculation is repeated for all nodes on the network until the change in the active value becomes less than a certain value. The above-mentioned association process is performed in the same manner when the word “stomach” whose part of speech included in the utterance content is a noun is processed.

The control unit 3 takes out, as a related word candidate, a word corresponding to a node having an active value of a certain value or more on the network stored in the associative dictionary 6a (step S).
9). Some of these candidates are recognized words themselves, or some are already included in the result display list, but they are removed from the candidates.

The control unit 3 stores the recognition word and the related word obtained in step S9 in the result display list (step S10). Then, the control unit 3 determines, based on the content of the data stored in the display list, translation words corresponding to the recognition word and the related word, and the recognition word and the related word, respectively.
2 is used to display the image on the display unit 4 as shown in FIG. 2 (step S11).

In this way, the word obtained by voice recognition of the speech content of the conversation partner and the word (related word) associated with the word are displayed on the display screen of the display unit 4 as needed. By referring to the content displayed in the recognition word display area 10, the utterance content of the conversation partner can be grasped, and by referring to the content displayed in the related word display area 12, the utterance content of the partner is displayed. Can be responded to.

Further, even if the conversation partner has little knowledge about the language used (English in this case), the translated words in Japanese are added to the recognition words and related words displayed on the display unit 4. Therefore, it is possible to have a conversation by appropriately using them.

Next, the dialogue support device of the second embodiment will be described. In the above-described first embodiment, the description has been made on the premise that the recognition unit 2 can accurately recognize the utterance content, but the recognition unit 2 always performs accurate voice recognition due to the influence of the speaker's pronunciation and the environment. In some cases, the voice recognition result contains ambiguity. The second embodiment is
It has an effective function when such a voice recognition result contains ambiguity.

Since the configuration of the dialogue support device in the second embodiment is almost the same as that of the first embodiment, the description thereof will be omitted.
In the second embodiment, the control unit 3 further executes the following processing.

Here, the utterance content similar to that of the first embodiment, "I have a painin my sto" is used.
match. Assuming that the utterance "" is made, the recognition unit 2 recognizes "pai" for the noun phrase "pain".
"n" and "point" are output as recognition candidates, and "s"
"tomach" and "stomach" and "sto"
It is assumed that "re" is output as a recognition candidate.

In such a case, the user cannot fully understand what the other party is talking by simply listing the output recognition candidates and their related words, so that the dialogue can proceed smoothly. It will be difficult. Therefore, the control unit 3 reduces ambiguity by the following method.

First, the control section 3 outputs recognition candidates of the outputted noun phrase (here, "pain", "paint" /
In order to check the relationship between "stomach" and "store"), the associative dictionary 6a in the dictionary storage unit 6 is used to calculate the activation value for each noun set by the associative process described above.

For example, if a network is constructed as shown in FIG. 7, possible combinations among the recognition candidates are "pain" and "stomach".
"Pain" and "store", "paint" and "s"
Three types of “tre” are conceivable, and the association value of each set can be represented by the activity value (the combination of “paint” and “stomach” is excluded because it has low association).

The control section 3 evaluates the validity of each combination, for example, using the magnitude of the activity value and the similarity of each word at the time of voice recognition as the evaluation value. That is, the control unit 3 determines the priority when outputting the combinations of three types of words, that is, the order of combinations that are expected to be correct as a recognition result. Here, for example, processing such as determining the priority as a product of the activity value and the similarity of each word, or multiplying the similarity by a weight α and the activity value by a weight (1-α) to determine the priority by the sum thereof Is possible.

Further, as a method of selecting a related word, the control unit 3 uses the active value to preferentially output the word when there is a word related to the noun phrase of the other party forming the set. Is processed. For example, the same "stor in the above example
If "e" is also paired with "pain", "s"
The related term of "tore" is "drugstore"
If "medicine" or "medicine" is selected and paired with "paint", "stationary", "department store", etc. are displayed as related words.

FIG. 8 shows a display example on the display unit 4 in the second embodiment. As shown in FIG. 8, priority is given to the combination of three types of words (evaluation result of validity).
In accordance with the above, the words are displayed in the recognized word display area 10 as (1), (2) and (3) in order. In addition, the related words obtained according to each combination of words are displayed in the related word display area 12.

Thus, even if the voice recognition result by the recognition unit 2 is ambiguous, the validity of the combination of voice recognition candidates is evaluated based on the evaluation value, and the priority of the combination ( By determining (ranking) and displaying the related words corresponding to each in the order according to the priority, it is possible to know what kind of content the partner is talking in the order of high possibility. Therefore, it is extremely effective in determining how the user will respond next. For example, it is possible to hear back about which topic is being talked about, and it is also possible to reply to a topic with the highest possibility, which can significantly reduce the burden on the user.

The present invention is not limited to the above embodiment. For example, in the embodiment, English is targeted, but this may be any language. Also, the translation displayed below English does not have to be Japanese, and may not necessarily be displayed.

Further, in the present embodiment, only the words recognized by voice are input, but it is also possible to change the association process depending on who is the speaker of the recognized voice. For example, the link in the associative network has only one weight per link, but different weights are assigned according to speakers, and in the associative process, the weight to be used is changed depending on who spoke the word. Can perform associative processing. As a result, in the case of one's own utterance, it is possible to increase the associativity to the synonyms, or to increase the associativity to the words corresponding to the superordinate concept and the subordinate concept in the case of the utterance of the other party.

Further, in the present embodiment, only the noun among the recognized words is displayed, but other parts of speech can be displayed in the same manner. Further, by specifying the part of speech of the word to be displayed from the input unit 5, it is easy to expand so that the user can appropriately change the type. As a result, the contents effective for the conversation support desired by the user can be displayed. Further, although the type of word to be displayed is set for each word based on the part of speech, the type of word may be designated by another classification. Others The present invention can be appropriately modified and implemented without departing from the spirit thereof.

[0058]

As described above in detail, according to the present invention, the words recognized by voice and the related words associated with the words are displayed on the display screen at any time. By referring to, it is possible to grasp the utterance content of the other party of the dialogue, and further appropriately utilize them to make a response to smoothly recommend the conversation.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a dialogue support device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of a screen configuration displayed on a display unit 4 in FIG.

FIG. 3 is a diagram showing an example of a network stored in an associative dictionary 6a in FIG.

FIG. 4 is a flowchart for explaining the operation of the dialogue support device in the first embodiment.

5 is a diagram showing an example of voice recognition result data obtained by a recognition unit 2 in FIG.

FIG. 6 is a diagram showing an example of a data structure of a result display list held in a control unit 3 in FIG.

FIG. 7 is a diagram showing an example of a network according to a second embodiment.

FIG. 8 is a diagram showing a display example on a display unit 4 in the second embodiment.

[Explanation of symbols]

1 ... Microphone 2 ... Recognition unit 3 ... Control unit 4 ... Display 5 ... Input section 6 ... dictionary storage 6a ... associative dictionary 6b ... bilingual dictionary

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI G10L 15/28 G10L 3/00 537G (56) References JP-A-4-42363 (JP, A) JP-B 2-52292 ( JP, B2) Patent 3383485 (JP, B2) Patent 2849342 (JP, B2) International Telecommunication Basic Research Institute Co., Ltd., ATR Advanced Technology Series Automatic Translation Phone, Japan, Ohmsha Co., Ltd., January 30, 1994 , P. 1-4 (58) Fields investigated (Int.Cl. ⁷ , DB name) G10L 15/00-15/28 G06F 17/28 JISST file (JOIS)

Claims (4)

(57) [Claims] 1. A recognition unit that recognizes a word included in an input voice, a control unit that searches for a word associated with the word recognized by the recognition unit, and a word that is searched by the control unit. comprising a display means for, has been a word retrieved by said control means to said display means
When displaying more, it will be displayed according to the words already displayed.
A dialogue support device characterized by determining contents to be shown . 2. The dialogue support device according to claim 1, wherein the display unit displays the word searched by the control unit by adding a headline in a language different from the input voice. . 3. Recognizing means for recognizing a word included in the input voice, control means for searching a word associated with the word recognized by the recognizing means, and among the words searched by the control means. Input means for inputting the designation of the type of word to be displayed, sorting means for sorting the words retrieved by the control means based on the type input by the input means, and sorting by the sorting means A dialogue support apparatus comprising: a display unit for displaying a word. 4. A recognition unit that obtains at least one recognition candidate for each of the words included in the input voice, and an inter-word space based on the recognition candidates for each word obtained by the recognition unit. For each combination of recognition candidates, the validity of the combination is evaluated based on the evaluation value, and the determination means for determining the rank of the combination based on the evaluation result, and each combination of the recognition candidates determined by the determination means The search means for searching for a word associated with each recognition candidate, the recognition candidate for the combination determined by the determining means, and the display means for displaying the word searched by the searching means. Characteristic dialogue support device.