JP7728878B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, voice recognition systems that are executed on vehicle navigation devices are known. For example, Patent Document 1 discloses a method for correcting erroneous recognition results when a voice recognition engine installed in a voice recognition system erroneously recognizes spoken voice.

Japanese Patent Application Laid-Open No. 2004-333703

However, the above-mentioned conventional technology does not always provide the control to perform the correct action in response to the user's voice input.

For example, in the above-mentioned conventional technology, when a speech recognition engine detects a misrecognition of spoken speech, words that the user has previously corrected for the misrecognized word are read from the recognized word link DB and presented as correct answer candidates, and the misrecognized word and the correct word corrected by the user are associated and newly registered in the recognized word link DB.

In this way, the above-mentioned conventional technology registers words in a dictionary so that even if the voice recognition engine misrecognizes a word spoken by a user, the correct word for the misrecognized word can be recognized by the voice recognition engine; however, this process does not correctly recognize mistakes made by the user.

Therefore, with the above-mentioned conventional technology, it is not always possible to control the system so that the correct action is taken in response to a user's mistake, even if the user makes a mistake. Furthermore, with the above-mentioned conventional technology, if the words spoken by the user differ from the words registered as operation actions, the voice recognition engine will not be able to correctly recognize the operation intended by the user.

The present invention has been made in consideration of the above, and aims to provide an information processing device, an information processing method, and an information processing program that can be controlled to perform the correct operation in response to a user's voice input.

The information processing device described in claim 1 is characterized in that, when a second input operation is performed to input information by touching a predetermined object after a first utterance is input, the information processing device has a determination unit that determines whether the second input operation is a correction operation for correcting the utterance content based on the utterance content indicated by the first utterance and the operation content indicated by the second input operation; a linking unit that links the operation content with the utterance content when the determination unit determines that the second input operation is the correction operation; and an information control unit that performs predetermined control on the utterance content based on the linking result by the linking unit.

The information processing method described in claim 9 is an information processing method executed by an information processing device, and is characterized in that it includes: a determination step, when a second input operation is performed to input information by touching a predetermined object after a first utterance is input, determining whether the second input operation is a correction operation for correcting the utterance content based on the utterance content indicated by the first utterance and the operation content indicated by the second input operation; a linking step, when the determination step determines that the second input operation is the correction operation, linking the operation content with the utterance content; and an information control step, performing predetermined control on the utterance content based on the linking result of the linking step.

The information processing program described in claim 10 is an information processing program for causing an information processing device to execute the following steps: when a second input operation is performed to input information by touching a predetermined object after a first utterance is input, a determination step for determining whether the second input operation is a correction operation for correcting the utterance content based on the utterance content indicated by the first utterance and the operation content indicated by the second input operation; when the determination step determines that the second input operation is the correction operation, a linking step for linking the operation content with the utterance content; and an information control step for performing predetermined control on the utterance content based on the linking result of the linking step.

FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment. FIG. 2 is an explanatory diagram illustrating information processing according to the first embodiment. FIG. 3 is a diagram illustrating an example of the configuration of the information processing apparatus according to the first embodiment. FIG. 4 is a diagram illustrating an example of an utterance information database according to the first embodiment. FIG. 5 is a diagram illustrating an example of an association information database according to the embodiment. FIG. 6 is a diagram illustrating an example of a user dictionary database according to the embodiment. FIG. 7 is a flowchart showing the procedure of information processing according to the first embodiment. FIG. 8 is an explanatory diagram illustrating information processing according to the second embodiment. FIG. 9 is a diagram illustrating an example of the configuration of an information processing apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of an operation information database according to the second embodiment. FIG. 11 is a flowchart showing the procedure of information processing according to the second embodiment. FIG. 12 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device.

Below, an example of a form for implementing an information processing device, an information processing method, and an information processing program (hereinafter referred to as an "embodiment") will be described in detail with reference to the drawings. Note that the information processing device, information processing method, and information processing program are not limited to this embodiment. Furthermore, identical components in the following embodiments will be given the same reference numerals, and duplicate explanations will be omitted.

[Embodiment]
(Common features in each embodiment)
1. Introduction
For example, a vehicle navigation device may be equipped with a voice recognition system that recognizes speech input by a user and performs information processing (e.g., route guidance) according to the recognition result. In such cases, the user inputs speech to the navigation device, such as a speech instructing the navigation device to perform a specific action or a speech indicating a destination, but the user may make a mistake and input a speech that is different from the intended content. In such cases, the speech recognition system will execute an action according to the mistaken content, which is inconvenient for the user.

The present invention was developed with the above-mentioned circumstances in mind, and its purpose is to provide control so that the correct action can be taken in response to a user's slip-up, even if the user has made a mistake. To achieve this purpose, the present invention infers the user's intention to correct the slip-up from their actions. Then, based on the content of the actions taken with the intention of correcting the slip-up, the present invention links the misspoken content with the correct content for that content, and then performs processing according to the linking results.

Here, information processing corresponding to the present invention can be divided into information processing that detects the correction voice of a user attempting to correct the content of a slip of the tongue using spoken voice, and performs linking based on the content of the detected correction voice, and information processing that detects the correction operation of a user attempting to correct the content of a slip of the tongue using an input means other than spoken voice (e.g., manual input), and performs linking based on the content of the detected correction operation. Therefore, in the following embodiments, the former information processing will be described as the first embodiment, and the latter information processing will be described as the second embodiment.

[2. Overview of the system]
Before specifically describing the first and second embodiments, the configuration of an information processing system according to the embodiments will be described first as a feature common to both embodiments. FIG. 1 is a diagram illustrating an example of an information processing system according to the embodiments. FIG. 1 illustrates an information processing system 1 as an example of an information processing system according to the embodiments. The first and second embodiments described below may be realized within the information processing system 1 illustrated in FIG. 1.

As shown in FIG. 1, the information processing system 1 may include a terminal device 10 and an information processing device 100. The terminal device 10 and the information processing device 100 are connected to each other via a network N so that they can communicate with each other via a wired or wireless connection. The information processing system 1 shown in FIG. 1 may include any number of terminal devices 10 and any number of information processing devices 100.

The terminal device 10 may be an on-board device mounted on a vehicle, which is an example of a moving body. Figure 1 shows an example in which the terminal device 10 is an on-board device of a vehicle VEx. In such an example, the terminal device 10 may be, for example, a dedicated navigation device built into the vehicle VEx, or a dedicated navigation device attached to the vehicle VEx.

The terminal device 10 may also be configured to function as the information processing device 100 described below. For example, while the terminal device 10 and the information processing device 100 are shown as separate devices in FIG. 1, the terminal device 10 and the information processing device 100 may be integrated into one information processing device. In such a case, for example, some or all of the functions of the information processing device 100 may be implemented in the terminal device 10.

The terminal device 10 may be a portable terminal device (e.g., a smartphone, tablet device, notebook PC, desktop PC, PDA, etc.) that has an application corresponding to a specific navigation system installed. In such an example, the terminal device 10 may be one that is used daily by, for example, the driver of the vehicle VEx.

The terminal device 10 may also have a sound collection unit (e.g., a microphone) that collects the user's speech. Speech information indicating the speech collected via the sound collection unit may then be transmitted by the terminal device 10 to the information processing device 100.

The terminal device 10 may also have various sensors such as a camera, an acceleration sensor, a gyro sensor, a GPS sensor, and an air pressure sensor. Sensor information detected by the sensors may be transmitted by the terminal device 10 to the information processing device 100. The vehicle VEx may also have, for example, a sensor for a safe driving system, and sensor information from this sensor may also be transmitted to the information processing device 100.

The information processing device SV is a device that performs information processing according to the embodiment. For example, the information processing device SV may perform information processing according to the embodiment in accordance with an information processing method realized by an information processing program according to the embodiment.

Furthermore, for example, as information processing relating to the first embodiment, when a second utterance is input after a first utterance is input, the information processing device SV determines whether the second utterance is an utterance input to correct the first utterance content based on the first utterance content indicated by the first utterance and the second utterance content indicated by the second utterance.

Specifically, the information processing device SV estimates the user's intention, that is, whether the first utterance content is an error due to a slip of the tongue and whether the user input the second utterance with the intention of correcting this error. In other words, the information processing device SV estimates whether the user intends to correct the first utterance content with the second utterance content, based on the first utterance content indicated by the first utterance content and the second utterance content indicated by the second utterance content. Then, based on the estimation result, the information processing device SV determines whether the second utterance is correction speech input to correct the first utterance content.

In addition, if the information processing device SV determines that the second utterance is a correction voice input to correct the first utterance content, it links the first utterance content with the second utterance content and performs predetermined control over the first utterance content based on the linking result.

On the other hand, as information processing relating to the second embodiment, when a second input operation (e.g., a manual input operation) is performed to input information by touching a specified object after a first utterance is input, the information processing device SV determines whether the second input operation is a correction operation for correcting the utterance content based on the utterance content indicated by the first utterance and the operation content indicated by the second input operation.

Specifically, the information processing device SV estimates the user's intention by determining whether the spoken content is an error due to a slip of the tongue and whether the user performed the second input operation with the intention of correcting the error. In other words, the information processing device SV estimates whether the user intends to correct the spoken content by the operation content based on the spoken content indicated by the first spoken voice and the operation content indicated by the second input operation. Then, based on the estimation result, the information processing device SV determines whether the second input operation is a correction operation for correcting the spoken content.

In addition, when the information processing device SV determines that the second input operation is a correction operation, it links the operation content with the utterance content and performs predetermined control over the utterance content based on the linking result.

Here, if the terminal device 10 is an edge computer that performs edge processing near the user, the information processing device SV may be, for example, a cloud computer that performs processing on the cloud side. In other words, the information processing device SV may be a server device.

The following provides a specific description of the first and second embodiments. Note that the information processing device SV that performs information processing according to the first embodiment will be referred to as the "information processing device 100." Furthermore, the information processing device SV that performs information processing according to the second embodiment will be referred to as the "information processing device 200."

Furthermore, in each embodiment, the moving body is described as a vehicle VEx, but the moving body is not limited to a vehicle VEx. Furthermore, the user shown in each embodiment may be any person who has boarded a vehicle VEx and made voice input to the terminal device 10. For example, the user may be a person who uses the vehicle VEx on a daily basis, i.e., the owner of the vehicle VEx.

(First embodiment)
1. Overview of the First Embodiment
The first embodiment will now be described with reference to Fig. 2. Fig. 2 is an explanatory diagram illustrating information processing according to the first embodiment.

Figure 2 shows a scene in which user U1 is inputting speech by speaking into terminal device 10 mounted on vehicle VE1 (an example of vehicle VEx). More specifically, Figure 2 shows a scene in which user U1 is inputting speech instructing to be guided along a route to "destination XX located in Ibaraki City, Osaka Prefecture."

In such a case, each time the terminal device 10 receives input of a speech voice, it transmits speech information indicating the received speech voice to the information processing device 100. As a result, the information processing device 100 acquires the speech information from the terminal device 10 (step S11).

For example, Figure 2 shows an example in which user U1 inputs a speech VO11 with content C11 such as "Please guide me to XX in Ibaragi City!" In this example, in response to the input of the speech VO11, the terminal device 10 transmits audio information indicating the speech content C11 to the information processing device 100. As a result, the information processing device 100 acquires the audio information indicating the speech content C11.

Here, assume that user U1 realizes that he/she mistakenly said "Ibaragi" when he/she should have said "Ibaraki." Then, as shown in FIG. 2, user U1 re-inputs the spoken voice VO12 of content C12, such as "Please guide me to XX in Ibaraki City!" In this example, in response to the input of the spoken voice VO12, the terminal device 10 transmits audio information indicating the spoken content C12 to the information processing device 100. As a result, the information processing device 100 acquires the audio information indicating the spoken content C12.

Next, the information processing device 100 acquires first speech information indicating the first speech and second speech information indicating the second speech based on the context of the timing of the speech made by user U1 (step S12). For example, the information processing device 100 may recognize the first speech, which is the speech input first, and the second speech, which is the speech input after the first speech, based on the context of the timing of the speech. Furthermore, the information processing device 100 may thereby acquire the first speech information indicating the first speech and the second speech information indicating the second speech from the speech information that has been collected so far via the terminal device 10.

In the example of Figure 2, the information processing device 100 recognizes the speech voice VO11 as the first speech voice, thereby acquiring speech information indicating the speech content C11 as the first speech information. Also, in the example of Figure 2, the information processing device 100 recognizes the speech voice VO12 as the second speech voice, thereby acquiring speech information indicating the speech content C12 as the second speech information. Hereinafter, the speech voice VO11 may be referred to as the "first speech voice VO11," and the speech voice VO12 may be referred to as the "second speech voice VO12."

Next, the information processing device 100 performs an intention analysis to estimate the intention of correcting the mistake based on the similarity between the first keywords, which are the keywords that make up the first speech information (first utterance content), and the second keywords, which are the keywords that make up the second speech information (second utterance content) (step S13). Specifically, the information processing device 100 estimates the intention of user U1 to correct the first utterance content using the second utterance content based on the similarity between the first keywords and the second keywords. The specific method of intention analysis performed in step S13 will be described later.

In step S13, the information processing device 100 may perform morphological analysis on the text representing the first audio information to extract each word constituting this text as a first keyword. Similarly, the information processing device 100 may perform morphological analysis on the text representing the second audio information to extract each word constituting this text as a second keyword.

Next, based on the estimation results of the intention analysis, the information processing device 100 determines whether the second utterance VO12 is a correction speech input to correct the first utterance content corresponding to the first utterance VO11 (step S14). For example, if the information processing device 100 recognizes similarity between "Ibaragi," one of the first keywords, and "Ibaraki," one of the second keywords, it can infer that user U1 intends to correct the first utterance content (first keyword KW11) using the second utterance content (second keyword KW12). As a result, the information processing device 100 can determine that the second utterance VO12 is a correction speech input to correct the first utterance content.

In this way, if the information processing device 100 determines that the second speech VO12 is a correction speech input to correct the first speech content, the information processing device 100 associates the second keyword "Ibaraki" with the first keyword "Ibaragi," with the second keyword "Ibaraki" as the correct answer and the first keyword "Ibaragi" as the incorrect answer (step S15). Figure 2 shows examples of association results resulting from user U1's past mispronunciations of "Ibaraki" as "Ibaragi" and "Ibaraki" as "Ibaraku," including the current example in which the second keyword "Ibaraki" is the correct answer and the first keyword "Ibaragi" is the correct answer. Furthermore, such association results may be managed in the association information database 122 (Figure 5) using an association ID.

In addition, the information processing device 100 uses pairs of correct answer information and erroneous information as learning data to learn which of the first keywords indicated by the erroneous information are likely to be confused with the second keyword indicated by the correct answer information (step S16). In the example of Figure 2, the information processing device 100 learns which of the first keywords "Ibaragi" and "Ibaraku" are likely to be confused with the second keyword "Ibaraki."

Then, the information processing device 100 registers the keyword in the dictionary based on the learning results (step S17). For example, based on the learning results, the information processing device 100 registers the first keyword in the user dictionary (Figure 6) so that when a speech containing a keyword that is easily confused with a second keyword is input, the input first keyword will be recognized as the second keyword. Figure 2 shows an example in which the information processing device 100 registers the first keyword "Ibaragi" in the user dictionary so that when a speech containing the first keyword "Ibaragi" is input, the first keyword "Ibaragi" will be recognized as the second keyword "Ibaraki."

As explained above with reference to FIG. 2, in the first embodiment, when a second utterance is input after a first utterance, the information processing device 100 determines whether the second utterance is input to correct the first utterance based on the first speech content indicated by the first utterance and the second speech content indicated by the second utterance. If the information processing device 200 determines that the second utterance is input to correct the first utterance, it links the first utterance content with the second utterance content and registers the first utterance content in the user dictionary based on the linking result.

According to the information processing of the first embodiment, the information processing device 100 can be controlled so that the correct action is taken in response to the mistake even if the user makes a mistake.

2. Configuration of information processing device
From here, the information processing device 100 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the information processing device 100 according to the first embodiment. As shown in Fig. 3, the information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

(Regarding the communication unit 110)
The communication unit 110 is realized by, for example, a network interface card (NIC), etc. The communication unit 110 is connected to a network via a wired or wireless connection, and transmits and receives information to and from the terminal device 10, for example.

(Regarding the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 has an utterance information database 121, a linking information database 122, and a user dictionary database 123.

(Regarding the utterance information database 121)
The speech information database 121 stores information about speech input by a user. An example of the speech information database 121 according to the first embodiment is shown in Fig. 4. In the example of Fig. 4, the speech information database 121 has items such as "user ID,""utterance date and time," and "speech information."

"User ID" indicates identification information that identifies the user who inputs the spoken voice to the terminal device 10. For example, the information processing device 100 may recognize the user who inputs the spoken voice based on an image captured by a sensor (e.g., a camera) possessed by the terminal device 10, and assign a "user ID" to the recognized user.

"Speech date and time" indicates information related to the date and time when the utterance was input. Figure 4 shows an example in which user ID "U1" is associated with "Speech date and time #11." This example shows an example in which user U1 inputs speech at the date and time of speech #11. For example, by regarding the "Speech date and time" as the timing of the speech, the information processing device 100 can recognize a first speech, which is the speech input earlier, and a second speech, which is the speech input after the first speech.

"Voice information" is information indicating the content of the spoken voice input by the user identified by the "user ID." For example, "voice information" may be text information obtained by applying any voice recognition technology to the spoken voice, and may be text information indicating the content of the spoken voice. Note that the voice recognition process for the spoken voice may be performed by the terminal device 10, or may be performed by a voice recognition device not shown in Figure 1. Figure 4 shows an example in which user ID "U1," "speech date and time #11," and "voice information #11" are associated. This example shows an example in which voice information #11 indicating the content of the spoken voice is obtained from the spoken voice input by user U1 at the date and time of speech #11.

(Regarding the association information database 122)
The linked information database 122 links and manages correct answer information and error information. Fig. 5 shows an example of the linked information database 122 according to the embodiment. In the example of Fig. 5, the linked information database 122 has items such as "user ID,""linkingID,""correct answer information," and "error information."

"User ID" indicates identification information that identifies the user who inputs the spoken voice to the terminal device 10, and corresponds to "User ID" in Figure 4.

The "linking ID" is identification information for managing "incorrect information" for each keyword indicated by the "correct information." As shown in Figure 5, a "linking ID" may be assigned for each keyword indicated by the "correct information." Figure 5 shows an example in which a user ID "U1," a linking ID "H11," and correct information "Ibaraki" are associated with each other. This example shows an example in which a second keyword, "Ibaraki," as correct information is managed by the linking ID "H11" in response to the input of speech by user U1.

"Correct answer information" is information indicating the correct second keyword that corrects a specific first keyword, which is a second keyword contained in a second utterance that is input as correction speech in order to correct a specific first keyword among the first keywords contained in the first utterance.

"Error information" is information indicating the keyword among the first keywords contained in the first utterance that is to be corrected by the second keyword contained in the second utterance input as the correction voice.

Figure 5 shows an example in which user ID "U1," linking ID "H11," correct answer information "Ibaraki," and incorrect information "Ibaragi" are associated. This example shows the linking result when user U1 mistakenly says "Ibaragi" when the correct answer is "Ibaraki," and shows an example in which this linking result is managed using linking ID "H11."

Figure 5 also shows an example in which user ID "U1," linking ID "H11," correct answer information "Ibaraki," and incorrect information "Ibaraku" are associated. This example shows the linking result when user U1 mistakenly says "Ibaraku" when the correct answer is "Ibaraki," and shows an example in which this linking result is managed using linking ID "H11."

Figure 5 also shows an example in which user ID "U1," linking ID "H11," correct answer information "guidance interrupted," and incorrect information "guidance canceled" are associated. This example shows the linking result resulting from user U1 mistakenly saying "guidance canceled" when the correct answer should be "guidance interrupted," and shows an example in which this linking result is managed using linking ID "H12."

As explained in Figure 2, the pair of "correct answer information" and "error information" is used as learning data for learning patterns of keywords that are likely to be confused with the second keyword indicated by the "correct answer information" among the first keywords indicated by the "error information."

(Regarding the user dictionary database 123)
The user dictionary database 123 stores first keywords and second keywords in association with each other so that a first keyword that is easily mistaken for a second keyword can be recognized as the second keyword. An example of the user dictionary database 123 according to the embodiment is shown in FIG. 6. In the example of FIG. 6, the user dictionary database 123 has items such as "user ID,""utteredkeyword," and "recognized keyword."

"User ID" indicates identification information that identifies the user who inputs the spoken voice to the terminal device 10, and corresponds to "User ID" in Figures 4 and 5.

"Spoken keyword" refers to a first keyword that is estimated, as a result of learning using training data, to be prone to being mistaken for a second keyword indicated by a "recognition keyword." Furthermore, "spoken keyword" corresponds to condition information that conditions the first keyword to be recognized as the second keyword indicated by the "recognition keyword" when spoken speech containing this first keyword is input.

The "recognition keyword" corresponds to condition information that conditions the correct keyword that should be recognized for the first keyword indicated by the "spoken keyword" when spoken speech containing the first keyword is input.

Figure 6 shows an example in which the spoken keyword "Ibaragi" and the recognized keyword "Ibaraki" are associated with user ID "U1." This example shows an example in which the first keyword "Ibaragi" and the second keyword "Ibaraki" are registered in the user dictionary of user U1 in a state in which they are associated with each other, so that when user U1 inputs spoken audio containing the first keyword "Ibaragi," the first keyword "Ibaragi" is recognized as the second keyword "Ibaraki."

Figure 6 shows an example in which the spoken keyword "guidance canceled" and the recognized keyword "guidance interrupted" are associated with user ID "U1." This example shows an example in which the first keyword "guidance canceled" and the second keyword "guidance interrupted" are registered in the user dictionary of user U1 in a state in which they are associated with each other, so that when user U1 inputs a spoken voice containing the first keyword "guidance canceled," the first keyword "guidance canceled" is recognized as the second keyword "guidance interrupted."

(Regarding the control unit 130)
3, the control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs (e.g., the information processing program according to the embodiment) stored in a storage device inside the information processing device 100 using RAM as a work area. The control unit 130 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the control unit 130 has an acquisition unit 131, a corrected voice determination unit 132, a detection unit 133, a linking unit 134, a learning unit 135, and an information control unit 136, and realizes or executes the information processing functions and actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationships between the processing units of the control unit 130 are not limited to the connection relationships shown in FIG. 3, and may be other connection relationships.

(Regarding the Acquisition Unit 131)
The acquisition unit 131 acquires various pieces of information used in the information processing according to the first embodiment. Furthermore, the acquisition unit 131 may output the acquired information to an appropriate processing unit that performs processing using this information.

For example, when a user inputs a speech sound into the terminal device 10, the acquisition unit 131 acquires speech information indicating the input speech sound. For example, when the speech information is generated by the terminal device 10, the acquisition unit 131 may acquire the speech information from the terminal device 10. Also, for example, when the speech information is generated by a speech recognition device (not shown), the acquisition unit 131 may acquire the speech information from the speech recognition device.

The acquisition unit 131 may also acquire first speech information indicating the first speech speech and second speech information indicating the second speech speech. For example, the acquisition unit 131 may recognize the first speech speech, which is the speech speech input first, and the second speech speech, which is the speech speech input after the first speech speech, based on the context of the speech timing. In addition, the acquisition unit 131 may thereby acquire the first speech information indicating the first speech speech and the second speech information indicating the second speech speech from the speech information that has been collected so far (speech information stored in the speech information database 121).

(Regarding the corrected speech determination unit 132)
When a second speech is input after a first speech is input, the correction speech determination unit 132 determines whether the second speech is a speech input to correct the first speech content based on the first speech content indicated by the first speech and the second speech content indicated by the second speech.

For example, the correction speech determination unit 132 estimates the user's intention by determining whether the first utterance content is an error due to a slip of the tongue and whether the user input the second utterance with the intention of correcting this error. In other words, the correction speech determination unit 132 estimates whether the user intends to correct the first utterance content using the second utterance content based on the first utterance content indicated by the first utterance content and the second utterance content indicated by the second utterance content. Then, based on the estimation result, the correction speech determination unit 132 determines whether the second utterance is correction speech input to correct the first utterance content. For example, if the estimation result indicates that the user intends to correct the first utterance content using the second utterance content, the correction speech determination unit 132 can determine that the second utterance is correction speech input to correct the first utterance content.

Below, we will show a specific example of intention analysis in which we infer whether the user intends to correct the first utterance based on the second utterance, and based on the inference result, we determine whether the second utterance is a correction voice input to correct the first utterance.

For example, the corrected speech determination unit 132 may perform morphological analysis on text indicating a first speech content (first speech information) and extract each word constituting this text as a first keyword. Similarly, the corrected speech determination unit 132 may perform morphological analysis on text indicating a second speech content (second speech information) and extract each word constituting this text as a second keyword.

The corrected voice determination unit 132 may then detect similarity for each combination that is formed between, for example, one first keyword and one second keyword, and determine, based on the detected similarity, whether the second utterance is corrected voice input to correct the content of the first utterance.

As a first example, the corrected speech determination unit 132 may detect similarities in pronunciation. For example, for each combination between a first keyword and a second keyword, the corrected speech determination unit 132 may calculate a similarity that indicates how similar the first keyword and the second keyword included in this combination are in terms of characters (pronunciation).

As a specific example, the corrected speech determination unit 132 may calculate a similarity indicating the similarity between the sequence of vowels in the first keyword and the sequence of vowels in the second keyword. As another example, the corrected speech determination unit 132 may calculate a similarity indicating the similarity between the sequence of consonants in the first keyword and the sequence of consonants in the second keyword.

Then, if there is a combination whose similarity exceeds a predetermined value, the correction voice determination unit 132 may determine that the second utterance voice is a correction voice input to correct the content of the first utterance.

As a second example, the corrected speech determination unit 132 may detect semantic similarity. For example, for each combination between a first keyword and a second keyword, the corrected speech determination unit 132 may calculate a similarity that indicates how similar the first keyword and the second keyword included in this combination are in terms of characters (meaning).

For example, suppose the first keyword is "Grandpa's house" and the second keyword is "Grandpa's home." In this example, the corrected speech determination unit 132 may calculate a similarity index indicating how similar the first keyword "Grandpa's house" and the second keyword "Grandpa's home" are in terms of characters (readings), and then weight the calculated similarity index according to how similar the meanings of "Grandpa's house" and "Grandpa's home" are. As an example, the corrected speech determination unit 132 may calculate a similarity index indicating how similar the meanings of "Grandpa's house" and "Grandpa's home" are, and use the calculated similarity index as a weighting value to weight the similarity as characters (readings).

As a third example, the corrected speech determination unit 132 may detect similarities in the pronunciation of kanji characters. As an example, as described in Figure 2, there may be similarities due to the existence of multiple pronunciations (e.g., "Ibaraki" and "Ibaragi") that are easily confused for a single word written in kanji.

As a fourth example, the corrected speech determination unit 132 may detect similarity based on the interval between input times of the speech. For example, the corrected speech determination unit 132 may determine whether the second speech is corrected speech input to correct the content of the first utterance based on the similarity between the first keyword and a second keyword included in the second speech input within a predetermined time period after the first speech is input. For example, the corrected speech determination unit 132 may determine whether the second speech is corrected speech input to correct the content of the first utterance based on the similarity between the second keyword included in the second speech input immediately after the first speech and the first keyword included in the first speech.

In addition, the correction voice determination unit 132 may determine whether the second utterance is a correction voice input to correct the content of the first utterance, based on the number of times that an utterance that satisfies the conditions (similarity of reading, similarity of meaning, similarity of kanji reading, and similarity according to the input time interval) is input, using the degree to which these conditions are met as a condition.

(Regarding the detection unit 133)
The detection unit 133 may detect an input situation in which the second utterance is input. For example, the detection unit 133 can detect the input situation based on sensor information from a sensor included in the terminal device 10 or sensor information from a sensor included in the vehicle VEx.

For example, the detection unit 133 may detect the number of times the second utterance has been input as the input status. As an example, the detection unit 133 may detect the number of times the second utterance has been input within a predetermined period after the first utterance has been input.

Furthermore, the correction speech determination unit 132 may estimate whether the user intends to correct the first utterance content with the second utterance content from a perspective other than the degree of similarity described above, based on the number of inputs detected by the detection unit 133. For example, the correction speech determination unit 132 may determine whether the second utterance is correction speech input to correct the first utterance content, based on whether the number of inputs detected by the detection unit 133 exceeds a predetermined number. Specifically, if the number of times the second utterance is input within a predetermined period exceeds a predetermined number, the correction speech determination unit 132 may determine that the second utterance input within the predetermined period is speech input to correct the first utterance content.

In addition, if the second utterance is determined to be correction speech input to correct the first utterance, which second keyword contained in the second utterance is to be treated as correct information and which first keyword contained in the first utterance is to be treated as incorrect information may be determined from the perspective of the similarity between the second keyword and the first keyword.

As another example, the detection unit 133 may detect a frequency indicating a second utterance as the input status. In such a case, the correction speech determination unit 132 may estimate, based on the frequency detected by the detection unit 133, whether the user intends to correct the first utterance using the second utterance from a perspective other than the similarity described above. For example, the correction speech determination unit 132 may determine, based on the tone of the second utterance identified from the frequency, whether the second utterance is correction speech input to correct the first utterance. Specifically, if the identified tone indicates a predetermined speech pattern, the correction speech determination unit 132 may determine that the second utterance is correction speech input to correct the first utterance.

(Regarding the linking unit 134)
When the linking unit 134 determines that the second speech voice is a correction voice input to correct the first speech content, it links the first speech content with the second speech content indicated by the second speech voice.

For example, when the linking unit 134 determines that the second utterance is speech input to correct the first utterance, it extracts a combination of a second keyword and a first keyword that are determined to be similar to each other from among combinations of a second keyword included in the second utterance and a first keyword included in the first utterance.The linking unit 134 then links the second keyword and the first keyword in the extracted combination as correct answer information and as incorrect information for the correct answer information.

Furthermore, when the linking unit 134 determines that the second speech is correction speech input to correct the first speech, it may link the second keyword and the first keyword by treating one of the second keywords included in each second speech input within a predetermined period after the first speech is input as correct answer information and treating the first keyword included in the first speech as error information for the correct answer information. For example, the linking unit 134 may extract combinations of the second keyword and the first keyword that are determined to be similar to each other from combinations of the second keyword included in the second speech indicated by the second speech input within a predetermined period and the first keyword included in the first speech. Then, the linking unit 134 may determine the second keyword in the extracted combination as correct answer information and link the first keyword in this combination as error information for the correct answer information.

Furthermore, for example, when the linking unit 134 determines that the second speech is correction speech input to correct the first speech content, the linking unit 134 may link the second keyword and the first keyword, with the second keyword contained in the second speech spoken in a tone indicated by a predetermined speech style as correct answer information and the first keyword contained in the first speech as error information for the correct answer information. For example, the linking unit 134 may extract combinations of the second keyword and the first keyword that are determined to be similar to each other from combinations of the second keyword contained in the second speech spoken in a tone indicated by a predetermined speech style and the first keyword contained in the first speech content. Then, the linking unit 134 may link the second keyword and the first keyword in the extracted combination as correct answer information and the first keyword in this combination as error information for the correct answer information.

For example, the linking unit 134 may register the linking result of linking the second keyword with the first keyword in the linking information database 122, with the second keyword as correct information and the first keyword as incorrect information relative to the correct information, and associate the linking ID assigned to this second keyword with the linking result.

(Regarding the learning unit 135)
The learning unit 135 uses as learning data the pairs of correct answer information and error information linked by the linking unit 134 to learn patterns of utterance content indicated by the error information that are likely to be mistaken for the utterance content indicated by the correct answer information. For example, the learning unit 135 uses as learning data the pairs of correct answer information and error information to learn patterns of keywords indicated by the first keywords indicated by the error information that are likely to be mistaken for the second keyword indicated by the correct answer information.

(Regarding the information control unit 136)
The information control unit 136 performs predetermined control on the first utterance content based on the linking result by the linking unit 134 .

For example, based on the relationship between the correct answer information and the incorrect information linked by the linking unit 134, the information control unit 136 registers the incorrect information as correct answer information in the user dictionary (user dictionary database 123) so that when speech of the speech content indicated by the incorrect information is input, the input speech content is recognized as the speech content indicated by the correct answer information associated with the incorrect information.

For example, based on the learning results by the learning unit 135, the information control unit 136 registers the error information as correct information in the user dictionary so that when a speech that is easily mistaken for ...

3. Processing Procedure
Next, the procedure of information processing according to the first embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing the procedure of information processing according to the first embodiment. In the example of FIG. 7, it is assumed that the terminal device 10 transmits speech information indicating the received speech to the information processing device 100 every time it receives input of a speech. It is also assumed that the information processing device 100 accumulates the speech information transmitted from the terminal device 10 in the speech information database 121 as needed. In addition, in FIG. 7, the information processing procedure will be described using a user U1 of a vehicle VE1 as an example.

In this state, the acquisition unit 131 determines whether it is time to perform intention analysis (step S701). For example, the acquisition unit 131 may determine whether it is time to perform intention analysis based on whether a sufficient amount of speech information for performing intention analysis has been stored in the utterance information database 121.

While the acquisition unit 131 determines that it is not time to perform intention analysis (step S701; No), it waits until it determines that it is time to perform intention analysis.

On the other hand, if the acquisition unit 131 determines that it is time to perform intention analysis (step S701; Yes), it acquires first speech information indicating the first speech and second speech information indicating the second speech based on the context of the speech timing by user U1 (step S702). For example, the acquisition unit 131 recognizes the first speech, which is the speech input by user U1 first, and the second speech, which is the speech input by user U1 after the first speech, based on the context of the speech timing. Then, the acquisition unit 131 acquires the first speech information indicating the first speech and the second speech information indicating the second speech from the speech information stored in the speech information database 121 and corresponding to user U1.

Next, the corrected speech determination unit 132 determines whether intention analysis has not been completed for all pairs of first speech information and second speech information (step S703). Note that the pair of first speech information and second speech information referred to here may be a pair corresponding to a first speech speech and a second speech speech that are in a consecutive speech timing relationship.

If the corrected voice determination unit 132 determines that intention analysis has been completed for all pairs of first voice information and second voice information (step S703; No), it terminates the information processing related to the first embodiment at this point.

On the other hand, if the corrected voice determination unit 132 determines that intention analysis has not been completed for all pairs of first voice information and second voice information (step S703; Yes), it acquires unprocessed pairs of first voice information and second voice information for which intention analysis has not been completed (step S704).

Next, based on the first speech information and second speech information acquired in step S704, the correction speech determination unit 132 estimates the intention of user U1, i.e., whether or not a second speech has been input to correct the first speech content indicated by the first speech information (step S705). Specifically, based on the first speech content indicated by the first speech information and the second speech content indicated by the second speech information, the correction speech determination unit 132 estimates the intention of user U1, i.e., whether or not user U1 has input a second speech that indicates the second speech content to correct the first speech content with the second speech content.

For example, the corrected speech determination unit 132 performs morphological analysis on text indicating a first speech content (first speech information) to extract each word constituting this text as a first keyword. The corrected speech determination unit 132 also performs morphological analysis on text indicating a second speech content (second speech information) to extract each word constituting this text as a second keyword. Based on the similarity between the extracted first keywords and the second keywords, the corrected speech determination unit 132 then estimates the intention of user U1, i.e., whether user U1 input a second speech speech indicating the second speech content in order to correct the first speech content with the second speech content.

Next, the correction voice determination unit 132 determines whether the second utterance is a correction voice input to correct the content of the first utterance based on the estimation result of the intention of user U1 through intention analysis (step S706).

If the correction voice determination unit 132 determines that the second utterance voice is not a correction voice input to correct the content of the first utterance (step S706; No), it returns to step S703 to process other unprocessed pairs of first voice information and second voice information for which intention analysis has not been completed.

On the other hand, if the linking unit 134 determines that the second utterance is correction speech input to correct the first utterance (step S706; Yes), it performs a linking process to link the first keyword with the second keyword (step S707). For example, the linking unit 134 extracts combinations of the second keyword and the first keyword that are determined to be similar to each other from among combinations of the second keyword included in the second utterance content (second utterance information) and the first keyword included in the first utterance content (first utterance information). The linking unit 134 then links the second keyword with the first keyword, regarding the second keyword in the extracted combination as correct answer information and the first keyword in this combination as incorrect information for the correct answer information.

The linking unit 134 may also register the linking result of linking the second keyword with the first keyword in the linking information database 122 in a state in which a pair of a user ID and a linking ID indicating user U1 is associated with the result. As a result, the linking information database 122 shown in Figure 5 is obtained.

Next, the learning unit 135 uses the pairs of correct answer information and erroneous information obtained as a result of the linking process as learning data to learn patterns of keywords among the first keywords indicated by the erroneous information that are likely to be confused with the second keywords indicated by the correct answer information (step S708).

Next, the information control unit 136 registers the keywords in the dictionary based on the learning results (step S709). For example, based on the learning results, when a speech containing a first keyword that is easily confused with a second keyword is input, the information control unit 136 registers the first keyword as an "utterance keyword" and the second keyword as a "recognition keyword" in the user dictionary in a corresponding state, so that the input first keyword is recognized as the second keyword. As a result, a user dictionary database 123 such as that shown in Figure 6 is obtained.

At this point, the information control unit 136 returns the process to step S703. If it is determined that intention analysis has been completed for all pairs of first voice information and second voice information, the information processing according to the first embodiment ends at this point.

Second Embodiment
1. Overview of the Second Embodiment
Next, the second embodiment will be described with reference to Fig. 8. In the first embodiment, the correction voice of a user attempting to correct the content of a mistake made by speech is detected, and linking is performed based on the content of the detected correction voice. In contrast, the second embodiment differs from the first embodiment in that the correction operation of a user attempting to correct the content of a mistake made by speech using an input means other than speech is detected, and linking is performed based on the content of the detected correction operation.

In the following, information processing according to the second embodiment will be described using Figure 8. Figure 8 is an explanatory diagram illustrating information processing according to the second embodiment. In addition, in the second embodiment, the input means other than spoken voice is a manual input operation in which information is input by touching a predetermined object (for example, the display panel (display screen) of the terminal device 10).

Figure 8 shows a scene in which user U1 inputs speech by speaking into a terminal device 10 installed in a vehicle VE1 (an example of a vehicle VEx) and inputs information by touching the terminal device 10.

For example, each time the terminal device 10 receives input of a speech, it transmits speech information indicating the received speech to the information processing device 200. As a result, the information processing device 200 acquires the speech information from the terminal device 10 (step S21).

Figure 8 shows a scene in which user U1 is inputting a speech voice instructing that route guidance be provided to "destination XX located in Ibaraki City, Osaka Prefecture." Specifically, Figure 8 shows an example in which user U1 inputs a speech voice VO11 with content C11 such as "Please guide me along the route to XX in Ibaraki City!" In this example, in response to the input of the speech voice VO11, the terminal device 10 transmits audio information indicating the speech content C11 to the information processing device 200. As a result, the information processing device 200 acquires the audio information indicating the speech content C11.

Furthermore, each time the terminal device 10 receives input through a manual input operation, it may also transmit operation information indicating the input operation content to the information processing device 200. As a result, the information processing device 200 acquires the operation information through the manual input operation from the terminal device 10 (step S22).

Now, let's say user U1 realizes that he/she mistakenly said "Ibaragi" when he/she should have said "Ibaraki." In Figure 2, user U1 re-inputs the speech VO12 with content C12 such as "Please guide me to XX in Ibaraki City!"

However, in the example of Figure 8, user U1 is unsure about speaking and believes that manual input is more reliable. In the example of Figure 8, user U1 performs manual input operation IO12 of operation content C12, which involves typing in the correct destination, "Ibaraki," while a navigation screen related to route guidance is displayed on terminal device 10. In this example, terminal device 10 transmits operation information indicating operation content C12 to information processing device 200 in response to manual input operation IO12. As a result, information processing device 200 acquires operation information indicating operation content C12.

In the example of Figure 8, the operation information indicating operation content C12 may include a keyword indicating the destination "Ibaraki." Therefore, the manual input operation IO12 can be considered a destination setting operation.

Next, the information processing device 200 acquires first voice information indicating the first voice speech and second operation information indicating a second input operation, which is a manual input operation (destination setting operation) performed after the first voice speech, based on the context between the timing of the speech by user U1 and the timing of the manual input operation by user U1 (step S23). For example, the information processing device 200 may recognize the first voice speech, which is the voice speech input first, and the second input operation, which is the manual input operation performed after the first voice speech, based on the context between the timing of the speech and the manual input operation. Furthermore, the information processing device 200 may thereby acquire first voice information indicating the first voice speech from the voice information previously collected via the terminal device 10. Furthermore, the information processing device 200 may acquire second operation information indicating the second input operation from the operation information previously collected via the terminal device 10.

In the example of Figure 8, the information processing device 200 recognizes the spoken voice VO11 as the first spoken voice, thereby acquiring voice information indicating the spoken content C11 as the first voice information. Also, in the example of Figure 8, the information processing device 200 recognizes the manual input operation IO12 as the second input operation, thereby acquiring operation information indicating the operation content C12 as the second operation information. Hereinafter, the spoken voice VO11 may be referred to as the "first spoken voice VO11," and the manual input operation IO12 may be referred to as the "second input operation IO12."

Next, the information processing device 200 performs an intention analysis to estimate the intention of correcting the mistaken utterance based on the similarity between the first keywords, which are the keywords that make up the first voice information (first utterance content), and the second keywords, which are the keywords that make up the second operation information (second operation content) (step S24). Specifically, the information processing device 200 estimates the intention of whether user U1 intends to correct the first utterance content by using the second operation content, based on the similarity between the first keywords and the second keywords. The specific method of intention analysis performed in step S24 will be described later.

In step S24, the information processing device 200 may perform morphological analysis on the text indicating the first audio information to extract each word constituting this text as a first keyword. Similarly, the information processing device 200 may perform morphological analysis on the text indicating the second operation information to extract each word constituting this text as a second keyword.

Next, based on the estimation result of the intention analysis, the information processing device 200 determines whether the second input operation IO12 is a correction operation manually input to correct the first utterance content corresponding to the first speech audio VO11 (step S25). For example, if the information processing device 200 recognizes similarity between "Ibaragi," one of the first keywords, and "Ibaraki," one of the second keywords, it can infer that user U1 intends to correct the first utterance content (first keyword KW11) using the second operation content (second keyword KW12). As a result, the information processing device 200 can determine that the second input operation IO12 is a correction operation manually input to correct the first utterance content.

In this way, if the information processing device 200 determines that the second input operation IO12 is a manual correction operation to correct the first utterance, the information processing device 200 associates the second keyword "Ibaraki" with the first keyword "Ibaragi," with the second keyword "Ibaraki" as the correct answer and the first keyword "Ibaragi" as the incorrect answer (step S26). Figure 8 shows examples of association results resulting from user U1's past mispronunciations of "Ibaraki" as "Ibaragi" and "Ibaraki" as "Ibaraku," including the current example in which the second keyword "Ibaraki" is the correct answer and the first keyword "Ibaragi" is the correct answer. Furthermore, such association results may be managed in the association information database 122 (Figure 5) using an association ID.

The information processing device 200 also uses pairs of correct answer information and erroneous information as learning data to learn which of the first keywords indicated by the erroneous information are likely to be confused with the second keyword indicated by the correct answer information (step S27). In the example of Figure 8, the information processing device 200 learns which of the first keywords "Ibaragi" and "Ibaraku" are likely to be confused with the second keyword "Ibaraki."

Then, the information processing device 200 registers the keyword in the dictionary based on the learning results (step S28). For example, based on the learning results, the information processing device 200 registers the first keyword in the user dictionary (Figure 6) so that when a speech containing a keyword that is easily confused with a second keyword is input, the input first keyword will be recognized as the second keyword. Figure 8 shows an example in which the information processing device 200 registers the first keyword "Ibaragi" in the user dictionary so that when a speech containing the first keyword "Ibaragi" is input, the first keyword "Ibaragi" will be recognized as the second keyword "Ibaraki."

As described above with reference to FIG. 8, in the second embodiment, when a second input operation is performed after a first utterance is input, the information processing device 200 determines whether the second input operation is a correction operation manually input to correct the first utterance, based on the first utterance content indicated by the first utterance and the second operation content indicated by the second input operation. If the information processing device 200 determines that the second input operation is a correction operation manually input to correct the first utterance, it links the first utterance content with the second operation content, and registers the first utterance content in the user dictionary based on the linking result.

According to the information processing of the second embodiment, the information processing device 200 can be controlled so that the correct action can be taken in response to the mistake even if the user makes a mistake.

As another example, when the guidance interrupt button is operated after the first keyword "stop guidance," the information processing device 200 can infer that this is an operation that includes an intention to correct "stop guidance," one of the first keywords, to "interrupt guidance," one of the second keywords that corresponds to the guidance interrupt button. In this way, the information processing device 200 can infer that this is an operation that includes an intention to correct, not by direct input of a keyword, but from an operation button or the like that corresponds to a keyword and a function.

2. Configuration of information processing device
From here, an information processing device 200 according to the second embodiment will be described with reference to Fig. 9. Note that descriptions of processing units in the information processing device 200 that are assigned the same reference numerals as those in the information processing device 100 may be omitted. Fig. 9 is a diagram showing an example of the configuration of the information processing device 200 according to the second embodiment. As shown in Fig. 9, the information processing device 200 has a communication unit 110, a storage unit 220, and a control unit 230.

(Regarding the storage unit 220)
The storage unit 220 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 220 may further include an operation information database 224.

(Regarding the operation information database 224)
The operation information database 224 stores information about manual input operations performed by a user. An example of the operation information database 224 according to the second embodiment is shown in Fig. 10. In the example of Fig. 10, the operation information database 224 has items such as "user ID,""operation date and time," and "operation information."

"User ID" refers to identification information that identifies a user who manually inputs information (e.g., information indicating a destination) into the terminal device 10. For example, the information processing device 100 may recognize the user who performed the manual input operation based on an image captured by a sensor (e.g., a camera) possessed by the terminal device 10, and issue a "user ID" to the recognized user.

"Operation date and time" indicates information regarding the date and time when information was entered by manual input operation. Figure 10 shows an example in which user ID "U1" is associated with "operation date and time #11." This example shows an example in which user U1 inputs a destination, etc. into the terminal device 10 by performing a manual input operation on the date and time of operation date and time #11. For example, by regarding the "utterance date and time" in the utterance information database 121 as the timing of the utterance and the "operation date and time" in Figure 10 as the operation timing, the information processing device 100 can recognize the first utterance, which is the utterance inputted earlier, and the second input operation, which is the manual input operation performed after the first utterance was inputted.

"Operation information" is information that indicates the content of the manual input operation (e.g., the destination) entered by the user indicated by the "user ID." In other words, "operation information" may include keywords that indicate the destination entered by touching the display panel of the terminal device 10 to set the destination. Figure 10 shows an example in which user ID "U1," "operation date and time #11," and "operation information #11" are associated. This example shows an example in which the content of operation information #11 was entered by user U1 through a manual input operation performed on operation date and time #11.

(Regarding the control unit 230)
9 , the control unit 230 is realized by a CPU, an MPU, or the like executing various programs (for example, the information processing program according to the embodiment) stored in a storage device inside the information processing device 200 using RAM as a work area. The control unit 230 is also realized by an integrated circuit such as an ASIC or FPGA.

As shown in FIG. 9, the control unit 230 may further include a correction operation determination unit 237 in addition to the acquisition unit 131, correction voice determination unit 132, detection unit 133, linking unit 134, learning unit 135, and information control unit 136. The correction operation determination unit 237 then realizes or executes the information processing functions and actions described below. Note that the internal configuration of the control unit 230 is not limited to the configuration shown in FIG. 9, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationships between the processing units in the control unit 230 are not limited to the connection relationships shown in FIG. 9, and may be other connection relationships.

(Regarding the Acquisition Unit 131)
The acquisition unit 131 acquires various pieces of information used in the information processing according to the second embodiment. Furthermore, the acquisition unit 131 may output the acquired information to an appropriate processing unit that performs processing using this information.

The acquisition unit 131 may also acquire first speech information indicating the first speech and second operation information indicating the second input operation. For example, the acquisition unit 131 may recognize the first speech, which is the speech input earlier, and the second input operation, which is the manual input operation performed after the first speech was input, based on the context between the timing of the speech and the timing of the manual input operation. As a result, the acquisition unit 131 may also acquire first speech information indicating the first speech from among the speech information that has been collected so far via the terminal device 10. The acquisition unit 131 may also acquire second operation information indicating the second input operation from among the operation information that has been collected so far via the terminal device 10.

(Regarding the correction operation determination unit 237)
When a second input operation is performed to input information by touching a specified object after a first utterance has been input, the correction operation determination unit 237 determines whether the second input operation is a correction operation for correcting the first utterance content based on the utterance content indicated by the first utterance (first utterance content) and the operation content indicated by the second input operation (second operation content).

For example, the correction operation determination unit 237 estimates the user's intention by determining whether the first utterance is an error due to a slip of the tongue and whether the user manually inputs the second operation content with the intention of correcting this error. In other words, the correction operation determination unit 237 estimates whether the user intends to correct the first utterance content by using the second operation content, based on the first utterance content indicated by the first utterance audio and the second operation content indicated by the second input operation. Then, the correction operation determination unit 237 determines whether the second input operation is a correction operation for correcting the first utterance content, depending on the estimation result. For example, if the estimation result indicates that the user intends to correct the first utterance content by using the second input operation, the correction operation determination unit 237 can determine that the second input operation is correction audio for correcting the first utterance content.

Below, we will show a specific example of intention analysis in which we estimate whether the user intends to correct the first utterance content by the second input, and based on the estimation result, we determine whether the second input operation is a correction operation for correcting the first utterance content.

For example, the correction operation determination unit 237 may perform morphological analysis on text indicating the first voice content (first voice information) to extract each word constituting this text as a first keyword. Furthermore, the correction operation determination unit 237 may extract words related to the destination included in the second operation content (second operation information) as a second keyword.

In such a case, the correction operation determination unit 237 determines whether the second input operation is a correction operation for correcting the first utterance content based on the first keyword corresponding to the first utterance content and the second keyword corresponding to the second operation content.

Here, as described above, the second input operation may be a destination setting operation performed following the first utterance. For this reason, the correction operation determination unit 237 uses a second keyword indicating the destination entered in the second input operation as a destination setting operation to determine whether the second input operation is a correction operation for correcting the content of the first utterance.

For example, the correction operation determination unit 237 may determine whether the second input operation is a correction operation for correcting the first utterance content based on the similarity between the first keyword and the second keyword. For example, the correction voice determination unit 132 may detect similarity for each combination between one first keyword and one second keyword, and then determine whether the second input operation is a correction voice input for correcting the first utterance content based on the detected similarity.

For example, the correction operation determination unit 237 may use a method similar to the information processing method according to the first embodiment to detect the similarity between the first keyword and the second keyword. Specifically, the correction operation determination unit 237 may calculate the similarity by detecting similarity in pronunciation, similarity in meaning, similarity in the pronunciation of kanji characters, etc., and may infer the user's intention based on the calculated similarity.

In addition, the correction operation determination unit 237 may determine whether the second input operation is a correction operation for correcting the content of the first utterance based on the similarity with the second keyword entered in the second input operation until a predetermined time has elapsed since the first utterance was input.

Here, if the user inputs a spoken voice indicating a destination but realizes that they misspoken the destination, they will typically reset the destination by manually entering the destination into the terminal device 10 while the vehicle VEx is stopped. For this reason, if the correction operation determination unit 237 detects that a second input operation has been performed between the time the first spoken voice is input and the time the vehicle VEx starts moving (while the vehicle VEx is stopped after the first spoken voice is input), it may determine whether the second input operation is a correction operation for correcting the content of the first utterance based on the similarity with the second keyword input in this second input operation.

(Regarding the linking unit 134)
When the linking unit 134 determines that the second input operation is a correction operation for correcting the first utterance content, the linking unit 134 links the first utterance content with the second input content indicated by the second input operation.

For example, when the linking unit 134 determines that the second input operation is a correction operation for correcting the first utterance, it extracts a combination of the second keyword indicated by the second operation and the first keyword included in the first utterance that is determined to be similar to each other from among the combinations.The linking unit 134 then links the second keyword and the first keyword, regarding the second keyword in the extracted combination as correct answer information and the first keyword in this combination as incorrect information for the correct answer information.

For example, the linking unit 134 may register the linking result of linking the second keyword with the first keyword in the linking information database 122, with the second keyword as correct information and the first keyword as incorrect information relative to the correct information, and associate the linking ID assigned to this second keyword with the linking result.

(Regarding the learning unit 135)
The learning unit 135 uses as learning data the pairs of correct answer information and error information linked by the linking unit 134 to learn patterns of utterance content that is likely to be mistaken for the operation content indicated by the correct answer information, among the utterance content indicated by the error information. For example, the learning unit 135 uses as learning data the pairs of correct answer information and error information to learn patterns of keywords that are likely to be mistaken for the second keyword indicated by the correct answer information, among the first keywords indicated by the error information.

(Regarding the information control unit 136)
The information control unit 136 performs predetermined control on the first utterance content based on the linking result by the linking unit 134 .

For example, based on the relationship between the correct answer information and the incorrect information linked by the linking unit 134, the information control unit 136 registers the incorrect information as correct answer information in the user dictionary (user dictionary database 123) so that when speech of the speech content indicated by the incorrect information is input, the input speech content is recognized as the operation content indicated by the correct answer information associated with the incorrect information.

For example, based on the learning results by the learning unit 135, the information control unit 136 registers the error information as correct information in the user dictionary so that when a speech that is easily mistaken for an operation indicated by correct information is input, the input speech is recognized as the operation indicated by the correct information associated with the error information. For example, the information control unit 136 registers keywords in the user dictionary based on the learning results by the learning unit 135. For example, based on the learning results, when a speech that includes a first keyword that is easily mistaken for a second keyword is input, the information control unit 136 registers the first keyword in the user dictionary so that the input first keyword is recognized as the second keyword.

3. Processing Procedure
Next, the procedure of information processing according to the second embodiment will be described with reference to Fig. 11 . Fig. 11 is a flowchart showing the procedure of information processing according to the second embodiment. In the example of Fig. 11 , it is assumed that the terminal device 10 transmits speech information indicating the received speech to the information processing device 100 every time it receives input of a speech voice. It is also assumed that the information processing device 100 accumulates the speech information transmitted from the terminal device 10 in the speech information database 121 as needed.

On the other hand, in the example of Figure 11, it is assumed that the terminal device 10 transmits operation information indicating the input operation content to the information processing device 100 each time it receives input by manual operation. It is also assumed that the information processing device 100 accumulates the operation information transmitted from the terminal device 10 in the operation information database 224 as needed.

In addition, Figure 11 explains the information processing procedure using user U1 of vehicle VE1 as an example.

In this state, the acquisition unit 131 determines whether it is time to perform intention analysis (step S801). For example, the acquisition unit 131 may determine whether it is time to perform intention analysis based on whether a sufficient amount of speech information for performing intention analysis has been stored in the utterance information database 121 and whether a sufficient amount of operation information for performing intention analysis has been stored in the operation information database 224.

While the acquisition unit 131 determines that it is not time to perform intention analysis (step S801; No), it waits until it determines that it is time to perform intention analysis.

On the other hand, if the acquisition unit 131 determines that it is time to perform intention analysis (step S801; Yes), it acquires first speech information indicating a first speech and second operation information indicating a second input operation based on the context between the speech timing and the operation timing (step S802). For example, based on the context between the speech timing and the operation timing, the acquisition unit 131 may recognize the first speech, which is the speech input earlier by user U1, and the second input operation, which is the manual input operation performed by user U1 after the first speech was input.

Then, the acquisition unit 131 acquires first speech information indicating a first speech voice from the speech information stored in the speech information database 121 and corresponding to user U1. The acquisition unit 131 also acquires second operation information indicating a second input operation from the operation information stored in the operation information database 224 and corresponding to user U1.

Next, the correction operation determination unit 237 determines whether intention analysis has not been completed for all pairs of first voice information and second input information (step S803). Note that the pair of first voice information and second voice information referred to here may be a pair corresponding to a first voice utterance and a second input operation in which the utterance timing and the operation timing are in a continuous relationship.

If the correction operation determination unit 237 determines that intention analysis has been completed for all pairs of first voice information and second input information (step S803; No), it terminates the information processing related to the second embodiment at this point.

On the other hand, if the correction operation determination unit 237 determines that intention analysis has not been completed for all pairs of first voice information and second input information (step S803; Yes), it obtains unprocessed pairs of first voice information and second input information for which intention analysis has not been completed (step S804).

Next, the correction operation determination unit 237 estimates the intention of user U1 based on the first voice information and second input information acquired in step S804, by determining whether a second input operation was performed to correct the first utterance content indicated by the first voice information (step S805). Specifically, the correction operation determination unit 237 estimates the intention of user U1 based on the first utterance content indicated by the first voice and the second operation content indicated by the second input operation, by determining whether user U1 performed a second input operation indicating the second utterance content in order to correct the first utterance content using the second operation content.

For example, the correction operation determination unit 237 performs morphological analysis on text indicating the first speech content (first speech information) to extract each word constituting this text as a first keyword. The correction operation determination unit 237 may also extract words related to the destination included in the second operation content (second operation information) as a second keyword. Then, based on the similarity between the extracted first keyword and the second keyword, the correction operation determination unit 237 estimates the intention of user U1, i.e., whether user U1 performed a second input operation to correct the first speech content using the second operation content.

Next, the correction operation determination unit 237 determines whether the second input operation is a correction operation for correcting the first utterance content based on the estimation result of the intention of user U1 through intention analysis (step S806).

If the correction operation determination unit 237 determines that the second input operation is not a correction operation for correcting the first utterance content (step S806; No), it returns to step S803 to process other unprocessed pairs of first voice information and second operation information for which intention analysis has not been completed.

On the other hand, if the linking unit 134 determines that the second input operation is a correction operation for correcting the first utterance content (step S806; Yes), it performs a linking process to link the first keyword with the second keyword (step S807). For example, the linking unit 134 extracts combinations of the second keyword and the first keyword that are determined to be similar to each other from among combinations of the second keyword included in the second operation content (second operation information) and the first keyword included in the first utterance content (first utterance information). Then, the linking unit 134 links the second keyword and the first keyword, regarding the second keyword in the extracted combination as correct answer information and the first keyword in this combination as incorrect information for the correct answer information.

The linking unit 134 may also register the linking result of linking the second keyword with the first keyword in the linking information database 122 in a state in which a pair of a user ID and a linking ID indicating user U1 is associated with the result. As a result, the linking information database 122 shown in Figure 5 is obtained.

Next, the learning unit 135 uses the pairs of correct answer information and erroneous information obtained as a result of the linking process as learning data to learn patterns of keywords among the first keywords indicated by the erroneous information that are likely to be confused with the second keywords indicated by the correct answer information (step S808).

Next, the information control unit 136 registers the keywords in the dictionary based on the learning results (step S809). For example, based on the learning results, when a speech containing a first keyword that is easily confused with a second keyword is input, the information control unit 136 registers the first keyword as an "utterance keyword" and the second keyword as a "recognition keyword" in the user dictionary in a corresponding state, so that the input first keyword is recognized as the second keyword. As a result, a user dictionary database 123 such as that shown in Figure 6 is obtained.

At this point, the information control unit 136 returns the process to step S803. If it is determined that intention analysis has been completed for all pairs of first voice information and second operation information, the information processing according to the second embodiment ends at this point.

(Other embodiments)
The information processing device 100 (information processing device 200) may be realized in various different modes other than the above-described embodiment. Therefore, other embodiments of the information processing device 100 (information processing device 200) will be described below.

1. Detecting words that suggest slip-ups
In the above first embodiment, an example was shown in which the acquisition unit 131 acquires first speech information and second speech information by recognizing a first speech voice, which is a speech voice that was input first, and a second speech voice, which is a speech voice that was input after the first speech voice was input, based on the context of the speech timing.

However, if the acquisition unit 131 detects a word that suggests a slip of the tongue, it may acquire the first speech information and the second speech information by recognizing the first speech audio and the second speech audio based on the timing at which the word was uttered.

For example, when a user realizes that they have made a mistake, they may reflexively say, "I made a mistake!" or "Oh no!" Immediately after making such an utterance, the user tends to input speech to correct the mistake.

For this reason, when the acquisition unit 131 detects a word that suggests a slip of the tongue, such as "I made a mistake!" or "Oh no!", it may recognize the speech input immediately before the word as the first speech, and may recognize the speech input immediately after the word as the second speech.

In addition, when a word suggesting a slip of the tongue is detected, the correction voice determination unit 132 may determine that the speech input immediately after the word (i.e., the second speech) is correction voice input to correct the content of the first speech.

Furthermore, in the second embodiment, the acquisition unit 131 may also detect words that suggest a slip of the tongue. Then, in the second embodiment, if the acquisition unit 131 detects a word that suggests a slip of the tongue, the acquisition unit 131 may acquire first voice information and second operation information by recognizing the first utterance and the second input operation based on the timing at which the word was uttered.

For example, if the acquisition unit 131 detects a word that suggests a slip of the tongue, such as "I made a mistake!" or "Oh no!", it may recognize the speech input immediately before the word as the first speech, and may also recognize the manual input operation performed immediately after the word as the second input operation.

In addition, when a word suggesting a slip of the tongue is detected, the correction operation determination unit 237 may determine that the manual input operation (i.e., the second input operation) performed immediately after the word is a correction operation for correcting the content of the first utterance.

[2. Measures to improve registration accuracy]
In the above second embodiment, an example was shown in which the information control unit 136, based on the learning results by the learning unit 135, associates a second keyword included in the second operation content with a first keyword included in the second speech voice that is likely to be mistaken for the second keyword, and registers the first keyword in the user dictionary.

However, the information control unit 136 may associate the second keyword with the first keyword and register them in the user dictionary based on whether the user has actually arrived at the destination indicated by the second keyword. For example, the information control unit 136 may determine whether the user has arrived at the destination (second keyword) based on the location information of the user (user's vehicle VEx) and the destination set by the user, and if it detects that the user has arrived at the destination, it may associate the second keyword with the first keyword and register them in the user dictionary.

According to the information processing of the second embodiment, the information processing device 200 can improve the accuracy of registration in the user dictionary.

(others)
[1. Hardware Configuration]
The information processing apparatus 100 according to the first embodiment and the information processing apparatus 200 according to the second embodiment described above are realized, for example, by a computer 1000 configured as shown in Fig. 12. The information processing apparatus 100 will be described below as an example. Fig. 12 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing apparatus 100. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a HDD 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F) 1700.

The CPU 1100 operates based on programs stored in the ROM 1300 or the HDD 1400 and controls each component. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

HDD 1400 stores programs executed by CPU 1100 and data used by such programs. Communication interface 1500 receives data from other devices via a specified communication network and sends it to CPU 1100, and transmits data generated by CPU 1100 to other devices via a specified communication network.

The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input/output interface 1600. The CPU 1100 acquires data from the input devices via the input/output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input/output interface 1600.

Media interface 1700 reads a program or data stored on recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 1000 functions as the information processing device 100 in the first embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 130 by executing a program (e.g., an information processing program according to the embodiment) loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

Furthermore, for example, when the computer 1000 functions as the information processing device 200 in the second embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 230 by executing a program (e.g., an information processing program according to the embodiment) loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[2. Other]
Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

In addition, the above embodiments can be combined as appropriate to the extent that the processing content is not contradictory.

(summary)
Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have undergone various modifications and improvements based on the knowledge of those skilled in the art.

In addition, the "parts" (sections, modules, units) mentioned above can be read as "means" or "circuits." For example, an acquisition unit can be read as an acquisition means or an acquisition circuit.

REFERENCE SIGNS LIST 1 Information processing system 10 Terminal device 100 Information processing device 120 Storage unit 121 Speech information database 122 Linking information database 123 User dictionary database 130 Control unit 131 Acquisition unit 132 Corrected speech determination unit 133 Detection unit 134 Linking unit 135 Learning unit 136 Information control unit 200 Information processing device 220 Storage unit 224 Operation information database 230 Control unit 237 Correction operation determination unit

Claims (9)

a voice information acquisition unit that acquires information indicating a user's voice;
an operation information acquisition unit that acquires information indicating an input operation input by the user by touching a predetermined object;
a determination unit that, when the voice information acquisition unit acquires information indicating a first utterance and then acquires information indicating a word suggesting a slip of the tongue, and when the operation information acquisition unit acquires information indicating a second input operation, determines whether the second input operation is a correction operation for correcting the speech content indicated by the first utterance ;
and a registration control unit that, when the determination unit determines that the second input operation is the correction operation, links the operation content indicated by the second input operation with the speech content indicated by the first speech voice and registers them in a predetermined dictionary . The information processing device according to claim 1, characterized in that the determination unit determines whether the second input operation is a correction operation for correcting the speech content based on a similarity between a first keyword included in the first speech voice as the speech content indicated by the first speech voice and a second keyword input by the second input operation as the operation content indicated by the second input operation. the second input operation is a destination setting operation performed subsequent to the first utterance,
The information processing device according to claim 2, characterized in that the determination unit determines whether the second input operation is a correction operation for correcting the speech content indicated by the first speech voice using a second keyword indicating the destination input in the destination setting operation as the second keyword. 3. The information processing device according to claim 2, wherein the determination unit determines whether the second input operation is a correction operation for correcting the speech content indicated by the first speech voice based on a similarity between the second keyword and a second keyword included in the second input operation inputted within a predetermined time period after the information indicating the first speech voice is acquired. The information processing device of any one of claims 2 to 4, characterized in that when the determination unit determines that the second input operation is the correction operation, the registration control unit associates the second keyword with the first keyword, treating the second keyword as correct information and the first keyword as error information for the correct information. The information processing device according to claim 5 , characterized in that the registration control unit, based on the relationship between the linked correct answer information and the linked error information, when speech of the speech content indicated by the error information is acquired , associates the error information with the correct answer information and registers the error information in the specified dictionary so that the acquired speech content is recognized as the operation content indicated by the correct answer information linked to the error information. a learning unit configured to learn, as learning data, a set of linked correct answer information and error information, a pattern of utterance content that is likely to be mistaken for an operation content indicated by the correct answer information, among utterance content indicated by the error information;
The information processing device according to claim 6, characterized in that, when a speech content indicated by the error information is input that is likely to be mistaken for an operation content indicated by the correct answer information, the registration control unit registers the error information in the specified dictionary in association with the correct answer information based on the learning result by the learning unit , so that the input speech content is recognized as the operation content indicated by the correct answer information associated with the error information. An information processing method executed by an information processing device,
a voice information acquisition step of acquiring information indicating a voice uttered by a user;
an operation information acquiring step of acquiring information indicating an input operation input by the user by touching a predetermined object;
a determining step of determining whether or not the second input operation is a correction operation for correcting the content of the utterance indicated by the first utterance, when information indicating a word suggesting a slip of the tongue is obtained after information indicating a first utterance is obtained in the voice information obtaining step and information indicating a second input operation is obtained in the operation information obtaining step;
and a registration control step of linking the operation content indicated by the second input operation with the speech content indicated by the first speech voice and registering them in a predetermined dictionary when the second input operation is determined to be the correction operation in the determination step. a voice information acquisition step of acquiring information indicating a user's voice;
an operation information acquisition step of acquiring information indicating an input operation input by the user by touching a predetermined object;
a determination step of determining whether or not the second input operation is a correction operation for correcting the content of the utterance indicated by the first utterance, when information indicating a word suggesting a slip of the tongue is obtained after information indicating a first utterance is obtained in the voice information acquisition step and information indicating a second input operation is obtained in the operation information acquisition step;
and a registration control procedure for linking the operation content indicated by the second input operation with the speech content indicated by the first speech voice and registering them in a predetermined dictionary, if the second input operation is determined to be the correction operation in the determination procedure.