JP3273620B2 - Voice interaction device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech dialogue apparatus for recognizing a voice uttered by a user and executing a predetermined operation, and more particularly to a technique for executing a dialogue according to the behavior of the user.

[0002]

2. Description of the Related Art In recent years, a voice dialogue apparatus between a user and a computer, which combines technologies such as voice and image, has been put to practical use.
In such a conventional voice interaction device, it is detected that the user has performed a predetermined operation such as pressing a button, inserting money, uttering a keyword, and the like. Therefore, a user who uses the interactive device for the first time has a drawback that it is difficult to use the information if the guidance of the usage is not sufficiently provided. Also, there is a problem that even if the user knows how to use it, the conversation does not start unless the user performs a predetermined operation, which is troublesome.

Further, in the conventional voice interaction device, the state and reaction of the user when starting the interaction are not taken into account. That is, since the voice conveys information continuously in time, it is important to take a pause while watching the reaction of the other party during conversation and to select and speak words. However, in the past, the spoken dialogue device suddenly started talking to the user, and the user felt that the user was psychologically forced or ignored,
Sometimes there was no choice but to talk without a willingness to talk.

[0004] Also, to always start talking in a fixed phrase,
There is a disadvantage that it is unnecessary, unnecessary, or redundant for the user, and is troublesome.

[0005]

As described above, in the conventional voice dialogue apparatus, the dialogue is started after the user performs a predetermined operation, so that it is difficult to use. Also,
Since the state and reaction of the user are not taken into account, an inappropriate message may be output or a dialog may be started against the intention. Furthermore, since the movement of the user is not respected even during the dialogue, there is a disadvantage that the movement of the user is restricted.

[0006] The present invention has been made to solve such a conventional problem, and an object of the present invention is to provide a voice dialogue apparatus capable of performing a more natural dialogue according to the behavior of a user. is there.

[0007]

Means for Solving the Problems] To achieve the above object, the present invention recognizes the voice uttered by the user, the voice dialogue system for performing a predetermined operation, Tomah standing user
Motion information that is located near the
Information input unit, and that the user has stopped at the motion information input unit
A stop monitoring unit for measuring the determined time;
Output a message when the stop time exceeds a specified threshold
And a message output unit .

[0008]

[0009] Further, a means for recognizing a user's response to the message and selecting a message to be output next according to the response result is provided.

[0010] Further, there is provided means for changing the threshold value of the stationary time in accordance with a position where the user stops.

[0011] Further, an ID identification unit for specifying a user ID, an ID information storage unit for storing ID information of each user,
Means for determining a user's message in accordance with the ID information.

[0012] The present invention is further characterized in that a speed measuring unit for measuring a speed at which the user approaches the voice interactive device is provided, and a means for determining a message to the user according to the approach speed is provided.

[0013]

With the above configuration, the stop monitoring unit can
Detecting that the user has stopped in front of the voice interaction device,
Call the user. Therefore, we do not simply address those who have passed or those who have no intention of dialogue. Further, since the dialogue is started even if the user does not perform a specific operation, even a user who does not know how to use can easily use it.

[0014] Further, when the user has gone away, does not respond, or responds, it is possible to output an appropriate message according to the behavior of the user.

Further, when the user stops at a position close to the apparatus, a message is output immediately, and when the user is far from the apparatus, a message is output slowly, so that an appropriate dialogue meeting the user's request can be performed.

Furthermore, if the user's ID information is obtained, and if the user's preference is known as the ID information, for example, a message that meets the needs can be output according to the information, a natural conversation that meets the user's needs can be performed. Become.

Further, if the approach speed of the user is detected, for example, a message of a hurried tone can be output to a user having a fast approach speed, and a dialogue that does not irritate the user can be performed. Become.

[0018]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a first embodiment of a voice interaction apparatus to which the present invention is applied.

The voice interactive device shown in FIG. 1 determines a message sentence to the user, a motion information input unit 1 for capturing the user's motion as an electric signal, a stop monitoring unit 2 for monitoring the user's stop. It comprises a message sentence deciding unit 3, a message output unit 4 for outputting a message, and a control unit 5 for controlling the voice interactive device in general.

The motion information input unit 1 includes a pressure sensor, an optical switch, a camera, and the like, and detects a motion of the user. FIG. 2 shows an example in which a camera 14 is mounted on the apparatus main body 15 and a pressure sensor 13 is provided near a stop position of the user.

The stop monitoring unit 2 includes a center-of-gravity detecting unit 6, a center-of-gravity moving amount monitoring unit 7, and a stop time measuring unit 8.

The center of gravity detector 6 detects the center of gravity of the user. When a pressure sensor is used as the motion information input unit 1, the position of the center of gravity is defined as shown in FIG. In other words, the center of the rectangle (position 5c in the figure) surrounding the sensor turned on by the pressure of the shoe is set as the center of gravity.

The center-of-gravity shift monitor 7 is provided with the center-of-gravity detector 6.
Monitor the amount of movement of the center of gravity of the user obtained in the step, and if the amount of movement is within a predetermined threshold value, determine that the user has stopped. It is determined that there is. For example, the threshold value may be set to a half of a general human step length (about 20 [cm]).

The stop time measuring section 8 measures the time during which the user has stopped, and when the user continues to stop for a predetermined time (threshold), the message output request is sent to the message sentence determination. Output to section 3. Note that the predetermined time may be set to about 2 [seconds] so that the user does not rush or wait for a long time.

The message sentence determination unit 3 includes a message sentence dictionary 9 in which dialogue sentence examples are stored in advance, and a message sentence selection unit 10 for selecting an appropriate message sentence from the message sentence dictionary 9. If the message sentence selection unit 10 always selects a different message sentence using the time at that time and the random number table, a fresh conversation without making a rut becomes possible.

The message output unit 4 includes a voice synthesizing unit 11 that outputs the message selected by the message selecting unit 10 as voice, and a display unit 12 that displays the message as characters. Note that the voice synthesis unit 11
In this case, a speech waveform may be stored in advance in accordance with the purpose of use or price of the interactive device, and a device that reproduces the speech waveform or a commercially available sentence / speech conversion device may be used. The display unit 1
As 2, a device suitable for the purpose of use, such as a CRT or a liquid crystal display, may be used.

Next, the operation of the first embodiment will be described. First, when the user approaches the spoken dialogue device, this motion is detected by the motion information input unit 1. Then, the center of gravity detection unit 6 measures the position of the center of gravity of the user.

FIG. 4 is a flowchart showing the operation of the stop monitoring unit 2. The procedure for confirming the stop of the user will be described with reference to FIG.

The center-of-gravity movement monitor 7 determines whether the position of the center of gravity detected by the center-of-gravity detector 6 has moved (step ST1). If it is moving (YES in step ST1), the moving amount is measured (step ST1).
2) It is determined whether or not this movement amount is larger than a predetermined threshold value (step ST3).

If the amount of movement is larger than the threshold value (NO in step ST3), it is determined that the vehicle has not stopped, and the stop time measuring counter is reset (step ST7). On the other hand, when the movement amount of the center of gravity is smaller than the threshold value (YES in step ST3), the stop time (discrete time) is increased by one (step ST4). Also, when the center of gravity of the user is not moving, the stop time is similarly increased by one count (NO in step ST1, step ST4).

When the stop time exceeds a predetermined threshold value (for example, 2 seconds) (YES in step ST5), it is determined that there is a dialogue intention, and a message output request is determined by a message sentence determination. Output to unit 3 (step S
T6).

After that, the message sentence selecting unit 10 of the message sentence determining unit 3 selects an appropriate message sentence from the message sentence dictionary 9 and supplies it to the message output unit 4.
Then, this message is output as a voice from the voice synthesizing unit 11 and displayed as characters on the display unit 12. Thereby, the dialog between the user and the voice interactive device is started.

As described above, in the first embodiment, the stop of the user is detected, and it is determined whether or not the user intends to have a dialog, thereby outputting a message. Therefore, the dialogue is started without the user performing an operation such as pressing a button or inserting money. Therefore, it is possible for a user who does not know how to use to easily communicate.

Further, since the user does not speak to the person who merely passes near the voice interactive device, the user is not surprised.

In this embodiment, an example is shown in which the position of the user is determined by the center of gravity. However, the present invention is not limited to this, and another method may be used.

FIG. 5 is a block diagram showing the configuration of the second embodiment of the present invention. In the figure, a motion information input unit 1, a stop monitoring unit 2, a message sentence determination unit 3, a message output unit 4, and a control unit 5 have the same configuration as that of the first embodiment, and a description thereof will be omitted. I do.

In the figure, a user input section 21 receives an input from a user for a message.
It comprises a voice recognition unit 23 for recognizing voice uttered by the user, and a keyboard 24 for inputting information by key operation. The speech recognition unit 23 can be realized by an isolated word speech recognition technology if it is designed so that the dialogue is performed in a one-question, one-answer format. Also, the keyboard 24 is used for the voice recognition unit 2.
This is used when it is not possible to correctly recognize in step 3, or when it is more convenient to press a key, and a touch display or the like may be used.

The user response recognition / processing unit 22 receives the user's movement information output from the stop monitoring unit 2 and the information from the user input unit 21, and receives the user's response to the message output by the voice interactive device. Is recognized, and the subsequent processing is determined.

For example, the user's reaction is (1) leaving, (2) responding. (3) When classified into three types of no response, the processing operation in the user response recognition / processing unit 22 is as shown in FIG.
The flowchart is as shown in FIG.

That is, after outputting a message to the user,
It is determined whether or not the center of gravity of the user exists (step ST11). If there is no center of gravity (step S
(NO in T11), the user determines that the user has left, and terminates the dialog (step ST15).

On the other hand, when the center of gravity exists (step ST
11 (YES), it is determined whether or not there is an input from the user (step ST12). When there is an input from the user (YES in step ST12), the dialogue is continued (step ST16).

If there is no input from the user (NO in step ST12), step ST12 is performed until a predetermined time (for example, 5 seconds) elapses after outputting the message.
11 and ST12 are repeated (NO in step ST13), and when a predetermined time has elapsed (Y in step ST13).
ES), it is determined that there is no response from the user, and a message sentence output request corresponding to this is output (step ST1).
4).

As described above, in the second embodiment, it is possible to perform an interactive process in accordance with the response of the user to the message, and for example, it is unnecessary to output the message to the user without using it. In addition, it is possible to appropriately give advice to a user who is in trouble because he does not understand the operation.

FIG. 7 is a block diagram showing a third embodiment of the present invention, in which all parts except the stop monitoring unit 2 are shown in FIG.
Therefore, only the stop monitoring unit 2 is shown, and the others are omitted.

The stop monitoring unit 2 shown in FIG. 2 includes a center-of-gravity detecting unit 6, a center-of-gravity moving amount monitoring unit 7, and a stop time measuring unit 7.
And an area information storage unit 31 for storing information corresponding to the position where the user has stopped.

As shown in FIG. 8, for example, as shown in FIG. 8, the area information storage section 31 stores the pressure sensor 13 in the area A and the distance b up to a distance a (for example, 60 cm) in accordance with the distance from the apparatus main body 15.
(For example, up to 80 cm) is set as a region B, and up to a distance c (for example, 100 cm) is set as a region C, and information corresponding to each region is stored.

That is, when the user is in the area A, it is determined that the user is in a hurry, and the threshold value for determining the stop is set to 1.5 seconds. The threshold is set to 2.5 seconds when in the area B, and about 4 seconds in the area C in order to make a late call. This enables a conversation suitable for the user's request.

Further, it is possible to change the message text for each area. That is, in the case of the area A, a good-tempo conversation can be performed with a short sentence such as "Hello!", And in the case of the area C, the user is asked "Welcome. What happened?" If it is called in a sentence, the conversation can be started without causing any discomfort to the user who is in a hurry or the user who is hesitant.

In the third embodiment, an example is shown in which the areas A, B, and C are divided as shown in FIG.
When 5 is at a corner, as shown in FIG. 9, the regions A, B, and C may be divided by the distance from the corner. In addition, it is needless to say that the number of area divisions is not limited to three but may be any number.

FIG. 10 is a block diagram showing the configuration of the fourth embodiment of the present invention. In this figure, the same parts as those in FIG. 5 shown in the second embodiment are denoted by the same reference numerals, and description thereof will be omitted.

The ID identifying section 41 shown in FIG. 1 identifies an ID card with an oscillator, a magnetic card, and the like possessed by a user. The ID information storage unit 42 stores information of each user, for example, user name, gender, number of times of use, time of use, frequently purchased products, and the like.

[0052] For a user who uses frequently, a conversation can be immediately started without detecting a stop. In addition, it is possible to output a message such as “Mido”, “Mr. XX, come”, “I recommend XX today”.

As described above, in the fourth embodiment, an appropriate dialog according to the user becomes possible, and the usability is improved for the user.

Next, a fifth embodiment of the present invention will be described. In this example, the speed at which the user approaches the voice interactive device is detected, and a message corresponding to this speed is output.

The approach speed of the user is shown in FIG.
It can be easily obtained from the moving distance a of the position of the center of gravity and the time required for the movement. Then, when approaching at a speed of 4 [km / h] or more, which is a normal human walking speed,
What is necessary is just to make small the threshold value of the time which recognizes a stop.

Hereinafter, this operation will be described with reference to the flowchart shown in FIG. When the user approaches, the approach speed is obtained first (step ST21). Then, a time threshold for recognizing the stop is set (step ST26), and it is determined whether the user has stopped more than the time threshold (step ST22). The predetermined time at this time is a short time (for example, 1 second) when the approach speed is faster than 4 [km] per hour, and a normal time (for example, 2 seconds) when the approach speed is slow.

If the user has not stopped for a predetermined time (NO in step ST22), the user simply judges that the vehicle has passed, and does not output a message. If the vehicle has stopped for a predetermined time (YES in step ST22),
It is determined whether or not the approach speed is equal to or higher than a predetermined threshold (step ST23).
At step ST25, a normal message is output (step ST25).

On the other hand, if the approach speed is equal to or higher than the threshold value (YES in step ST23), a rush message is output (step ST24).

As described above, in the fifth embodiment, a prompt message is output quickly to a user approaching at a high speed, and the message text is a sentence corresponding to the urgency. Appropriate conversation with existing users.

Next, a sixth embodiment will be described. In this example, an operation in the case where the user comes out of the pressure sensor 13 during the dialog is shown. Hereinafter, this operation will be described with reference to the flowchart shown in FIG.

During the dialogue, it is constantly monitored whether or not the center of gravity of the user exists (step ST31). If the center of gravity is present (YES in step ST31), the dialogue is continued (step ST35). If the user goes out of the pressure sensor 13 for some reason during the conversation (NO in step ST31),
An appeal message is output to the user such as "Is it OK?" (Step ST32).

Then, it is determined whether or not a response to this message is within a predetermined time (for example, 5 seconds) (step ST33). If there is no response (step ST33)
NO). It is determined whether the center of gravity has appeared (step S
T36), if it does not appear (N in step ST36)
O), the user determines that the user has left, and ends the dialog (step ST37). If the center of gravity appears (YES in step ST36), the dialogue is continued (step ST35).

On the other hand, if there is a response within the predetermined time (YES in step ST33), it is determined whether or not the center of gravity has appeared (step ST34), and if it does not appear (NO in step ST34), another call is made. Output a message. When the center of gravity appears (step ST3)
The dialogue is continued (step ST35).

FIG. 14 is a time chart when a conversation is actually performed using the sixth embodiment. First, when the user gets on the pressure sensor 13 at time t ₁ ,
The voice output is (a) "Welcome. Order?"
And utter. On the other hand the user "Please orange juice." In (b) and say, audio output speaks as "orange juice ..." in (c), the user is outside the sensor in the middle of the time t ₂ , The voice output utters "Are you OK?" In (d).

On the other hand, the user needs to
in (e) "Oh, all right," referred to as, and return on the pressure sensor at time t _3, the audio output is speaks "It is orange juice. Thank you." at (f).

Then, if the user is silent in (g),
Conversation ends at time t _4.

As described above, in the sixth embodiment, even if the user falls outside the pressure sensor 13 by dropping money, for example, a smooth dialogue can be performed.

At the end of the dialogue, if the user goes out of the pressure sensor before the dialogue device says a greeting or a cliche, it is determined that the user has no intention of the dialogue.
You can also end the conversation.

In each of the above-described embodiments, an example in which a pressure sensor is used as the motion information input unit 1 has been described. However, for example, as shown in FIG. it can. By using this, it is possible to avoid difficulty in walking due to unevenness of the pressure sensor.
Further, if the user is detected by using the camera, the position, orientation, facial expression, and movement of the whole body of the user can be detected, so that an appropriate message can be determined based on this information.

In the above-described embodiment, the message sentence determining unit 3 determines the message sentence to be output. At the same time, if the loudness, pitch, and prosody characteristic of the voice are also added, A wider dialog is possible.

[0071]

As described above, according to the present invention, the user recognizes that the user has stopped in front of the voice interactive device and outputs a message for calling. Therefore, the dialog starts without the user performing a specific action,
It is possible for a user who does not know how to use to easily communicate.

Further, since a message for calling is not output to a person who merely passes nearby, the problem of inadvertently calling an unrelated person is solved.

Similarly, a smooth dialogue can be performed during or at the end of the dialogue.

Further, an appropriate message can be output in accordance with the position where the user stops, the user's ID information, and the user's approaching speed, so that the user is not forced or intimidated, and naturally The effect that a simple dialogue is attained is acquired.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an arrangement of a pressure sensor and a camera.

FIG. 3 is an explanatory diagram showing a principle of obtaining a center of gravity position.

FIG. 4 is a flowchart showing the operation of the first embodiment.

FIG. 5 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 6 is a flowchart showing the operation of the second embodiment.

FIG. 7 is a block diagram showing a configuration of a stop monitoring unit which is a main part of a third embodiment of the present invention.

FIG. 8 is an explanatory diagram showing a region division of a pressure sensor.

FIG. 9 is a diagram illustrating a modification of the area division of the pressure sensor.

FIG. 10 is an explanatory diagram showing a configuration of a fourth embodiment of the present invention.

FIG. 11 is an explanatory diagram showing the principle of measuring the approach speed of a user.

FIG. 12 is a flowchart illustrating an operation of changing a message according to a user's approach speed.

FIG. 13 is a flowchart showing an interactive operation when the user comes out of the pressure sensor.

FIG. 14 is a time chart showing the timing of the dialogue between the user and the voice dialogue device.

FIG. 15 is an explanatory diagram illustrating an example in which an optical sensor is used as a motion information input unit.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 motion information input unit 2 stop monitoring unit 3 message sentence determination unit 4 message output unit 5 control unit 6 center of gravity detection unit 7 center of gravity movement amount monitoring unit 8 stop time measurement unit 9 message sentence dictionary 10 message sentence selection unit 13 pressure sensor 21 User Input Unit 22 User Reaction Recognition / Processing Unit 31 Area Information Storage Unit 41 ID Identification Unit 42 ID Information Storage Unit

Continuation of the front page (56) References JP-A-59-216242 (JP, A) JP-A-57-147700 (JP, A) JP-A-61-46521 (JP, A) JP-A-1-155430 (JP) JP-A-4-25863 (JP, A) JP-A-2-47699 (JP, U) JP-A-4-58783 (JP, U) (58) Fields studied (Int. Cl. ⁷ , DB Name) G10L 13/00 G10L 15/24-15/28

Claims (5)

(57) [Claims] 1. Recognizing a voice uttered by a user, and
In a spoken dialogue device that performs operations,
Motion information that is located near the
Information input unit, and that the user has stopped at the motion information input unit
A stop monitoring unit for measuring the determined time;
Output a message when the stop time exceeds a specified threshold
And a message output unit . 2. The method according to claim 1, wherein a user's response to said message is
Recognizes the next message to be output according to the response result.
2. The voice interaction apparatus according to claim 1, further comprising means for selecting . 3. The system according to claim 2, wherein the threshold value of the stationary time is set by a user.
2. The apparatus according to claim 1, further comprising: means for changing the stop position.
Of voice dialogue system. 4. An ID identifying unit for identifying a user ID,
An ID information storage unit for storing ID information of each user;
Means for determining a user's message according to the D information.
The voice interaction device according to claim 1 provided . 5. The speed at which the user approaches the spoken dialogue device.
A speed measurement unit for measuring the degree
A contract with means for determining a message to the user
The voice interaction device according to claim 1 .