JP4811059B2 - Agent device - Google Patents

Description

  The present invention relates to an agent device that changes an agent's correspondence according to usage conditions.

  Conventionally, image display in various electric and electronic devices such as personal computers, car navigation systems, mobile phones, mobile devices such as PDA (Personal Digital Assistants), and AV (Audio Visual) devices such as VCRs and DVD players. Development and research on agent functions to display anthropomorphic agent images on the device, or perform agent voice output from the voice output device to perform operation guidance of various devices, route guidance, e-mail transmission and reception, etc. Has been done.

  In recent years, with the rapid development of information technology, the agent function has also been further developed, and agent technology has been developed that provides a more natural human response. For example, Patent Document 1 discloses an agent device in a car navigation system, which grasps a vehicle situation based on detection signals transmitted from various sensors provided in the vehicle, and determines the agent according to the type and degree of the situation. An invention for automatically changing the height, strength, sound quality, length, etc. of the voice to be uttered is disclosed.

Further, Patent Document 2 includes a voice recognition unit that recognizes a user's voice, and autonomously determines the user's emotion that changes with the strength and time of the voice recognized by the voice recognition unit, and is appropriate for the situation. An invention related to an agent interface that automatically executes an action is disclosed.
JP 11-259271 A JP 2005-222331 A

  By the way, as disclosed in Patent Document 1 and Patent Document 2, the invention of the agent technology for changing the behavior of the agent to the behavior suitable for the user according to the user's mood and the situation on the spot has been made. However, the present situation is that there are not so many inventions related to the agent function of more positively acting against the user.

  By executing the repulsive response, the agent's response, which tends to be uniform, can be made more expressive. The agent itself can become more human and the user can be more familiar.

The present invention has been made in order to solve the above-mentioned problems, and its object is as follows.
It is to make the agent function more suitable by making the agent execute repulsive behavior to make the agent feel closer.

In order to solve the above-described problem, the invention according to claim 1 includes a display unit that displays a plurality of image data indicating the state of an anthropomorphic agent;
Storage means for storing a plurality of audio data including the image data and specific audio data including gratitude ;
Voice synthesis means for synthesizing voice data stored in the storage means;
Voice output means for outputting the voice data synthesized by the voice synthesis means;
An input means for inputting an external instruction;
Voice recognition means for recognizing voice data among instructions input by the input means ;
Detection means for detecting a speech data input before Symbol speech recognition means,
Timing means for performing timed the audio data from being detected by the detection means to the next audio data is detected,
Determining means for determining whether or not the time measured by the time measuring means exceeds a predetermined time;
Wherein when it is determined to exceed the predetermined time by the judging means, the external control means to further outputs audio data to prompt an input by said input means to said audio output means by synthesized by the voice synthesis section and, with a,
The control means temporarily stops outputting the audio data from the audio output means when the determination means determines that a predetermined time is exceeded after the output of the audio data prompting the input by the input means. Let
After the output of the voice data from the voice output unit is temporarily stopped, when the voice data is recognized by the voice recognition unit, the voice data recognized by the voice recognition unit and the storage unit Compare with the stored specific audio data to determine whether they match,
If the voice data recognized by the voice recognition unit determines that matches the particular voice data, and wherein Rukoto restarts the output of the audio data from the temporarily stopped the voice output unit To do.

The invention according to claim 2 is the agent device according to claim 1 ,
The control means includes
Wherein when it is determined to exceed the predetermined time by the judging means, characterized in that to further display the image data for prompting an input by the input means on the display means.

The invention according to claim 3 is the agent device according to claim 1 ,
The control means includes
If it is determined to exceed the predetermined time by the determination means, the image data of the voice data urging input by said input means in place of the Rukoto is more output to the audio output means, prompting the user to input by said input means characterized in that that presents on the display means.

The invention according to claim 4 is the agent device according to claim 2 or 3 , wherein
The image data that prompts the input by the input means is one that shows a cast or boring aspect of the anthropomorphic agent.

According to the present invention, it is provided with time measuring means for measuring the time from detection of the voice data input by the detection means until detection of the next voice data, and when this time exceeds a predetermined time by the determination means. In the determination, a voice prompting a dialogue with the agent can be outputted from the voice output means. For example, by producing a repulsive presentation to the user by outputting a voice indicating that it is in a timeless or boring state, it is possible to make the response of agents that tend to be uniform, surprisingly, The user can be more interested in the interaction with the agent, and the agent function can be made more suitable.
In particular, the image display means is further provided to display an image in which the agent is in a tedious state or a boring state, whereby the user can be more interested in the agent.

Next, the best mode for carrying out the present invention will be described with reference to the drawings.
The agent device according to the present invention can be applied to various electronic devices as long as the driving state of the electrical / electronic device can be detected. For example, the present invention can be applied to various mobile devices such as mobile phones, various electronic devices such as car navigation systems and desktop personal computers. The driving state of the electric / electronic device is, for example, when detecting input signals from various input devices (user interfaces) that transmit input signals by user's own operation such as a remote control device, a keyboard or a mouse, This means that the input signal can be detected by various sensors (for example, a temperature sensor that detects heat generated by driving the electronic device) that can detect a change in the external environment of the electronic device. That is, it is only necessary to detect the state in which the electric / electronic device is used by operating these interfaces and various sensors. In this embodiment, an example in which the agent device of the present invention is applied to a mobile phone will be described.

The functional configuration of a mobile phone 100 to which the present invention is applied is shown in the block diagram of FIG. The cellular phone 100 includes a cellular phone control unit 20, a communication / call function unit 21, a display unit 22, an input unit 23, an agent CPU 1, a RAM 2, a storage unit 3, a DSP (Digital Signal Processor) 4, a microphone 5, a sound source IC 6, and a speaker. 7 and a clock circuit 8, which are electrically and electronically connected by a bus 9.
Of course, the agent CPU 1 can be configured integrally with the mobile phone control unit 20, but for simplicity, both are configured separately.

  The mobile phone control unit 20 includes a CPU, a ROM, a RAM, and the like, and performs overall control of the mobile phone 100 in accordance with instructions of application programs such as an operation program, a voice call program, and a mail transmission / reception program stored in advance. Display control of the display unit 22 is also performed, and control for receiving the image data of the agent transmitted from the agent CPU 1 and displaying it on the display unit 22 is also performed. Furthermore, control is also performed to transmit an input signal from the input unit 23 composed of various keys for inputting telephone numbers and characters to the agent CPU 1.

  The agent CPU 1 reads the operation program and the agent program 10 stored in the storage unit 3, develops them in the RAM 2 as a work area, and performs control over the agent function in cooperation with these programs. More specifically, “agent function temporary stop processing” described later is performed in accordance with instructions from the operation program and the agent program 10.

  The “agent function suspension process” is an agent that measures the time when the mobile phone 100 is not used and outputs the agent image to be displayed on the display unit 22 or the speaker 7 according to the unused time. This is a process of temporarily suspending the agent function when the use of the mobile phone 100 is urged through the voice and the mobile phone 100 is not used in spite of such a urgency. Further, after the agent function is temporarily stopped, a process of requesting a predetermined action from the user and returning the agent function when the request is satisfied is also included.

  The storage unit 3 stores various programs and data such as an operation program that performs an agent function, an agent program 10, image data 11, audio data 12, and input audio sample data 13.

  The agent program 10 is an application program that performs “agent function temporary stop processing” in cooperation with the agent CPU 1. The agent program 10 stores threshold time data used as a reference when the agent CPU 1 determines a time when the mobile phone 100 is not used. More specifically, the agent CPU 1 measures time with a counter (not shown) triggered by detection of an input signal input via the input unit 23. The agent CPU 1 keeps timing until the next input signal is detected. The agent CPU 1 monitors whether or not the measured time exceeds the threshold time data, and if it exceeds, the image data of the agent is displayed on the display unit 22 or the speaker 7. And / or audio data. In the present embodiment, three threshold time data from first to third are stored as an example.

  As the image data 11, image data is stored that indicates that the anthropomorphic agent is in a tedious state or a boring state. 2A to 2D show examples of the image data 11. FIG. 2A shows an agent image displayed on the display unit 22 when the agent function is operating normally (that is, when the mobile phone 100 is frequently used). The image may be a still image, but is preferably a moving image in order to make the user feel closer. When the mobile phone 100 is used normally, a facial expression that smiles or smiles is displayed as in the case of a conventional agent.

  FIG. 2B shows an image of an agent yawning. When the mobile phone 100 is not used much, an image of the agent who is freed up is displayed, so that the actual usage status of the mobile phone 100 and the image of the agent can be ideally linked.

  FIG. 2C shows an image of the agent sleeping. Similarly to the case shown in FIG. 2B described above, the agent image can be ideally linked with the actual usage status of the mobile phone 100. Furthermore, in general, the action of “yawn” and the action of “sleep” are more prominent in expressing the state of dullness and boredom when the action of “sleeping” is expressed in a dull state and bored state. To express. Therefore, when the mobile phone 100 has not been used for a longer time, an actual usage situation and agent that the mobile phone 100 has not been used for a longer time by displaying an image as shown in FIG. 2C. It is possible to more ideally link these images.

  FIG. 2D shows an image in which the agent “runs away”, “runaway”, “missing” or “disappears”. Similar to FIGS. 2B and 2C described above, the image of the agent can be conceptually linked to the actual usage status of the mobile phone 100. In particular, actions such as “run away”, “escape”, “missing”, or “disappearance” are actions that remind the other party of feelings of exhaustion. Therefore, human repulsion can be produced as an expression of the agent, and the user's interest in the agent can be more strongly attracted.

  As the voice data 12, voice data is stored that indicates that the agent is in a tedious state or a boring state. By uttering such voice, the user can be prompted to use the mobile phone 100. FIG. 3 shows an example of the audio data 12. The audio data 12 is “Ah… It ’s time…”, “Oh! Play! ! ”And the like are in a timeless or boring state. In the “agent function pause process”, the agent CPU 1 performs a voice synthesis process in accordance with an instruction of the program, and outputs such a dull state or a boring state from the speaker 7. In the speech synthesis process, the agent CPU 1 reads speech data that reads out the speech of the agent as shown in FIG. 3 based on phoneme data representing the waveform of the phoneme and the speech piece data that is data representing the waveform of the word. Synthesize. As a synthesis method, a recording editing method or a rule-based synthesis method can be used. The recording method is a method in which speech in units of words read out by a person in advance is made into a database, and these are connected and output. The rule synthesis method is a method in which relatively small units such as phonemes (consonants and vowels) and kana are connected and output.

The input voice sample data 13 is obtained by digitalizing a specific language serving as a trigger when the agent function is paused by the “agent function pause process” and then returned (resumed). When the agent CPU 1 acquires the voice data from the user input via the microphone 5 and the DSP 4, the agent CPU 1 compares the voice data with the sample voice data stored in the input voice sample data 13, and matches or a predetermined threshold (ratio). Search for sample audio data that matches as the number exceeds. As a result, by detecting sample voice data that matches (or matches so as to exceed a predetermined threshold), it recognizes that the input voice is in a specific language and restores the agent function. .
As the input voice sample data 13, voice data in a specific language whose content is an apology from the user to the agent is stored. FIG. 4 shows an example of input voice sample data. "I'm sorry", "Please!" Come back! ”,“ I am bad ”, etc., and“ Ah, the train is delayed ”,“ What's overtime now, ” Classified) and stored.

The DSP 4 is an audio processing device that samples digital audio from a user via a microphone 5 at a predetermined cycle via an A / D converter (not shown) and generates digital audio data. The generated voice data is compared with the input voice sample data by the agent CPU 1 and is recognized as voice when both data match or match a predetermined threshold value or more.
The microphone 5 acquires sound from a user or the like and outputs it to the DSP 4 as an analog signal. Note that a transmitter (not shown) of the mobile phone 100 may also be used as the microphone 5.

  The sound source IC 6 is an integrated circuit that generates analog audio data from the audio data transmitted from the agent CPU 1. The generated analog audio data is output from the speaker 7. In the present embodiment, the agent CPU 1, the voice IC, and the speaker 7 function as voice output means.

  The clock circuit 8 is composed of an element such as a crystal resonator and outputs a signal to the agent CPU 1 at a constant period. The agent CPU 1 can measure time by continuing to receive this signal. In the present embodiment, the agent CPU 1 and the clock circuit 8 function as time measuring means.

  The display unit 22 displays an agent image that is anthropomorphic based on the image data 11 and various images that are provided in the call function of the mobile phone 100.

Next, the operation of the mobile phone 100 will be described using the flowchart shown in FIG. Although the following processing may be directly performed by a processor or the like (not shown), for the sake of simplicity, description will be made assuming that the agent CPU 1 performs the processing according to a program instruction.
Note that, as the above-described threshold time data serving as a reference for detecting the time when the mobile phone 100 is not used, the first threshold time data is “2 hours”, the second threshold data is “10 hours”, 3 The threshold time data is “36 hours”.

In step S <b> 101, the agent CPU 1 detects an input signal transmitted from the input unit 23 via the mobile phone control unit 20. The fact that the input signal is transmitted from the input unit 23 composed of various keys means that the user is actually using the mobile phone 100.
In step S <b> 102, the agent CPU 1 starts measuring time based on the signal transmitted from the clock circuit 8 using the detection of the input signal in step S <b> 101 as a trigger.

  In step S103, the agent CPU 1 monitors the input signal from the input unit 23 received via the mobile phone control unit 20, and determines whether or not to further detect the input signal after starting the timing in step S102. Do. When the agent CPU1 determines that the input signal is not further detected, the agent CPU1 proceeds to step S104 (step S103: NO). Conversely, if the agent CPU 1 determines that an input signal is detected, the agent CPU 1 returns to step S102, resets the counter that has timed until then, and starts time measurement again (step S103: YES).

  In step S104, the agent CPU 1 compares the elapsed time from the start of time measurement in step S102 with the first threshold time data (2 hours) stored in the agent program 10 in advance. It is determined whether or not the first threshold time data is exceeded. When determining that the elapsed time exceeds the first threshold time data, the agent CPU 1 proceeds to step S105 (step S104: YES). Conversely, when the agent CPU 1 determines that the elapsed time does not exceed the first threshold time data, the agent CPU 1 returns to step S103 (step S104: NO).

  In step S105, the agent CPU 1 synthesizes predetermined audio data from the audio data 12 stored in the storage unit 3 and transmits the synthesized audio data to the sound source IC, and the audio (for example, “Ah ... Dana ... ”) is output.

  In step S106, the agent CPU 1 monitors the input signal input from the input unit 23 via the mobile phone control unit 20 again, and determines whether to detect this input signal. If the agent CPU 1 determines to detect the input signal, the agent CPU 1 returns to step S102, resets the counter, and starts counting again. On the contrary, if the agent CPU 1 does not detect the input signal, the agent CPU 1 compares the second threshold time data (10 hours) stored in the agent program 10 in advance, and whether the elapsed time exceeds the second threshold time data. Make a decision. When the agent CPU1 determines that the elapsed time exceeds the second threshold time data, the agent CPU1 proceeds to step S108 (step S107: YES). Conversely, when the agent CPU 1 determines that the elapsed time does not exceed the second threshold time data, the agent CPU 1 returns to step S106 (step S107: NO).

  In step S108, the agent CPU 1 reads the image data 11 (for example, the image data of FIG. 2B or 2C) stored in the storage unit 3, and displays the anthropomorphized agent image on the display unit 22. To do.

  In step S109, the agent CPU 1 again monitors the input signal input from the input unit 23 via the mobile phone control unit 20, and determines whether to detect the input signal. When the agent CPU1 does not detect the input signal, the agent CPU1 proceeds to step S110 (step S109: NO). On the other hand, when detecting the input signal, the agent CPU 1 returns to step S102, resets the counter that has timed until then, and starts counting again (step S109: YES).

  In step S110, the agent CPU 1 compares the elapsed time from the start of timing in step S102 with the third threshold time data (36 hours) stored in the agent program 10 in advance, and the elapsed time is It is determined whether or not the third threshold time data is exceeded. When determining that the elapsed time exceeds the third threshold time data, the agent CPU 1 proceeds to step S111 (step S110: YES). Conversely, when the agent CPU 1 determines that the elapsed time does not exceed the third threshold time data, the agent CPU 1 returns to step S109 (step S110: NO).

  In step S111, the agent CPU 1 reads the image data 11 (image data in FIG. 2D) stored in the storage unit 3, and displays the image on the display unit 22.

In step S112, the agent CPU 1 determines whether or not to continuously detect an input signal from the input unit 23 within a predetermined time. When the agent CPU 1 does not detect the input signal continuously, the agent CPU 1 is in a standby state until it detects the continuous signal (step S112: NO). That is, the agent CPU 1 continues to display the image displayed in step S111 (FIG. 2D) until the input signal is continuously detected. Thereby, when the agent “runs away” or the like, it is possible to perform an effect of acting a repulsive attitude toward the user who does not return until the predetermined condition is satisfied.
Conversely, if the agent CPU 1 detects an input signal continuously from the input unit 23 within a predetermined time, the process proceeds to step S113 (step S112: YES).

In step S113, the agent CPU 1 determines whether or not a voice that matches the input voice sample data 13 (or matches a predetermined threshold or more) is recognized from the voice input through the microphone 5. That is, since a word of apology is stored as the input voice sample data 13, the user needs to input a predetermined apology word to the agent via the microphone 5. Thereby, when the agent “runs away” or the like, the agent that does not come back until the predetermined condition is satisfied can be more effectively performed such that the agent behaves repulsive toward the user.
If the determination in step S113 is NO, that is, if the agent CPU1 does not recognize an apology, the process returns to step S112. By configuring in this way, when the agent “runs away” or the like, it is possible to more effectively perform an effect of acting a more repulsive attitude toward the user that the agent does not return until two predetermined conditions are satisfied. .

  In step S114, the agent CPU 1 cancels the temporary stop function of the agent function and returns to the normal processing state. That is, the image shown in FIG. 2A is displayed on the display unit 22 and the agent function in various processes of the mobile phone 100 is executed.

  As described above, according to the mobile phone 100 to which the present invention is applied, the elapsed time from the detection of the input signal input from the user via the input unit 23 to the detection of the next input signal is a predetermined threshold value. Judgment whether or not the time is exceeded, and if it exceeds the time, the agent displays a picture that is in a dull state or a dull state, or outputs a sound that indicates a dull state or a dull state. Thus, it is possible to produce an effect of being repulsive to the user. This makes it possible to realize an agent function that performs a more human operation.

  In addition, the mobile phone 100 displays a rebellious image or outputs a sound to the user in a state of being in a dull state or bored based on the time when the mobile phone 100 is not actually used. It is supposed to be. Therefore, the non-use time of the mobile phone 100 and the operation of the agent are linked conceptually, and the effect of the original agent function that makes it appear as if the mobile phone 100 itself has a human intention is more effectively performed. be able to.

  In addition, the mobile phone 100 displays an image that causes the agent to “run away” based on the non-use time of the mobile phone 100 (finally). By causing the human operation of “run away” to occur, the user can be reminded of feelings as if the agent had exhausted his / her affection. Thereby, the unexpectedness with respect to the agent can be improved, and the interest in communication with the agent can be increased.

  In addition, after causing the agent to “run away”, the mobile phone 100 temporarily detects the input signal from the input unit 23 at predetermined intervals and continues to temporarily display the agent function until the user's apology is recognized. It stops functioning. In order to cancel the temporary function suspension state, it is configured to require the user to perform such an action that occurs between actual humans, thereby bringing the relationship between the agent and the user closer to equality. There is an effect that can be.

As mentioned above, although the best form for implementing this invention was demonstrated, this invention is not limited to the said various example. In particular, in the above-described example, the repulsive image data and audio data for the user have been described as having contents of a dull state or a dull state, but an angry state, a displeased state, a neglected state, a selfish state Of course, it is possible to apply a state that reminds the user of a repulsive feeling, such as a state, a state of sadness, and a state of teasing.
Similarly, in order to return the agent function to the normal function state after falling into the temporary function stop state, it is configured to require continuous pressing of dial keys and the input of an apology word. It is not a thing. For example, as shown in FIG. 4, an excuse may be entered instead of an apology, or a dial key may be pressed and an excuse may be requested after an apology, or vice versa. Of course, it is possible to apply various forms such as a configuration for recognizing the above word continuously a predetermined number of times.

It is the block diagram which showed the structure of the mobile telephone provided with the agent apparatus of this invention. FIGS. 2A to 2D are schematic views showing examples of agents displayed on a display unit of a mobile phone provided with the agent device of the present invention. It is the schematic diagram which showed the example of the audio | voice data synthesize | combined and output with the mobile telephone provided with the agent apparatus of this invention. It is the schematic diagram which showed the example of the input audio | voice sample data stored in the mobile telephone provided with the agent apparatus of this invention. It is the flowchart which showed the process sequence of the mobile telephone provided with the agent apparatus of this invention.

Explanation of symbols

1 Agent CPU
2 RAM
3 Storage unit 4 DSP
5 Microphone 6 Sound source IC
7 Speaker 8 Clock circuit 9 Bus 10 Agent program 11 Image data 12 Audio data 13 Input audio sample data 20 Mobile phone control unit 21 Communication / call function unit 22 Display unit 23 Input unit

Claims (4)

Display means for displaying a plurality of image data indicating the state of the anthropomorphic agent;
Storage means for storing a plurality of audio data including the image data and specific audio data including gratitude ;
Voice synthesis means for synthesizing voice data stored in the storage means;
Voice output means for outputting the voice data synthesized by the voice synthesis means;
An input means for inputting an external instruction;
Voice recognition means for recognizing voice data among instructions input by the input means ;
Detection means for detecting a speech data input before Symbol speech recognition means,
Timing means for performing timed the audio data from being detected by the detection means to the next audio data is detected,
Determining means for determining whether or not the time measured by the time measuring means exceeds a predetermined time;
Wherein when it is determined to exceed the predetermined time by the judging means, the external control means to further outputs audio data to prompt an input by said input means to said audio output means by synthesized by the voice synthesis section and, with a,
The control means temporarily stops outputting the audio data from the audio output means when the determination means determines that a predetermined time is exceeded after the output of the audio data prompting the input by the input means. Let
After the output of the voice data from the voice output unit is temporarily stopped, when the voice data is recognized by the voice recognition unit, the voice data recognized by the voice recognition unit and the storage unit Compare with the stored specific audio data to determine whether they match,
If the voice data recognized by the voice recognition unit determines that matches the particular voice data, and wherein Rukoto restarts the output of the audio data from the temporarily stopped the voice output unit Agent device to perform. The agent device according to claim 1 ,
The control means includes
Wherein when it is determined to exceed the predetermined time by the judging means, the agent device, characterized in that to further display the image data for prompting an input by the input means on the display means. The agent device according to claim 1 ,
The control means includes
If it is determined to exceed the predetermined time by the determination means, the image data of the voice data urging input by said input means in place of the Rukoto is more output to the audio output means, prompting the user to input by said input means the agent device for causing displayed in said display means. The agent device according to claim 2 or 3 , wherein
The agent device characterized in that the image data that prompts input by the input means shows a cast or boring aspect of the personified agent.

Images