JP7664606B2 - Systems etc. - Google Patents

Description

The present invention relates to a system, etc.

Patent Document 1 describes a stereoscopic image display device that does not require special glasses. This stereoscopic image display device includes a light source, a liquid crystal display panel, and a parallax barrier that is arranged on the viewer's side of the liquid crystal display panel.

JP 2003-295113 A

One of the objectives of the present invention is to provide a technique for outputting information associated with a character.

The purpose of the invention of this application is not limited to this, and the applicant intends to obtain rights to configurations that aim to obtain effects from parts of the configuration disclosed in this specification and drawings, etc., through divisional applications, amendments, etc. For example, this specification discloses problems in which the phrase "can be" in this specification is read as "the problem is that...". The problems are described as independent, and the applicant intends to obtain rights to the configurations for solving each problem separately through divisional applications, amendments, etc. Even if the problems are implicitly understood from the description in the specification, the applicant intends to include part of the configuration described in this specification in the scope of the patent claim by amendment or divisional application. In addition, the applicant has disclosed a configuration that solves a problem that combines these independent problems, and the applicant intends to obtain rights to it.

(1) It is desirable to provide a system that includes a display unit capable of displaying a character, and a control means having a function of changing the experiential information of the character and a function of changing the behavior of the character in accordance with the experiential information. A change in the character's experiential information can be, for example, an increase in the experiential information, but it is also possible that the experiential information can be decreased.

In this way, you can enjoy watching the virtually displayed character's movements change in response to the character's experiential information.

(2) The control means may have a function of displaying a behavioral pattern as if the character were acting according to a virtual instinct function possessed by the character.

In this way, the character's movements are displayed as if it were acting using the virtual instinct function, allowing you to enjoy everyday life with a realistic character that seems to actually exist.

(3) The character may be an imitation of an animal, and the control means may have a function of displaying a behavior as if the character is performing alert or threatening behavior due to a virtual instinct function possessed by the character.

In this way, the characters can be realistically expressed as if they possessed the instinctual functions of animals.

(4) The control means may have a function to display a three-dimensional virtual space on the display unit, and a function to display the movement of the character within the virtual space.

In this way, it is possible to display the movements of a character moving through a three-dimensional virtual space, creating a sense of presence as if the virtually displayed character actually exists in reality.

(5) The control means may have a function to divide the area into a plurality of areas with different senses of distance and display them on the display unit, and a function to display the movement of the character between the plurality of areas.

In this way, the character moves between multiple areas with different senses of distance when viewed from the front, making it appear as if the character is moving away from or towards the system user, for example, making it possible to display an even more realistic character.

(6) The control means may have a function of displaying the character moving between a first position, which is the farthest, and a second position, which is the closest, within a range in which the character can be displayed, when the user looks at the display unit.

This way, it becomes possible to clearly understand how the character moves away from or towards the user.

(7) The control means may have a function to change the experience information when the user takes an action, and a function to control the character's behavior pattern to change when the experience information is changed. An action taken by the user may be, for example, feeding the character, but may also be the amount of time the user has used the system 1, the number of times it has been used, the number of times the character has been spoken to, the number of times the user has taken an action, or a combination of these. Also, the character's behavior pattern may change, for example, when the experience information increases, but may also change when the experience information decreases, or a combination of these.

In this way, when the user takes action, the value of the experience information is changed upward, and when the experience value is changed upward, a display is displayed indicating that the intimacy level with the character has increased. In this way, it is possible to give the user a sense of closeness to the character.

The control means may have a function of displaying the character so that the frequency (e.g., probability) of the character moving between a relatively distant position and a relatively close position when the user looks at the display unit changes as the user takes an action. As described above, the action taken by the user is, for example, feeding the character, but it may also be the amount of time the user has used the system 1, the number of times it has been used, the number of times the user has spoken to the character, the number of times the user has taken an action, or a combination of these. In addition, when the user takes an action, if the action taken by the user is one that the character prefers, the probability of the character moving from a relatively distant position to a relatively close position, for example, is increased, but if the action taken by the user is not one that the character prefers, the probability of the character moving from a relatively close position to a relatively distant position, for example, is increased.

In this way, if the action taken by the user is in line with the character's preferences, the character will be displayed closer to the user and approaching the user, whereas if the action taken by the user is not in line with the character's preferences, the character will be displayed farther away from the user and moving away, making for a more realistic experience.

The value of the experience information may be changed upward by a control means included in the system, or may be changed upward by, for example, an external server provided separately from the system.

(8) The control means may have a function of displaying the character at a first position that is the furthest from the range in which the character can be displayed when the user looks at the display unit when the value of the experience information is an initial value.

This way you can start by familiarizing your unfamiliar character.

(9) The control means may have a function of making the rate at which the character moves to a first position that is the furthest when the display unit is viewed within a range in which the character can be displayed, higher than the rate at which the character moves to a second position that is the closest when the display unit is viewed, during a first stage in which the value of the experience information ranges from an initial value to a first specified value.

In this way, in the first stage in which the value of the experience information changes from the initial value to the first specified value, the percentage of the character staying at the first position can be made higher than the percentage of the character staying at the second position. This allows the user to embody the fact that the character is not accustomed to the initial value for a while.

The first stage corresponds to, for example, "low intimacy" in the specification, the first specified value corresponds to, for example, "30" in the specification, the first position corresponds to, for example, "area 5" in the specification, and the second position corresponds to, for example, "area 1" in the specification. The same applies below.

(10) The control means may have a function of decreasing a rate at which the character moves to a first position, which is the furthest position when the display unit is viewed within a range in which the character can be displayed, in accordance with a change in the value of the experience information, during a first stage in which the value of the experience information ranges from an initial value to a first specified value. The rate at which the character moves to the first position is decreased, for example, as the value of the experience information increases, but may also be decreased as the value of the experience information decreases, for example, when an unfriendly parameter is used.

In this way, in the first stage, the rate at which the character moves to the first position can be reduced as the value of the experience information changes. This allows the user to embody the process in which the character becomes less wary and more accustomed to the situation as the value of the experience information changes.

(11) The control means may have a function of increasing a movement rate of the character to a second position that is closest when viewed from the display unit within a range in which the character can be displayed, in a first stage in which the value of the experience information ranges from an initial value to a first specified value, in accordance with a change in the value of the experience information.

In this way, in the first stage, the character's rate of movement to the first position can embody the process in which the character becomes accustomed to changes in the experience information.

(12) The control means may have a function of increasing the degree of increase in the movement rate of the character to the second position in response to a change in the value of the experience information in the first stage.

In this way, the user can embody the process in which the character becomes accustomed to the change in experience information as it changes from the initial value to the first specified value through the character's rate of movement to the first position.

(13) The control means may have a function of decreasing the rate at which the character moves to a first position that is the furthest when the display unit is viewed within a range in which the character can be displayed, in a second stage in which the value of the experience information ranges from a first specified value that is greater than the initial value to a second specified value, in accordance with a change in the value of the experience information.

In this way, in the second stage, the rate at which the character moves to the first position can be decreased as the value of the experience information changes. In this way, the process in which the character's vigilance is relaxed can be embodied by the rate at which the character moves to the first position for the user.

The second stage corresponds to, for example, "medium intimacy" in the specification, and the second specified value corresponds to, for example, "60" in the specification. The same applies below.

(14) The control means may have a function of increasing the rate at which the character moves to a second position that is closest to the display unit when the user looks at the display unit within a range in which the character can be displayed, in a second stage in which the value of the experience information is greater than the initial value, from a first specified value to a second specified value, in accordance with a change in the value of the experience information.

In this way, in the second stage, the rate at which the character moves to the second position can be increased as the value of the experience information changes. In this way, the process in which the character becomes accustomed to the new position can be embodied by the rate at which the character moves to the second position.

(15) The control means may have a function of increasing the degree of increase in the character's movement rate to the second position in the second stage in accordance with a change in the value of the experience information.

In this way, the user can embody the accelerating process of the character's familiarity in the second stage through the rate at which the character moves to the second position.

(16) The control means may have a function of decreasing the rate at which the character moves to a first position that is the furthest when the user looks at the display unit within a range in which the character can be displayed in response to a change in the value of the experience information at a specific stage in which the value of the experience information exceeds a specific specified value, and of decreasing the degree of decrease in the rate at which the character moves to the first position in response to a change in the value of the experience information.

In this way, the rate at which the character's movement rate to the first position decreases can be slowed down at a specific stage, allowing the character's movements to be displayed more realistically.

(17) The control means may have a function of increasing the rate at which the character moves to a second position that is closest to the character when the user looks at the display unit within a range in which the character can be displayed in response to a change in the value of the experience information at a specific stage in which the value of the experience information exceeds a specific specified value, and decreasing the degree of increase in the rate at which the character moves to the second position in response to a change in the value of the experience information.

In this way, the rate at which the character moves towards the second position can be slowed down at a specific stage, allowing the character's movements to be displayed more realistically.

(18) The control means may have a function of determining whether or not to move the position of the character.

By doing this, the timing of the character's movements can be made random, making it appear as if the character is moving of its own volition.

(19) The control means may have a function of determining the timing of moving the character by lottery.

This will allow the timing of character movements to be more random.

(20) The device may include a sensor capable of detecting the approach of a person, and the control means may have a function of moving the character to a relatively distant position within the range in which the character can be displayed when the person is viewing the display unit, when the sensor detects the approach of the person.

This way, the character can be displayed as running away when a person approaches, and this type of character movement can make the image appear even more realistic.

(21) The control means may have a function of changing the experience information downward.

By doing this, it is possible for experience information to decrease rather than increase unilaterally, making it even more realistic.

(22) The value of the experience information represents the intimacy of the character, and the control means has a function of increasing the frequency with which the character is moved to a relatively closer position within a range in which the character can be displayed when the user views the display unit, the greater the value of the experience information indicating the higher the intimacy.

By doing this, the higher the intimacy level, the more frequently the character will move to a position relatively closer to the user, making for an even more realistic experience.

(23) The control means may have a storage means for storing a preset language, a language recognition function capable of recognizing a language from outside, and a counting function for counting the number of times the language is recognized if the language recognized by the language recognition function is a language stored in the storage means, and may be configured to allow the character to respond if the counted value counted by the counting function reaches a specified value.

By doing this, when the count value counted by the counting function reaches a specified value, the character will be able to respond, making it possible to communicate with the character.

When the count value counted by the counting function reaches a specified value, the control means of the system may allow the character to respond, or data may be sent and received between the system and, for example, an external server provided separately from the system, and the external server may then allow the character to respond.

(24) When the count value counted by the counting function reaches a specified value and the value of the experience information reaches a predetermined specified value, a response by the character may be permitted.

In this way, even if the count value counted by the counting function reaches a set value, the character will not be allowed to respond unless the value of the experience information reaches a set value. This allows communication only with familiar characters.

(25) The control means may have a function of causing the character to respond at a position within a range in which the character can be displayed that is relatively close when the user looks at the display unit.

In this way, the character responds from a relatively close position, giving the user a sense of closeness to the character.

(26) The control means may have a repetition function for causing the character to repeat the language recognized by the language recognition function if the language is a language stored in the storage means.

This allows the character to repeat the language recognized by the language recognition function, giving the user a sense of closeness to the character.

(27) The repetition function may have a function of having the character repeat the language at a position within the range in which the character can be displayed that is relatively close when the display unit is viewed.

In this way, the character responds from a relatively close position, giving the user a greater sense of closeness to the character.

(28) The control means may enable the repetition function when the value of the experience information reaches a predetermined value.

In this way, the character will not be allowed to respond unless the experience information value reaches a certain specified value. This will further deepen the bond between the user and JUNO.

(29) The control means may have a function of establishing a communication connection with an external mobile terminal and a function of setting up a connection with the mobile terminal upon receiving a connection request from the mobile terminal.

This will increase convenience for users.

(30) The control means may have a function of connecting to an external mobile device for communication and a function of, when receiving instruction information from the mobile device, causing the character to act based on the instruction information.

This allows the user to control the character by issuing commands from the mobile device, improving user convenience.

(31) The control means may have an alarm function that performs an alarm action according to settings, and a function that executes the alarm action according to the action of the character.

In this way, convenient alarm actions can be performed by the character's movements, allowing you to enjoy daily life with the character while also increasing convenience.

(32) The control means may have a home appliance control function for executing home appliance control in response to a request, and a function for executing the home appliance control by the movement of the character.

In this way, convenient home appliances can be controlled by the character's movements, allowing users to enjoy daily life with the character while also increasing convenience.

(A1) It is preferable to provide a display device having a housing with a light-shielded image display space therein and an image observation area on a first direction side of the image display space, and an image display unit that focuses an aerial image in the image display space.

In this way, the floating image that is combined with the image display space is displayed in a relatively dark space when viewed from the observer's side, which enhances the three-dimensional effect of the floating image that can be given to the observer. This enhances the visual effect of the display device.

(A2) It is preferable to provide a display device having a light-reducing member that reduces external light that enters the image display space from the observation area.

In this way, the light-reducing member reduces the amount of external light entering the image display space, allowing the image display space to be kept dark, enhancing the three-dimensional effect of the aerial image that can be presented to the viewer.

(A3) The image display unit may include a first display unit that emits a first image light, and an optical system that forms the aerial image based on the first image light, and may be a display device that has a light-shielding member between the light-reducing member and the optical system on the optical path of the light reflected by the light-reducing member from the first image light.

In this way, it is possible to prevent the image displayed in the image display space due to the first image light being reflected by the dimming member from being seen by the observer.

(A4) The light blocking member may be disposed in a position that does not overlap the observation area when viewed from the first direction.

In this way, even if the observer looks into the image display space from the observation area, the image displayed in the image display space due to the first image light being reflected by the dimming member can be prevented from being observed by the observer.

(A5) The image display unit may include a first display unit that emits a first image light, and an optical system that forms the aerial image based on the first image light, and the optical system may be a display device that is positioned to avoid an area on the optical path of the reflected light of the first image light that is reflected by the dimming member.

In this way, it is possible to prevent the image displayed in the image display space due to the first image light being reflected by the dimming member from being seen by the observer.

(A6) The optical system may include an optical element having a first surface on which the first image light is incident and a second surface opposite the first surface, and the optical element may be a display device that reflects the first image light toward the first direction and transmits light incident on the second surface toward the first surface.

In this way, the second surface of the optical member can be superimposed on the aerial image combined using the optical member, allowing the viewer to view the second surface of the optical member, thereby further enhancing the three-dimensional effect of the aerial image that can be provided to the viewer.

(A7) It is preferable that the display device has a second display unit that emits a second image light that overlaps with the aerial image when viewed from the first direction.

This allows for greater freedom in selecting the image that the viewer can view by overlaying it onto the aerial image.

(A8) It is preferable that the display device has a higher emission luminance of the first display unit than the emission luminance of the second display unit.

In this way, the difference in luminance between the first display section and the second display section can enhance the three-dimensional effect of the aerial image that can be conveyed to the observer.

(A9) The first display unit may be a display device that is positioned above the observation area with the display surface facing downward and is tilted so that the second direction side opposite the first direction is lower than the first direction side.

In this way, an aerial image can be formed using the optical element by the first display unit, which is positioned above the observation area.

(A10) The display surface of the first display unit may be a display device that is inclined at approximately 20 degrees relative to the horizontal direction.

In this way, the three-dimensional effect of the aerial image that can be given to the observer can be enhanced, and the image displayed in the image display space can be made more difficult to observe due to the first image light being reflected by the dimming member.

(A11) The second display unit may be a display device that is positioned below the observation area with the display surface facing upward.

In this way, the image displayed on the second display surface appears to be located below the aerial image, further enhancing the three-dimensional effect of the aerial image that can be given to the viewer.

(A12) The first display unit may be a display device that emits the first image light so that the aerial image can be observed on the second display surface.

In this way, the floating image is observed to be positioned above the image displayed on the second display surface, which enhances the three-dimensional effect of the floating image given to the viewer.

(A13) The housing may be a display device having a first storage space above the first display unit and a processing substrate arranged on the second direction side of the first storage space.

In this way, the processing substrate can be placed in an area of the first storage space that is secured above the first display unit and that has a relatively large vertical dimension.

(A14) The processing board may have an attachment section to which an external device can be attached and detached, and the attachment section may be a display device that is exposed to the outside of the housing from the second direction side of the housing.

In this way, the part of the attachment section to which the external device can be detached that is exposed to the outside of the housing can be made less visible to the observer.

(A15) It is preferable that the display device has a sensor disposed below the observation area, and the processing substrate is a display device that performs processing according to information measured by the sensor.

This can prevent the sensor configuration from interfering with the observation of aerial images.

(A16) The display device may have a sensor board on which the sensor is mounted, the sensor board being provided on the first direction side of the housing.

This can prevent the sensor configuration from interfering with the observation of aerial images.

(A17) The sensor may be a display device including a sensor that detects gestures.

This can prevent parts of the observer's body from interfering with the observation of the aerial image when the observer makes a gesture.

(A18) The housing may be a display device having a second storage space below the image display space, and a control board provided in the second storage space that supplies a signal from the sensor to the processing board.

This prevents the dimensions above the observation area of the housing from becoming larger than when the control board is housed in the first housing space above the first display unit.

(A19) It is preferable that the display device has a wiring section that electrically connects the control board and the processing board, and a partition member that separates the wiring section so that it cannot be observed from the image display space side.

This prevents the wiring that electrically connects the control board and the processing board from being seen.

(A20) The processing substrate may be a display device that controls the display of the first display unit in response to information measured by the sensor.

In this way, the aerial image can be changed based on the information measured by the sensor.

(A21) The display device may have a viewing angle control member that is provided on the display surface of the first display unit and that narrows the viewing angle in the first direction.

This prevents the image displayed on the first display surface from being directly viewed by the viewer.

(A22) The display device may be such that the first image light is image light of an image that is recognized as a two-dimensional image.

In this way, even when using the first display unit that displays an image that is recognized as a two-dimensional image, the observer can be given a three-dimensional sense of the aerial image.

(A23) The housing may have multiple parts connected by fasteners, and the fasteners may be a display device that is not exposed at least from the first direction side.

This prevents the fixing member from being seen by the viewer, improving the aesthetic appearance of the display device.

(A24) It is preferable that the display device has at least a portion of the area facing the image display space treated to reduce light reflection.

This makes it easier to keep the image display space dark, further enhancing the three-dimensional effect of the aerial image that can be conveyed to the viewer.

(A25) The housing may be a display device having a portion that surrounds the observation area when viewed from the first direction side.

In this way, the floating image displayed in the image display space, combined with the housing positioned in front of it, can enhance the three-dimensional effect of the floating image given to the viewer.

(A26) The image display unit may be a display device that displays an image of a character representing a cat.

In this way, it is possible to display a three-dimensional image of a cat character.

(A27) The image display unit may be a display device that changes the image based on a communication function that reproduces communication between the character and an observer observing the character.

In this way, the image of the character showing the cat can be changed based on the communication function.

(A28) It is preferable to further include an odor sensor that detects odors, and an operation control means that executes at least one of a display output operation for displaying on the video display unit an image of a character performing a movement corresponding to the detection result by the odor sensor, and an audio output operation for outputting an audio corresponding to the detection result by the odor sensor as an audio emitted by the character displayed on the video display unit.

In this way, it is possible to reproduce communication regarding smells between the user and the character displayed on the video display unit. For example, when an odor is detected by the odor sensor, the character can be controlled to perform a sound output action in which the character makes a sound such as "I smell something," or to display and output a video of the character sniffing an odor. This allows for more diverse communication between the character and the user.

(A29) The display output operation may be an operation of displaying on the video display unit an image of a character displayed on the video display unit performing an action corresponding to the type of odor detected, and the audio output operation may be an operation of outputting an audio corresponding to the type of odor detected as an audio emitted by the character displayed on the video display unit.

In this way, it is possible to reproduce communication regarding types of odors between the character displayed on the video display unit and the user. For example, when the odor of tobacco is detected, a display output operation may be executed to display an image of the character making a smoky motion, and a voice output operation may be executed to utter words expressing concern for the user's health. In addition, when the odor of a fragrance such as aromatic oil is detected, a voice output operation may be executed to utter a question regarding the fragrance. By performing such control, it is possible to expand the scope of communication between the character displayed on the video display unit and the user, and to increase the user's attachment to the character.

(A30) The device may further include a voice detection means capable of detecting voice, and a detection means capable of detecting the presence of a person in the detection target area, and when the voice detection means detects the presence of a person in the detection target area, the voice detection means may enable voice recognition for interacting with a character displayed on the video display unit.

In this way, when a user interacts with a character displayed on the video display unit, the user does not need to say a trigger word to activate voice recognition, for example, and can communicate more naturally.

(A31) It is preferable to further include a wind detection means for detecting wind, and an operation control means for executing at least one of a display output operation for displaying on the video display unit an image of a character performing a movement corresponding to the detection result by the wind detection means, and an audio output operation for outputting an audio corresponding to the detection result by the wind detection sensor as an audio uttered by the character displayed on the video display unit.

In this way, it is possible to reproduce communication between the character displayed on the video display unit and the user through wind applied to the display device. The wind can be, for example, generated artificially by the user, for example by moving a body part such as waving a hand towards the display device, or by blowing on the device.

(A32) The operation control means may perform the display output operation or the audio output operation in response to the breath blown by the user based on the detection result by the wind detection means.

In this way, communication can be recreated between the character displayed on the video display unit and the user in response to the breath blown onto the display device.

The inventions shown in (1) to (32) above and the inventions shown in (A1) to (A32) above can be combined in any way. For example, it is preferable to add at least a part of the configuration of at least one of the inventions from (2) onwards to all or a part of the configuration of the invention shown in (1). In particular, it is preferable to add at least a part of the configuration of at least one of the inventions from (2) onwards to the invention shown in (1). In addition, any configuration may be extracted from the inventions shown in (1) to (32) and the extracted configurations may be combined. The applicant of this application intends to acquire rights to the inventions including these configurations. Furthermore, even if there is a description such as "in the case of" or "when", it is not intended to describe the configuration as being limited to that case or time. These are examples of better configurations, and the applicant intends to acquire rights to configurations other than these cases or times. Furthermore, the parts described in order are not limited to this order. Configurations in which some parts have been deleted or the order has been changed are also disclosed, and the applicant intends to acquire rights.

The present invention provides technology for outputting information associated with a character.

The effects of the invention of this application are not limited to this, and effects achieved from the configuration parts disclosed in this specification and drawings, etc. are also disclosed, and it is the intention to obtain rights to the configuration that achieves said effects through divisional applications, amendments, etc. For example, in this specification, the phrase "can..." is a description that clearly indicates the effect achieved, and there are also parts that show the effect even without the phrase "can...". Also, there are effects that can be understood from the configuration even without such a description.

FIG. 1 is a diagram illustrating an overview of a display device according to an embodiment. FIG. 1 is a diagram illustrating an overview of a display device according to an embodiment. FIG. 2 is a diagram illustrating a character according to an embodiment. 1 is a diagram showing an external configuration of a display device according to an embodiment; 1 is a diagram showing an external configuration of a display device according to an embodiment; 1 is a diagram showing an external configuration of a display device according to an embodiment; 1 is a front view of a display device according to an embodiment; FIG. 2 is a right side view of the display device according to the embodiment. FIG. 2 is a left side view of the display device according to the embodiment. 1 is a plan view (top view) of a display device according to an embodiment; FIG. 2 is a rear view of the display device according to the embodiment. FIG. 2 is a bottom view of the display device according to the embodiment. FIG. 2 is a rear view of the display device according to the embodiment with a rear cover removed. FIG. 2 is a diagram illustrating an example of how a display device according to an embodiment is used. FIG. 1 is a diagram illustrating a principle regarding display of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. FIG. 1 is a diagram illustrating an internal configuration of a display device according to an embodiment. 1A and 1B are diagrams illustrating a cause for a virtual image being displayed on a display device according to an embodiment. 1A and 1B are diagrams illustrating a cause of occurrence of reflection in a display device according to an embodiment. 1A to 1C are diagrams illustrating the principle of displaying a virtual image and suppressing reflection in a display device according to an embodiment. 1A and 1B are diagrams for explaining a cause of a virtual image being displayed on a display device. 1 is a front view of a display device according to an embodiment; FIG. 2 is a control block diagram of a display device according to an embodiment. 1A is a flowchart for reproducing communication related to odors, and FIG. 1B is a diagram showing a voice management table related to odors. 13 is a flowchart for reproducing communication regarding frequency of stay. FIG. 1A is a flowchart for reproducing communication related to a place of stay, and FIG. 1B is a diagram showing a voice management table related to a place of stay. 1A is a flowchart for reproducing communication related to the number of visits, and FIG. 1B is a diagram showing a voice management table related to the number of visits. 1A is a flowchart for reproducing communication related to stay time, and FIG. 1B is a diagram showing a voice management table related to stay time. 1A is a flowchart for reproducing communication related to a behavior pattern, and FIG. 1B is a diagram showing a voice management table related to a behavior pattern. 13 is a flowchart for reproducing an operation relating to detection of a suspicious person. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 4 is a diagram illustrating an example of an image displayed on a display device according to an embodiment. FIG. 1 is a diagram briefly illustrating an example of a development concept of a system according to an embodiment. FIG. 2 is a diagram showing an example of a display form in a video display space of a system according to an embodiment. 13 is a table showing an example of a formula for calculating the probability that a character will move to areas 1 and 2 depending on the number of times a user has fed the character in a system according to one embodiment. 13 is a table showing an example of a formula for calculating the probability that a character will move to areas 3 to 5 depending on the number of times a user has fed the character in a system according to one embodiment. 13 is a table showing an equation for calculating the probability that a character will move to each area depending on the number of times a user has fed the character in a system according to one embodiment. (A) is a table showing an example of values to be substituted for coefficients X (X1, X2, X3) included in the formula for calculating coefficients a, b, and c in a system according to an embodiment, and used to classify the degree of intimacy into low, medium, and high. (B) is an example of a table in which upper and lower limits are set for the probability of moving to each area at each degree of intimacy in a system according to an embodiment. In a system according to one embodiment, these are tables showing coefficients included in a basic formula showing the probability of moving to Area 1 depending on intimacy, where (A) is an example of a table showing coefficient a, (B) is an example of a table showing coefficient b, and (C) is an example of a table showing coefficient c. In a system according to one embodiment, these tables show coefficients included in a basic formula showing the probability of moving to Area 2 to Area 4 depending on intimacy, where (A) is an example of a table showing coefficient A, and (B) is an example of a table showing coefficient B. 13 is a graph showing the relationship between the number of days a user has fed a character and the probability of movement to each area in a system according to one embodiment. 64 is an example of a table showing the data of the graph shown in FIG. 63 by degree of intimacy in a system according to an embodiment. 11 is an example diagram for explaining a character's state, behavior pattern, etc. when the intimacy level is low in a system according to an embodiment. 13 is an example of an image displayed in a video display space of a system according to an embodiment when the intimacy level is low. FIG. 13 is an example of an explanatory diagram for explaining movement to another area when the intimacy level is low in a system according to an embodiment. 13 is an example image for explaining movement to another area when the intimacy level is low in a system according to an embodiment. 13 is an example of an explanatory diagram for explaining movement to another area when a feeding event is detected when the intimacy level is low in a system according to an embodiment. 13 is an example image showing movement from area 5 to area 1 when a feeding event is detected when intimacy level is low in a system according to one embodiment. 13 is an example image showing movement from area 1 or 2 to area 5 when a feeding event is detected when intimacy is low in a system according to one embodiment. 13 is an example of an explanatory diagram for explaining movement to another area when a scratching event is detected when the intimacy level is low in a system according to an embodiment. 13 is an example image showing movement from area 5 to area 3 when a scratching event is detected when intimacy is low in a system according to one embodiment. 13 is an example image showing movement from area 5 to area 3 when a scratching event is detected when intimacy is low in a system according to one embodiment. 13 is an example of an explanatory diagram illustrating movement to another area when a human detection event is detected when the intimacy level is low in a system according to an embodiment. This is an example image for explaining movement from area 1 or area 3 to area 5 when a human detection event is detected when the intimacy level is low in a system 1 according to one embodiment. 11 is an example of a diagram for explaining a character's communication function when the intimacy level is low in a system according to an embodiment. 11 is an example of a diagram for explaining a character's communication function when the intimacy level is low in a system according to an embodiment. FIG. 13 is a diagram showing an example of a reproduced motion in which a character runs away when an event is detected in a system according to an embodiment. FIG. 13 is a diagram showing an example of a playback motion when a character turns and escapes while moving through an area in a system according to an embodiment. FIG. 13 is a diagram showing an example of a playback motion when a character turns and escapes while waiting in a system according to an embodiment. FIG. 13 is a diagram showing an example of a playback motion when an event is detected while a character is eating food in a system according to an embodiment; FIG. 13 is a diagram showing an example of a motion reproduced when an event is detected while a character is scratching its claws in a system according to an embodiment. FIG. 13 is a diagram showing an example of a motion played back when JUNO runs away after an event is detected while sleeping in a system according to an embodiment. 11 is an example diagram for explaining a character's state, behavior pattern, etc. when the intimacy level is at a medium level in a system according to an embodiment. 13 is an example of an image displayed in the video display space of the system according to an embodiment when the intimacy level is at a medium level. 13 is an example of an explanatory diagram for explaining movement to another area when intimacy is at a medium level in a system according to an embodiment. 13 is an example image for explaining movement to another area when intimacy level is at a medium level in the system 1 according to an embodiment. 13 is an example of an explanatory diagram for explaining movement to another area when a feeding event is detected during intimacy in a system according to an embodiment. 13 is an example image showing movement from area 5 to area 1 when a feeding event is detected during intimacy in a system according to one embodiment. 13 is an example of an explanatory diagram for explaining movement to another area when a scratching event is detected during intimacy in a system according to an embodiment; 13 is an example image showing movement from area 5 to area 3 when a scratching event is detected during intimacy in a system according to one embodiment. 13 is an example of an explanatory diagram illustrating movement to another area when a human detection event is detected when the intimacy level is at a medium level in a system according to an embodiment. 13 is an example diagram for explaining a character's communication function when the intimacy level is at a medium level in a system according to an embodiment. 13 is an example diagram for explaining a character's communication function when the intimacy level is at a medium level in a system according to an embodiment. 13 is an example diagram for explaining other states and actions of a character when the intimacy level is at a medium level in a system according to an embodiment. FIG. 11 is an example diagram for explaining a character's state, behavior pattern, etc. when the intimacy level is high in a system according to an embodiment. 13 is an example of an image displayed in a video display space of a system according to an embodiment when the intimacy level is high. FIG. 13 is an example of an explanatory diagram for moving to another area when the intimacy level is high in a system according to an embodiment. 13 is an example image for explaining movement to another area when the intimacy level is high in a system according to an embodiment. 13 is an example of an explanatory diagram for moving to another area when a feeding event is detected and intimacy is high in a system according to an embodiment. 13 is an example image showing an example of a case where a feeding event is detected when the intimacy level is high in a system according to an embodiment. 13 is an example of an explanatory diagram for explaining movement to another area when a scratching event is detected when the intimacy level is high in a system according to an embodiment. 13 is an example image showing movement from area 1 to area 3 when a scratching event is detected when intimacy is high in a system according to one embodiment. 13 is an example of an explanatory diagram illustrating movement to another area when a human detection event is detected when the intimacy level is high in a system according to an embodiment. 13 is an example image illustrating movement from an area other than area 1 to area 1 when a human detection event is detected when the intimacy level is high in a system according to an embodiment. 13 is an example of a diagram for explaining a call-and-response, which is an example of a character's communication function when the intimacy level is high, in a system according to an embodiment. 13 is an example image for explaining area movement when a call is made when the intimacy level is high in a system according to an embodiment. 13 is an example diagram for explaining a summary of character actions when intimacy is high in a system according to an embodiment; 11 is an example of a diagram for explaining a function specific to a high intimacy level in a system according to an embodiment. FIG. 13 is an example of a diagram for explaining a function specific to a high level of intimacy in a system according to an embodiment, showing an example of a group of call words and character response words for a timer function. FIG. 13 is an example of a diagram for explaining a function specific to a high intimacy level in a system according to an embodiment, showing an example of a group of call words and character response words for an alarm function. FIG. 13 is an example of a diagram for explaining functions specific to a high intimacy level in a system according to an embodiment, showing an example of a group of call words and character response words for home appliance control functions. 1 is a diagram illustrating an example of a timer function and an alarm function in a system according to an embodiment. 11 is a diagram for explaining an example of a specification for a character to learn Japanese in a system according to an embodiment. FIG. 11 is a diagram for explaining an example of a specification for a character to learn Japanese in a system according to an embodiment. FIG. 13 is an example diagram illustrating a character's response when the intimacy level is low in a system according to an embodiment. 11 is an example diagram for explaining a character's response during intimacy in a system according to an embodiment; FIG. 13 is an example diagram for explaining a response of JUNO when the intimacy level is high in a system according to an embodiment. FIG. FIG. 1 is an example of a diagram for explaining recognition words in a system according to an embodiment. FIG. 11 is an example of a diagram for explaining a group of dialogue-related words in a system according to an embodiment. FIG. 96 is a diagram showing an example of word group names and call-out word groups in the dialogue word group table shown in FIG. 95. FIG. 96 is a diagram showing an example of word group names and call-out word groups in the dialogue word group table shown in FIG. 95. FIG. 97 is a diagram showing an example of word group names and character response words in the first, second and third stages in the dialogue word group table shown in FIG. 95. FIG. 97 is a diagram showing an example of word group names and character response words in the first, second and third stages in the dialogue word group table shown in FIG. 95. FIG. 97 is a diagram showing an example of word group names and character response words in the fourth stage in the dialogue word group table shown in FIG. 95. FIG. 97 is a diagram showing an example of word group names and character response words in the fourth stage in the dialogue word group table shown in FIG. 95. FIG. 97 is a diagram showing an example of word group names and character response words in the fourth stage in the dialogue word group table shown in FIG. 95. FIG. 13 is an example of a diagram for explaining a state in which JUNO is trying to learn Japanese in a system according to an embodiment. FIG. 13 is an example of a diagram for explaining a state in which JUNO is trying to learn Japanese in a system according to an embodiment. FIG. 13 is an example of a diagram for explaining a state in which JUNO is trying to learn Japanese in a system according to an embodiment. FIG. 11 is a diagram showing an example of a response flow in a system according to an embodiment. FIG. 11 is a diagram showing an example of a response flow in a system according to an embodiment. 1 is an example of a diagram for explaining a case where a response is made in Japanese in a system according to an embodiment; 11 is an example of a diagram illustrating a special call word in a system according to an embodiment. FIG. 11 is an example of a diagram illustrating a special call word in a system according to an embodiment. FIG. 11 is an example of a diagram illustrating a special call word in a system according to an embodiment. FIG. 11 is an example of a diagram illustrating a special call word in a system according to an embodiment. FIG. FIG. 13 is a diagram for explaining an overview of priority of response words in a system according to an embodiment; FIG. 13 is a diagram for explaining an overview of priority of response words in a system according to an embodiment; FIG. 13 is a diagram for explaining an overview of priority of response words in a system according to an embodiment; FIG. 11 is an example of a diagram for explaining dialogue words in a system according to an embodiment. FIG. 11 is a diagram illustrating an example of priority of response words in a system according to an embodiment. FIG. 11 is a diagram illustrating an example of priority of response words in a system according to an embodiment. FIG. 11 is a diagram illustrating an example of priority of response words in a system according to an embodiment. FIG. 11 is a diagram illustrating an example of priority of response words in a system according to an embodiment. FIG. 11 is a diagram illustrating an example of priority of response words in a system according to an embodiment. 13 is an example of character-specific saved information in a system according to an embodiment. FIG. 13 is an example of a diagram for explaining feeding in a system according to an embodiment. FIG. 13 is an example of a diagram for explaining feeding in a system according to an embodiment. FIG. 13 is an example of a diagram for explaining feeding in a system according to an embodiment. 11 is an example of a diagram illustrating a behavior pattern of a character during a sleep transition in a system according to an embodiment. 11 is an example of a diagram illustrating a behavior pattern of a character during a sleep transition in a system according to an embodiment. 11 is an example of a diagram illustrating a behavior pattern of a character during a sleep transition in a system according to an embodiment. 11 is an example of a diagram illustrating a behavior pattern of a character during a sleep transition in a system according to an embodiment. 11 is an example of a diagram for explaining a Japanese response during sleep in a system according to an embodiment. FIG. 11 is an example of a diagram illustrating a behavior pattern of a character when waking up in a system according to an embodiment. 11 is an example of a diagram illustrating a zoom display of a character in a system according to an embodiment. 11 is an example of a diagram illustrating a zoom display of a character in a system according to an embodiment. 11 is an example of a diagram illustrating a zoom display of a character in a system according to an embodiment. 11A to 11C are diagrams illustrating an example of a screen transition displayed on an Android (registered trademark) smartphone that can be linked with a system according to an embodiment. 11A to 11C are diagrams illustrating an example of a screen transition displayed on an Android (registered trademark) smartphone that can be linked with a system according to an embodiment. 11A to 11C are diagrams illustrating an example of a screen transition displayed on an iOS (registered trademark) version smartphone that can be linked with a system according to an embodiment. 11A to 11C are diagrams illustrating an example of a screen transition displayed on an iOS (registered trademark) version smartphone that can be linked with a system according to an embodiment. 11A to 11C are diagrams illustrating an example of a screen transition displayed on an iOS (registered trademark) version smartphone that can be linked with a system according to an embodiment. 1A and 1B are examples of enlarged views of screens displayed on a smartphone, in which (A) is an enlarged view of a login screen, and (B) is an enlarged view of a menu screen. 10A and 10B are examples of enlarged views of screens displayed on a smartphone, in which (A) is an enlarged view of a setup screen, and (B) is an enlarged view of a connection standby screen. 11 is a flowchart showing an example of an area movement process executed by a processing substrate in a system according to an embodiment. 13 is a flowchart showing an example of an area movement lottery process executed by a processing board in a system according to an embodiment. 13 is a flowchart showing an example of a feeding event process executed by a processing board in a system according to an embodiment. 13 is a flowchart showing an example of a feeding event process executed by a processing board in a system according to an embodiment. 13 is a flowchart illustrating an example of a cat scratching event process executed by a processing board in a system according to an embodiment. 13 is a flowchart illustrating an example of a human detection event process executed by a processing board in a system according to an embodiment. 11 is a flowchart illustrating an example of a system update process executed by a processing board in a system according to an embodiment. 10 is a flowchart illustrating an example of a server process executed in a cloud server according to an embodiment. 11 is a flowchart illustrating an example of an external connection process in the system according to an embodiment. 11 is a flowchart illustrating an example of an alarm process in the system according to an embodiment. 13 is a flowchart illustrating an example of a greeting memorization process in the system according to an embodiment. 11 is a flowchart illustrating an example of a greeting response process in the system according to an embodiment. 11 is a flowchart illustrating an example of a dialogue learning process in the system according to an embodiment. 11 is a flowchart illustrating an example of a dialogue response process in the system according to an embodiment. 1 is an example of an outline for explaining assumed voice input information from a user in a system according to an embodiment. FIG. 11 is a diagram showing an example of names of defined time periods in a system according to an embodiment. 11 is an example of an outline for explaining a response sentence created based on acquired weather information in a system according to an embodiment. 11 is an example of an outline for explaining a response sentence created based on acquired weather information in a system according to an embodiment. 11 is an example of an outline for explaining a response sentence created based on acquired weather information in a system according to an embodiment. 13 is an example of a weather event process executed by a processing board in a system according to an embodiment.

The embodiments will be described in detail below with reference to the drawings. The embodiments described below are examples of embodiments of the present disclosure, and the present disclosure is not limited to these embodiments. In the drawings referred to in the present embodiment, identical parts or parts having similar functions are given the same or similar symbols (symbols consisting of a number followed by A, B, etc.), and repeated explanations may be omitted. In addition, in each figure referred to in the following description, the scale may be different from the actual scale in order to make each component, each area, etc. recognizable.

<1. Overview of Display Device 1>
1A and 1B are diagrams for explaining an overview of a display device according to an embodiment of the present invention. The display device 1 displays an image so that an observer can observe the image through an observation area SC. In this embodiment, observation may be interpreted as meaning to see or visually inspect. The observer is a user who uses the display device 1, for example, a person who observes an image displayed on the display device 1. The observation area SC is an area provided on the front side of the display device 1, and is a rectangular area here. The display device 1 displays an image by aerial imaging. Aerial imaging refers to forming an image in the air, for example, by using light reflection. Aerial imaging is sometimes adopted to give the observer the sensation that the image is floating in a specified space. An image displayed by aerial imaging is called an aerial image.

In the case shown in FIG. 1A, the display device 1 displays the character 1100 as an aerial image. The character 1100 is a female character. The display device 1 further displays a stage image 2100, which means a stage, below the character 1100. The stage image 2100 shows a magic circle here. An observer who sees such a display can get the impression that the character 1100 is standing on a stage. The colors of the character 1100 and the stage image 2100 are not particularly important. For example, the skin of the character 1100 is flesh colored. The hair color and the clothing of the upper body of the character 1100 are a relatively light blue color (e.g., light blue), the accessories placed on the knees are a darker blue color (e.g., blue), and the skirt and shoes are white. The stage image 2100 is a relatively dark blue color (e.g., dark light blue).

In the case shown in FIG. 1B(a), the display device 1 displays the character 1200 as an aerial image. The character 1200 is a cat character. In the case shown in FIG. 1B(b), the display device 1 displays a stage image 2200, which means a stage, below the character 1200. In this case, the stage image 2200 shows the character string "Yupiteru" and the character string "Juno" (pronounced "Yuno"), which is the name of the character 1200, alternately displayed along the perimeter of a circular stage.

The colors of the character 1200 and the stage image 2200 are not particularly important. Here, the character 1200 and the stage image 2200 will be described in more detail with reference to FIG. 1C. The character 1200 is roughly divided into the character body 1210 and the collar 1220. The character body 1210 is the cat character itself, whose name is "Juno" here. The character body 1210 includes body parts shown in white, such as the area between the eyebrows, around the nose and mouth, the abdomen, and the front side of the legs. Among the body parts of the character body 1210, the parts shown in light gray are a relatively light brown color (e.g., ocher), such as the areas above and beside the eyes of the face, the upper part of the torso, and the parts other than the front side of the legs. Among the body parts of the character main body 1210, the parts shown in darker gray are an even darker brown color (e.g., brown), and for example, the face, head, torso, and legs with stripes shown with thin lines correspond to these parts. The collar 1220 means a collar displayed at the neck position of the character main body 1210. The collar 1220 is a reddish color (e.g., red). A tag 1221 is attached to the collar 1220 at a position below the face of the character main body 1210. The tag 1221 is circular and has a predetermined mark written on it. The mark is a mark in which a "●" (a circle with a filled inside) is placed below a "V"-shaped symbol, and each of the marks is written in a reddish color (e.g., red). The mark written on the tag 1221 may function as a trademark indicating the source of the display device 1. The stage image 2200 is generally a brownish color, with the circle and text being a relatively dark brown color (e.g., brown), and the image resembling smoke inside the circle being a lighter brown color (e.g., ochre).

In Figures 1A, 1B, and 1C, the background of the characters 1100, 1200 is shown as black, but this is not limited to being completely black, and may be a relatively dark background (e.g., black or other dark colors).

The characters shown in FIG. 1A, FIG. 1B, and FIG. 1C are examples. When displaying a humanoid character, the display device 1 is not limited to a female character, and may display a male character. The display device 1 may display any of real people (e.g., family members), real people who existed in the past (e.g., historical figures), and fictional characters (e.g., characters appearing in fictional works such as manga and anime). When displaying a non-humanoid animal character, the display device 1 is not limited to a cat character, and may display any of animal characters that can be kept in a general household, such as dogs and hamsters, other animal characters such as horses and cows, and fictional animal characters (e.g., characters appearing in fictional works such as manga and anime). The display device 1 may also display other characters having a collar 1220.

Based on the communication function, the display device 1 moves the body parts of the character or outputs a sound that imitates the character's speech. The communication function is a function that reproduces communication between an observer and a character, or a function that reproduces pseudo-communication. In the communication function, the character moves in response to the behavior of the observer (e.g., speech or body movement). For example, the character 1100 performs actions that reproduce the actions that humans normally perform, such as moving the whole body, such as dancing, or changing facial expressions (e.g., expressing emotions such as joy, anger, sadness, and happiness). For example, the character 1200 performs movements that reproduce the actions that animals normally perform (e.g., yawning, moving the tail, walking, etc.). In addition, the display device 1 outputs, for example, a singing sound as if the character 1100 is singing, or outputs a sound related to a dialogue with the observer, in accordance with the movement of the character 1100. For example, the display device 1 outputs a sound that means a cat's meow, or outputs a sound related to a dialogue with the observer, in accordance with the movement of the character 1200. Such video and audio output is not limited to the communication function, but may be performed using various functions of the display device 1.

Dialogue with the user in the communication function may be realized, for example, by cooperation between a voice recognition application (an application having the function of a voice recognition engine) incorporated in the display device 1, an external voice recognition server (not shown), and an external dialogue server (not shown). For example, when the operation unit 2214 described later is pressed, the voice recognition application is started and voice recognition for dialogue with the character is enabled. In this embodiment, enabling voice recognition means operating the voice recognition function, for example, turning on the voice recognition engine and putting the device into a state of waiting for voice input. Then, the voice uttered by the user (also referred to as user voice) is transmitted from the communication control circuit 706 of the processing board 224 described later to the voice recognition server, and the voice recognition server executes voice recognition processing on the user voice and outputs the voice recognition result. For example, in the dialogue server, for each character string that can be input, the voice uttered by the character (e.g., character 1100) displayed on the display device 1 (e.g., the video display unit described later) is registered in advance in correspondence with each other. The voice recognition result by the voice recognition server (for example, a voice recognition result shown as a character string) is then input to the dialogue server, and a character voice corresponding to the voice recognition result is output from the dialogue server. The character voice output by the dialogue server is then output from the display device 1. In this way, a response process to the voice by the user is executed, and a dialogue between the user and the character is reproduced.

The display device 1 is used by general users, for example, as a display device for entertainment. In this case, the display device 1 is used, for example, in the user's home or other residence. Without being limited to this, the display device 1 may be used for commercial purposes (e.g., advertising) or other purposes. For example, in the case of commercial use, the display device 1 is installed in a public place such as a store or other commercial facility. The display device 1 may display a video in which a character introduces or advertises a product or service at a venue such as an exhibition of the product or service. The display device 1 may also be used for purposes other than these.

2. External Configuration of Display Device 1
Figures 2, 3 and 4 are perspective views showing the external configuration of the display device 1. Figure 2 is a view of the display device 1 as viewed from the diagonal front right. Figure 3 is a view of the display device 1 as viewed from slightly above the diagonal front left side. Figure 4 is a view of the display device 1 as viewed from slightly below the diagonal front right side. Figure 5 is a front view of the display device 1. Figure 6 is a right side view of the display device 1. Figure 7 is a front side view of the display device 1. Figure 8 is a plan view (top view) of the display device 1. Figure 9 is a bottom view of the display device 1. Figure 10 is a rear view of the display device 1.

The display device 1 has a rectangular parallelepiped appearance. The display device 1 has a housing 100. The housing 100 is a box-shaped member that forms the outer shape of the display device 1. The housing 100 is rectangular parallelepiped. The housing 100 is longer in the height direction (also called the up-down direction) than in the width direction (also called the left-right direction). The width and depth directions (also called the front-back direction) of the housing 100 are approximately the same. The ratio of the width, height, and depth directions (also called the front-back direction) of the display device 1 is approximately 2:3:2. In the following description, the side of the depth direction of the display device 1 where the viewer is located is called the "front side" (corresponding to the first direction side), and the opposite side is called the "rear side" (corresponding to the second direction side).

The corners of the housing 100 are rounded. As shown in FIG. 5, when the display device 1 is viewed from the front side, each of the four corners of the housing 100 is rounded. The surface of the housing 100 may be processed to give it a matte finish to give it a textured look. In this way, a rectangular parallelepiped display device 1 with excellent design is provided. The housing 100 may be formed of a resin material, but may also be formed of metal or other materials. The housing 100 has a light-shielded image display space inside. The image display space is a space inside the housing 100 where an aerial image is displayed (formed). The entire image display space or a large part of it is formed of a light-shielding material. Therefore, the material constituting the housing 100 prevents external light from entering the image display space. For example, the entire housing 100 may be formed of a material that blocks external light, or the outer surface or the surface facing the image display space may be covered with a material that blocks light. The color of the outer surface of the housing 100 is not important, but it is preferable to use a color such as white or black that exudes a sense of luxury or simplicity. The display device 1 should be of a size and weight that can be carried by a person such as an observer.

The parts of the housing 100 exposed to the outside are divided into a right panel 101, a front panel 102, a left panel 103, a rear panel 104, an upper cover 105, and a lower cover 106. The right panel 101, the front panel 102, the left panel 103, the rear panel 104, the upper cover 105, and the lower cover 106 are plate-shaped members. The right panel 101 is a panel located on the right side of the housing 100 when viewed from the front side. The left panel 103 is a panel located on the left side of the housing 100 when viewed from the front side. The right panel 101 and the left panel 103 have the same shape and the same dimensions. In addition, the right panel 101 and the left panel 103 are symmetrical in the height direction, and a common mold can be used to form the right panel 101 and the left panel 103. This allows the right panel 101 and the left panel 103 to use the same materials, which is expected to reduce costs and the effort required for assembly.

The front panel 102 is located on the front side of the display device 1. The front panel 102 has an opening 110 into which the dimming member 200 can be attached. The observation area SC is provided on the front panel 102. The observation area SC is an area observed by an observer who views an image displayed on the display device 1. The observation area SC is also an area in which the opening 110 formed in the housing 100 exists. The dimming member 200 is provided in the observation area. The dimming member 200 is provided upright in the vertical direction so as to cover the opening 110. The dimming member 200 and the opening 110 are provided above the center of the height direction of the housing 100 when viewed from the front side of the display device 1. The dimming member 200 and the opening 110 are square or rectangular.

The dimming member 200 includes a rectangular (or square) first region 210 used for observing an image, and a second region 220 surrounding the first region 210. The second region 220 is an edge portion that is not used for observing an image, and may be a region that does not transmit light. Therefore, the observation region SC can also be said to be the region in which the first region 210 exists. In the following description, when the dimming member 200 is mentioned, it may be thought of as the first region 210.

The dimming member 200 is an optical member that reduces the amount of external light that enters the image display space from the observation area SC. The dimming member 200 is provided to maintain the image display space in a dark state. The dimming member 200 is provided to suppress the occurrence of reflections in which the observer himself is reflected in the observation area SC, and to make it difficult for the observer to observe the internal components of the image display space. The dimming member 200 is a plate-shaped (including panel-shaped) member, but may also be a member called a sheet-shaped or film-shaped member. The dimming member 200 may also be a member called a smoke panel or smoke film. The dimming member 200 has, for example, a light transmittance of 24% and a thickness of about 2 mm. The dimming member 200 has a translucency that does not interfere with the observation of the image displayed in the image display space of the display device 1. The dimming member 200 is a semi-transparent member of a dark color such as black, but may also be a blue color or other color.

The information area ST is provided below the observation area SC (the dimming member 200 and the opening 110) in the front panel 102. The information area ST is located in an opening opened in the front panel 102. The information area ST is an area in which a sensor is used to collect predetermined information and to present the predetermined information to the observer. The electronic components provided in the information area ST include a first sensor 2331, a second sensor 2332, a microphone 2333, and a light-emitting unit 2334. The first sensor 2331 is a gesture sensor that detects gestures made by the observer. The second sensor 2332 includes one or more sensors other than the gesture sensor. The second sensor 2332 includes, for example, a gas sensor (odor sensor), a temperature sensor, and an illuminance sensor. The microphone 2333 collects sound and converts the collected sound into an audio signal. The light-emitting unit 2334 presents predetermined information by emitting light. The light-emitting unit 2334 includes, for example, a light-emitting body (e.g., a light-emitting diode) that emits light in a predetermined color, and a light-guiding member that guides the light emitted by the light-emitting body. In this embodiment, the light-guiding member of the light-emitting unit 2334 is arranged so as to surround the area in which the first sensor 2331, the second sensor 2332, and the microphone 2333 are provided. The light-emitting unit 2334 emits light in a color of, for example, a bluish color (e.g., light blue), but may be configured to emit light in one or more colors such as greenish, reddish, or other colors. The light-emitting unit 2334 is not limited to a configuration having a light-guiding member, and may emit light in a predetermined area by, for example, a configuration having a plurality of light-emitting bodies.

An operation area OT is provided on the top cover 105. The operation area OT is an area that accepts operations performed by the observer. The operation area OT is an area in which operation units 2211, 2212, 2213, and 2214 are provided. The operation units 2211, 2212, 2213, and 2214 are each a switch that accepts a press operation. The operation unit 2211 is operated when instructing to turn on or off the power of the display device 1. The operation unit 2212 is operated when instructing to increase the volume. The operation unit 2213 is operated when instructing to decrease the volume. The operation unit 2214 is operated when instructing to execute a specified function. The operation unit 2214 may be, for example, an operation unit called a special key. The operation unit 2214 may be operated, for example, when instructing the display device 1 to start voice recognition. The top cover 105 has openings corresponding to the positions where the operation units 2211, 2212, 2213, and 2214 are provided. The operation units 2211, 2212, 2213, and 2214 are exposed to the outside by being positioned in each opening. Note that the number of operation units, the functions assigned to the operation units, and the acceptable operation methods described here are merely examples.

The human presence sensor 234 is provided on the lower cover 106. The human presence sensor 234 is exposed to the outside by being positioned in the opening of the lower cover 106. The human presence sensor 234 is a detection unit that detects the presence of a person (moving object) in the detection target space. In this embodiment, the human presence sensor 234 is a microwave Doppler sensor that emits microwaves and detects the presence of a person in the detection target space based on the reception intensity of the microwaves, which are the reflected waves. For example, the human presence sensor 234 detects whether a person is present in front of the display device 1.

The rear panel 104 of the display device 1 is provided with a power jack 2311 and a rear cover 1041. The power jack 2311 is a jack to which a power cord is connected. The power jack 2311 receives power from an external power source or the like via the power cord. The rear cover 1041 is a removable cover. When the rear cover 1041 is removed, the slot 222 and the USB (Universal Serial Bus) connector 223 are exposed as shown in FIG. 11. The slot 222 is an attachment portion to which an external storage medium is attached. The external storage medium corresponds to, for example, the microSD (registered trademark) standard, but may also correspond to microSDHC or other SD card standards or other standards. The USB connector 223 is an attachment portion to which a USB cable or a USB port of an external device corresponding to the USB standard is attached. The display device 1 can input and output data to and from external devices via the slot 222 and the USB connector 223. The number of attachment parts may be even greater. The parts of the slot 222 and the USB connector 223 that are exposed to the outside of the housing are on the rear side of the display device 1, so they can be made difficult to be observed by an observer. In addition, there is no particular restriction on the number of attachment parts to which external devices are attached and the corresponding standards.

The housing 100 is formed by joining a right panel 101, a front panel 102, a left panel 103, a rear panel 104, an upper cover 105, and a lower cover 106. In this embodiment, the joining method is screw fastening using screws, which are an example of a fastener. However, there are no screwed parts on the front panel 102 and the upper cover 105.

The right panel 101 is screwed in at two locations, top and bottom, on the front side with screws 710 and 720. The left panel 103 of the housing 100 is screwed in at two locations, top and bottom, on the front side with screws 730 and 740. The rear panel 104 is screwed in at three locations on the upper side with screws 751, 752, and 753, at two locations near the center in the height direction with screws 754 and 755, and at three locations on the lower side with screws 756, 757, and 758. The lower cover 106 is screwed in at two locations on the front side with screws 761 and 762. The front panel 102 and the upper cover 105 are relatively easily observed by the observer using the display device 1. On the other hand, the other parts are relatively difficult to observe by the observer using the display device 1. Therefore, the display device 1 can prevent the screw fastening points from being seen by the viewer, and maintain the aesthetic appearance of the display device 1. This can also contribute to giving the viewer a sense of immersion in the image.

12 is a diagram for explaining an example of how the display device 1 is used by an observer. FIG. 12 shows the positional relationship between the display device 1 and an observer U using it, as viewed from the left side of the display device 1. The display device 1 is placed on a desk 3000. A chair 3100 is placed in front of (on the front side of) the display device 1. The observer U is seated on the chair 3100, facing the direction of the display device 1. In this example, the position of the eyes of the observer U is higher than the top surface of the display device 1. Therefore, the observer U looks down at the observation area SC at an angle θ. The angle θ is the angle between the horizontal direction and the line of sight of the observer U. The angle θ is in the range of angles θ1 to θ2. The angle θ is, for example, 15 degrees. The horizontal distance between the observer U and the observation area SC is approximately 50 cm.

3. Principle of display on the display device 1
Fig. 13 is a diagram for explaining the principle of display of the display device 1. Fig. 13 shows a schematic diagram of the inside of the display device 1 as viewed from the left side of the display device 1. A first display unit 310, an optical member 320, and a second display unit 330 are provided in an image display space S1 inside the housing 100. The first display unit 310 and the optical member 320 cooperate to form an image display unit that forms an aerial image in the image display space S1.

The first display unit 310 and the second display unit 330 are display units that display images. The first display unit 310 and the second display unit 330 are display units that have a light source, for example, a liquid crystal display. The light source is a backlight. The first display unit 310 and the second display unit 330 display color (multi-color) images based on the three primary colors of red (R), green (G), and blue (B). In this embodiment, the first display unit 310 and the second display unit 330 display images that are recognized as two-dimensional images.

The first display unit 310 is an 8-inch liquid crystal display. For example, in this case, the first display unit 310 includes a backlight, which is an example of a light source, and a display body in which liquid crystal elements, which are an example of a display element, are arranged two-dimensionally. The light source is, for example, a white light source. The liquid crystal element modulates light from the backlight on a pixel-by-pixel basis. The liquid crystal element is configured, for example, by sandwiching liquid crystal molecules between a pixel electrode provided on a pixel-by-pixel basis and a common electrode common to a plurality of pixels. The first display unit 310 has a first display surface 311 that emits a first image light L1. The first display unit 310 emits image light for displaying characters 1100 and 1200, as shown in Figures 1A to 1C. The first display unit 310 is arranged so that one end on the observation area SC side is located above the observation area SC and is inclined so that the back side is lower. This inclination is equal to the inclination of the first display surface 311, and is approximately 20 degrees with respect to the horizontal plane. The inventors have found that by tilting the display device 1 at approximately 20 degrees in this way, the three-dimensional effect of the aerial image that can be presented to the viewer can be enhanced, and that the virtual image formed in the image display space by the first image light L1 being reflected by the dimming member 200 can be made less likely to be observed. The basis for this will be described later. However, this angle is only one example. In the width direction of the display device 1, the height of each position on the first display surface 311 is the same.

The optical member 320 is an example of an optical system that forms an aerial image based on the first image light L1 from the first display unit 310. The optical member 320 has light transmittance and reflectance. The optical member 320 is plate-shaped. The optical member 320 is a half mirror here. The optical member 320 is made of an acrylic mitre and has a light transmittance of about 30%. The thickness of the optical member 320 is about 2 mm. The light transmittance is only one example and may be, for example, 6%, and it is sufficient that the transmittance is sufficient to allow the observer to observe the aerial image displayed in the image display space S1. The optical member 320 is arranged in the image display space S1 at an incline with respect to the opening 110 and the first display surface 311. The optical member 320 is inclined so that the tip on the observation area SC side is located below the observation area SC and is higher toward the back side. This inclination is 35 degrees with respect to the horizontal plane. However, this angle is one example. In the width direction of the display device 1, the height of each position of the optical member 320 is the same.

The second display unit 330 is a 7-inch liquid crystal display. The second display unit 330 has a second display surface 331 that emits a second image light L2 of the displayed image. The second display unit 330 displays an image V2 that corresponds to the stage images 2100 and 2200 in terms of FIG. 1A and FIG. 1B. The second display unit 330 is arranged so that the normal direction of the second display surface 331 faces upward. The second display surface 331 is located on the rear side of one end of the lower side of the optical member 320. A pedestal unit 400 is provided on the front side of the second display surface 331. The pedestal unit 400 forms a part of the member that constitutes the image display space S1 of the housing 100. The upper surface of the pedestal unit 400 is located at approximately the same height as the second display surface 331, but the second display surface 331 may be slightly lower.

With the above configuration, the first image light L1 emitted by the first display unit 310 is incident on the first surface 321 of the optical member 320 at a predetermined angle. The incident first image light L1 is reflected by the optical member 320 toward the front side of the display device 1 at the predetermined angle. This reflected light L1r causes an aerial image V1 to be recognized in the image display space S1 of the housing 100. The aerial image V1 is a virtual image recognized below the tip of the rear side of the first display surface 311, for example, below the intersection of the first display surface 311 and the optical member 320.

The second image light L2 emitted by the second display unit 330 is incident on the second surface 332 of the optical member 320 and passes through to the first surface 321. The second image light L2 directly enters the observer's eyes. The image V2 on the second display surface 331 is a real image that is recognized as being located below the aerial image V1. For example, the aerial image V1 is recognized at the center in the front-to-back direction of the second display surface 331. In this way, the aerial image V1 is recognized as the characters 1100 and 1200 in Figures 1A to 1C, and the image V2 is recognized as the stage images 2100 and 2200 in Figures 1A to 1C. According to the display device 1, the observer is given a three-dimensional effect of the image by observing the aerial image V1 combined using the optical member 320, and the three-dimensional effect can be further enhanced by having the observer observe the image V2 below it. Furthermore, by using the second display unit 330, it is possible to increase the freedom of selection of images that the viewer can view along with the aerial image, compared to when a light-emitting body and a light-guiding member are combined to emit light in a predetermined pattern.

It is desirable to provide a viewing angle control member 312 that narrows the viewing angle to the front side on the first display surface 311 of the first display unit 310. In this way, even if the observer approaches the observation area SC and looks up, it is possible to prevent the image displayed on the first display surface 311 from being directly observed by the observer. No viewing angle control member that narrows the viewing angle to the front side is provided on the second display surface 331 so that the observer can easily observe the image V2.

The basic display principle of the display device 1 is as described above. Furthermore, the display device 1 has a light-reducing member 200 and a light-shielding member 500 as components for enhancing the visual effect. The action and effect of these will be described later.

4. Internal configuration of display device 1
Next, the internal configuration of the display device 1 will be described. Figures 14, 15, 16, 17, and 18 are diagrams showing a state in which the right panel 101, the front panel 102, the left panel 103, and the rear panel 104 of the housing 100 are removed. As shown in Figure 14, the right inner panel 107 is exposed by removing the right panel 101. As shown in Figure 15, the front inner panel 108 is exposed by removing the front panel 102. As shown in Figure 16, the left inner panel 109 is exposed by removing the left panel 103. As shown in Figure 16, the wiring section 800 and the partition member 900 are exposed by removing the rear panel 104.

The right inner panel 107 and the left inner panel 109 hold the first display unit 310 and the optical member 320 described in FIG. 13 by sandwiching them from both the left and right sides.

The screws 710 and 720 shown in Figures 2, 4, and 6 pass through holes H1 and H2 that pass through the right panel 101, the front inner panel 108, and the right inner panel 107 in order to connect them. The screws 730 and 740 shown in Figures 3 and 7 pass through holes H3 and H4 that pass through the left panel 103, the front inner panel 108, and the left inner panel 109 in order to connect them. The screws 751, 752, and 753 shown in Figure 10 fasten the rear panel 104 to the top cover portion 105. The screw 754 fastens the rear panel 104 to the right panel 101. The screw 755 fastens the rear panel 104 to the left panel 103. The screws 765, 765, and 757 fasten the rear panel 104 to the bottom cover portion 106.

Figures 19, 20, 21, 22, 23, and 24 show the state after the right inner panel 107 and the left inner panel 109 have been removed. Figure 19 shows the display device 1 as viewed from the right side. Figure 20 shows the display device 1 as viewed from the left side. Figure 21 shows the display device 1 as viewed from diagonally below the right side. Figure 22 shows the display device 1 as viewed from diagonally above the right side. Figure 23 shows the display device 1 as viewed from diagonally below the left side. Figure 24 shows the display device 1 as viewed from diagonally above the left side.

Inside the housing 100, there are provided a first display unit 310, an optical member 320, a second display unit 330, a base unit 400, and a light-shielding member 500. The first display unit 310, the optical member 320, the second display unit 330, and the base unit 400 are configured as described with reference to FIG. 13.

The light shielding member 500 is provided on the optical path of the reflected light reflected from the light-reducing member 200 of the first image light L1 from the first display unit 310. The light shielding member 500 covers one end side of the optical member 320 on the light-reducing member 200 (observation area SC) side, in other words, one end side farther from the first display unit 310, so as to prevent this reflected light from entering the optical member 320. The light shielding member 500 is, for example, a member that absorbs visible light, but it is sufficient as long as it is a member that prevents the transmission of the first image light L1. The light shielding member 500 may be, for example, black or other dark colors, but may be other colors. The reason for this will be described later, but it is to prevent the image (virtual image) formed in the image display space S1 by the first image light L1 being reflected by the light-reducing member 200 from being observed by the observer. In particular, it prevents the observation of images that may be observed when an observer views an image by peering into the image display space S1 from the observation area SC. In addition to these images, it also prevents external objects such as the ceiling from being observed. The top surface of the light blocking member 500 is approximately horizontal, but is not limited to this.

A first plate 601 is provided above the first display unit 310, and a first storage space S2 (see Figures 19 and 20) for storing various components is formed above the first plate 601. A second plate 602 is provided below the second display unit 330, and a second storage space S3 (see Figures 25 to 27) for storing various components is formed below the base unit 400 and the second plate 602.

As shown in Figures 17 to 24, the first storage space S2 is a space separated from the image display space S1 by a first plate 601 provided on the first display unit 310. The first plate 601 is connected to the right inner panel 107 using screws 791, 792, and 793, and is connected to the left inner panel 109 using screws 794, 795, and 796. The first storage space S2 is provided with an operation board 221, a processing board 224, speakers 2251 and 2252, and a holding unit 226. The operation board 221 is a board on which the operation units 2211, 2212, 2213, and 2214 are mounted on the surface. The operation board 221 supplies signals corresponding to the operations of the operation units 2211, 2212, 2213, and 2214 to the processing board 224. The processing board 224 functions as a processing board that controls the display device 1. The processing board 224 may be called a main board. The processing board 224 is electrically connected to the operation board 221 and the speakers 225A and 225B via, for example, a printed wiring board (e.g., FPC). The processing board 224 is located on the rear side of the first housing space S2 in the depth direction of the display device 1. The processing board 224 is a so-called SoC board, and operates a processor and an OS (Operating System). The processing board 224 reads signals from the sensors mounted on the sensor board 233 and audio signals from the microphone 2333 using the OS, and obtains signals corresponding to the operations of the operation units 2211, 2212, and 2213 from the operation board 221. The processing board 224 performs control to display images on the first display unit 310 and the second display unit 330. The processing board 224 is equipped with a slot 222 and a USB connector 223, and is exposed from the rear side of the display device 1.

The speakers 2251, 2252 are provided on both the left and right sides when viewed from the front side, and emit sound to the outside through the openings 1051, 1052 opened in the top cover 105. It is desirable that the speakers 2251, 2252 are not too far away from the processing board 224. It is desirable that the speakers 2251, 2252 are spaced apart so that the sounds from each other do not interfere with each other as much as possible.

The holding part 226 is provided at a position overlapping the processing board 224, and holds the processing board 224 on its lower surface. A heat sink 2261 for heat dissipation or absorption is provided on the upper surface of the holding part 226 opposite to the side holding the processing board 224. The first display part 310 and the first plate 601 are inclined downward from the front side to the rear side. Therefore, the first storage space S2 gradually increases in height from the front side to the rear side, and there is more room in the height dimension toward the rear side. This is utilized to store members having a thickness such as the processing board 224 and the holding part 226. In this way, the housing part above the observation area SC is prevented from becoming longer, contributing to the weight and size reduction of the display device 1 and is also desirable in terms of design.

25, 26, and 27 are diagrams showing the internal configuration of the display device 1 when the lower cover 106 is removed. A second plate 602 is provided below the base 400 and the second display unit 330. A second storage space S3 is formed between the second plate 602 and the lower cover 106. A control board 231, wiring 232, a sensor board 233, and a human sensor 234 are provided in the second storage space S3. The front side inner panel 108 is connected to the second plate 602 using screws 771, 772, 773, and 774. The front side inner panel 108 is connected to the lower cover 106 using screws 775, 776, and 777. The front side inner panel 108 is fixed using screws 771 to 777, but since it is covered by the front side panel 102 under conditions in which the display device 1 is used, the screws 771 to 777 are not exposed to the front side.

The control board 231 controls the supply of power and signals to the display device 1. The control board 231 is electrically connected to the processing board 224 via the wiring section 800, and transmits and receives various signals and supplies power to the processing board 224. The control board 231 is electrically connected to the sensor board 233 and the human presence sensor 234 via wiring 232. The wiring 232 is, for example, a printed wiring board (e.g., FPC).

The sensor board 233 is a board on which the first sensor 2331, the second sensor 2332, and the microphone 2333 are mounted. The sensor board 233 is provided on the rear surface of the front-side inner panel 108, in an area below the dimming member 200 (observation area SC). The sensor board 233 is vertically erected so that its board surface extends in the height and width directions. The sensor of the sensor board 233 may be one that detects predetermined information. The first sensor 2331 is an optical sensor that emits light (e.g., infrared light) to the front side and detects the gesture of the observer by receiving the reflected light. The microphone 2333 collects sound and outputs a sound signal indicating the collected sound. The gas sensor as the second sensor 2332 detects odors such as gas odor and burnt odor. The temperature sensor detects temperature. The illuminance sensor detects illuminance. The first sensor 2331 is located approximately at the center in the width direction of the information area ST. The sensor board 233 outputs signals from the first sensor 2331, the second sensor 2332, and the microphone 2333 to the control board 231 via the wiring 232. The processing board 224 controls the light emitting unit 2334 to emit light when voice recognition is possible, via the wiring unit 800 and the control board 231.

The human presence sensor 234 is positioned at a predetermined distance, such as about 30 cm, behind the front internal panel 108 so as not to come into contact with the front internal panel 108. This is desirable in order to ensure the detection sensitivity of the human presence sensor 234. The human presence sensor 234 is positioned slightly forward from the center in the depth direction. The human presence sensor 234 outputs a signal indicating the detection result to the control board 231.

The wiring section 800 includes a power line 810, a signal line 820, and a signal line 830. The wiring section 800 is routed from the lower side to the upper side of the housing 100. The wiring section 800 passes between the rear panel 104 and the partition member 900. The control board 231 supplies power to the processing board 224 via the power line 810. The control board 231 transmits and receives signals to and from the processing board 224 via the signal line 820. The processing board 224 supplies signals (e.g., driving signals and video signals) that drive the second display unit 330 via the signal line 830. The power line 810, the signal line 820, and the signal line 830 are provided so as to pass through the spatial region between the partition member 900 and the rear panel 104. A background plate 910 is provided on the front side of the partition member 900. The background plate 910 is, for example, a dark color such as black. The partition member 900 is formed in a plate shape using, for example, a resin material, and is a member that does not transmit light. In this way, the wiring part 800 cannot be observed from the image display space S1 side.

The processing board 224 functions as an operation control means for controlling the display of the first display unit 310 and the second display unit 330 and controlling the audio output from the speakers 2251 and 2252 based on the communication function. For example, the processing board 224 controls the display of the first display unit 310 and the second display unit 330 and controls the audio output from the speakers 2251 and 2252 according to information measured by the sensor board 233 and the human presence sensor 234. For example, when the processing board 224 detects the presence of a person based on the human presence sensor 234, it may cause the displayed character to perform an action such as a greeting or output the character's voice. When the first sensor 2331 detects a gesture such as a hand wave, such as moving left and right, up and down, bye-bye, or back and forth, the processing board 224 controls the display of the first display unit 310 and controls the audio output from the speakers 2251 and 2252. For example, the processing board 224 rotates the character in the direction according to the hand wave left and right or up and down. When the processing board 224 detects a bye-bye gesture, it makes the character say bye-bye. When the processing board 224 detects a gesture of moving the hand back and forth, it returns the display of the character to the default (for example, FIG. 1A to FIG. 1C). The processing board 224 changes the display of at least one of the first display unit 310 and the second display unit 330 according to the illuminance measured by the illuminance sensor. For example, the processing board 224 may darken the display when the surroundings of the display device 1 become dark. The processing board 224 may perform control such as moving the eyes of the character based on the illuminance sensor. The processing board 224 controls the display of the first display unit 310 according to the temperature detected by the temperature sensor, controls the output of the sound from the speakers 2251 and 2252, and changes the content of the character's speech according to whether the surroundings of the display device 1 are hot or cold. The processing board 224 also controls the emission of light from the light-emitting unit 2334. The processing board 224 changes the volume of sound in response to the operation of the operation units 2211, 2212, 2213, and 2214. The voice engine may be realized by the functions of the processing board 224, or may be realized by an external recognition server. In the latter case, the processing board may have a wireless LAN such as Wi-Fi or a communication module for network communication, and communicate with the voice recognition server via this.

The processing board 224 makes the emission luminance of the light source of the first display unit 310 brighter than that of the light source of the second display unit 330. This is because the inventors have found that by doing so, the difference between the emission luminance of the first display unit 310 and the emission luminance of the second display unit 330 can further enhance the three-dimensional effect of the aerial image that can be given to the observer. If the image display space S1 is bright, the three-dimensional effect may be lost, so the emission luminance of the second display unit 330 is lowered.

The first display unit 310 is tilted at an angle of approximately 20 degrees from the horizontal. This is to prevent the observer from sitting in a chair and peering into the display, which may cause display problems when combined with the display on the second display unit 330. In this way, the observer can see the entire display 1 even when looking from slightly above it. Therefore, the observer does not need to peer into the display.

In this way, the ceiling is not reflected by the optical member 320 and observed by the observer, and the aerial image is not overlapped with the image of the second display surface 331, making it difficult to see clearly. The operation will be explained. As shown in FIG. 28, it is assumed that the first image light L31, L32 is incident on the dimming member 200. In this case, the reflected waves L31r, L32r of the first image light L31, L32 are emitted from the dimming member 200 and further reflected by the optical member 320, and the observer observes the image V3, which is a virtual image. For example, when the first display unit 310 displays an image of a character, the lower part of the character, for example, an image near the feet, may be observed. Also, as shown in FIG. 29, light L4 from the ceiling or lighting enters the dimming member 200, passes through it, and is further reflected by the optical member 320, causing the ceiling to be reflected near the lower end of the optical member 320. In this way, the display quality of the display device 1 may be affected by observing an image other than the original aerial image V1.

Therefore, by disposing the light shielding member 500 on the optical path of the reflected light from the light reducing member 200 and on the optical path of the light L4 from the ceiling, lighting, etc., the display of the image V3 and the reflection of the ceiling are suppressed. As shown in FIG. 30, the light shielding member 500 suppresses reflection from the optical member 320, so that the display of the image V3 and the reflection of the ceiling are suppressed. The light shielding member 500 is located below the observation area SC, and the light reducing member 200 is also present, so that the presence of the light shielding member 500 is difficult for the observer to recognize. For example, assuming that the horizontal distance between the observer and the observation area SC is 50 cm and the angle θ at which the observer's line of sight is incident is 35 degrees, the light shielding member 500 is provided on the optical path of the light related to the image V3 and the reflection. As shown in FIG. 31, if the inclination angle of the first display unit 310 is 10 degrees, the position of the image V3 will be displayed further back, making it easier to observe the image V3. To prevent this, it would be necessary to extend the light blocking member 500 further to the back, which is undesirable. Therefore, it is desirable to set the inclination angle of the first display section 310 to 20 degrees.

According to this idea, even if the light blocking member 500 is not used, the optical member 320 may be arranged to avoid the area on the optical path of the reflected light of the first image light L1 reflected by the light reducing member 200. In this way, it is possible to prevent the virtual image formed in the image display space by the reflection of the first image light L1 by the light reducing member from being observed by the observer.

The observer makes a gesture near the first sensor 2331, which is a gesture sensor. In this case, if the first sensor 2331 is located above the observation area SC, the hand making the gesture may get in the way, which may impede observation of the image through the observation area SC. In this embodiment, sensors such as the first sensor 2331 are provided below the observation area SC, so that even if a gesture is made while observing the observation area SC, the observation is not impeded.

The video display space S1 is darkened to make the aerial image appear more three-dimensional. For example, when a particular character is displayed, the dark surroundings give the character a sense of floating. Also, when viewed from the front, the observation area SC is surrounded by the housing 100 (front panel 102). In this way, the aerial image displayed in the video display space works in combination with the housing located in front of it to further enhance the three-dimensional feel of the aerial image that can be given to the viewer.

The parts facing the image display space S1 are treated (for example, matte finished) to reduce light reflection. In this way, even if external light or other light enters the image display space S1, reflection is suppressed, making it easier to maintain a dark state. This prevents the internal space from becoming bright. This treatment may be applied to some or all of the frames of the background board 910, the base portion 400, the first display portion 310, and the second display portion 330.

In addition, when a fastening method using fasteners such as screws is adopted, the front panel 102 and the front inner panel 108 can be removed more easily than when adhesives are used, and dust that has entered the image display space can be removed using an air blower or soft cloth. On the other hand, since the front panel 102 is not fixed with fasteners such as screws, it is possible to prevent the observer from noticing this and disappointing the observer.

5. Detailed configuration of processing substrate 224
FIG. 33 is a control block diagram of the processing board 224 of the display device 1 according to an embodiment. As shown in FIG. 33, the processing board 224 is a controller having a CPU 701 that functions as an arithmetic processing device that executes arithmetic processing, and a memory including a RAM 703 used as a work area and a ROM 702 that stores control programs. The processing board 224 may be read as, for example, a control unit (an example of an operation control means). Various programs are stored in the ROM 702 of the processing board 224, and the processing board 224 realizes various functions by executing these programs. For example, a voice recognition application for reproducing a dialogue between a user and a character is stored in the ROM 702, and the processing board 224 reproduces a dialogue between a user and a character by executing the voice recognition application based on input signals from various sensors 234, 2331 to 2332 or a GPS receiver 60. The processing board 224 also has a display control circuit 704 that controls images and the like displayed on the display device 1 (more specifically, the video display unit), and a voice control circuit 705 that controls voices output from the speakers 2551 and 2552. The processing board 224 also has a communication control circuit 706 that controls communication with other devices (for example, an external voice recognition server (also called a voice recognition engine) not shown, and an external dialogue server (also called a dialogue engine) not shown, etc.).

<6. Communication Function for Character 1100>
Various controls for reproducing communication between the user and the character 1100 displayed on the display device 1 (eg, a video display section) will be described below in order with reference to Figs.

<About the smell-related communication function>
For example, an odor sensor that detects an odor may be provided as the second sensor 2332 in the display device 1, and the processing board 224 may execute a voice output operation to output a voice corresponding to the detection result by the odor sensor as the character voice of the character 1100. The odor sensor may output, for example, a signal that changes according to the intensity of a certain odor, or a signal that changes according to the components (odor components) contained in the odor. The odor sensor may output, for example, a signal that changes according to the type of odor. The odor sensor may be configured to detect the concentration of a specific odor component (tar, ammonia, alcohol, carbon dioxide, etc.) using, for example, a deodorizing filter. In this way, the processing board 224 can reproduce communication regarding odors between the character displayed on the video display unit of the display device 1 and the user, and more diverse communication between the character and the user can be reproduced.

The processing board 224 may, for example, execute a voice output operation that outputs a voice corresponding to the type of odor detected as the character voice of the character 1100. Note that a single sensor capable of distinguishing multiple types of odor components may be configured as the odor sensor, or multiple sensors capable of distinguishing different types of odor components may be configured as the odor sensor.

Fig. 34(A) is a flowchart for reproducing communication related to odors, and Fig. 34(B) is a diagram showing a voice management table (also called a voice management table) related to odors. The voice management table is stored, for example, in the processing board 224 of the display device 1 or in the dialogue server. As shown in Fig. 34(B), the voice management table pre-registers, for each type of odor component (also called an odor substance) that can be detected by the odor sensor, the type of odor that mainly contains that odor component and the voice (called a character voice) that the character 1100 emits. For example, in the voice management table, a tobacco odor that mainly contains tar is registered in association with tar, and words of concern for the user's health, such as "Smoking too much is not good," are registered as the character voice when a tobacco odor is detected.

Then, first, in step S11, the processing board 224 (which may also be read as the control unit) determines whether an odor has been detected. For example, when the concentration of an odor component detected by the odor sensor exceeds a predetermined threshold (reference value) for a specific odor, the processing board 224 determines that the odor component has been detected. In step S12, the processing board 224 determines the type of odor registered in association with the detected odor component in the voice management table. For example, when a concentration of tar exceeding a predetermined threshold is detected, it is determined that a tobacco odor has been detected. Then, the processing board 224 outputs a character voice corresponding to the type of odor detected from the speakers 2551 and 2552. For example, when it is determined that a tobacco odor has been detected, the processing board 224 executes a voice output operation that outputs a voice of words that are concerned about the user's health (here, "Smoking too much is not good") as a character voice. Other examples include the following. For example, when the concentration of a specific main component contained in a fragrance such as aromatic oil or food exceeds a predetermined reference value, the processing board 224 determines that the odor of the fragrance or food has been detected, and outputs a voice as the character voice asking a question about the source of the odor. When alcohol exceeding a predetermined reference value is detected, the processing board 224 determines that an alcohol odor has been detected, and outputs a voice such as "You smell like alcohol" or "You've been drinking again, stop it" as the character voice of the character 1100. When a breath odor component (e.g., hydrogen sulfide, methyl mercaptan, dimer sulfide, etc.) exceeding a predetermined reference value is detected, the processing board 224 determines that bad breath has been detected, and outputs a voice such as "Your breath stinks a little, are you brushing your teeth?" or "Please take care of your periodontitis" as the character voice of the character 1100. Furthermore, when odor components contained in gas fuel, such as tertiary butyl mercaptan, dimethyl sulfide, or tetrahydrothiophene, are detected, the processing board 224 determines that the odor of gas fuel has been detected and outputs a voice such as "It smells like gas!" as the character voice of the character 1100.

In this way, the processing board 224 may execute control to output a sound corresponding to the type of odor detected as the character voice of the character 1100. By executing such control, it is possible to widen the scope of communication between the character displayed on the video display unit and the user, and to increase the user's attachment to the character.

The display device 1 is not limited to specifying the voice to be output based on a voice management table. For example, the display device 1 may specify the voice to be output according to a scenario in which the content of a character's speech changes based on the content of past dialogue (dialogue history). This scenario may prescribe a branching scenario in which data specifying the voice to be output by a character is managed in a tree structure.

Here, an example of an audio output operation of a sound according to the detection result by the odor sensor is shown, but the present invention is not limited to this, and the processing board 224 may execute a display output operation of displaying an image of the character making a movement according to the detection result by the odor sensor on the image display unit. For example, a display management table (not shown) may be provided in advance, similar to the audio management table, and the processing board 224 may execute a display output operation of displaying an image of the character 1100 making a movement according to the type of odor detected on the image display unit. The display management table may be, for example, a table in which data that defines the character 1100 is registered in association with the odor component and the type of odor. The data that defines the character 1100 may be data showing an image of the character 1100 to be displayed, or may be data that defines the content of the movement of the character 1100. For example, when some odor component is detected, the processing board 224 may display an image of the character 1100 making a movement to smell the odor on the image display unit. Furthermore, when the processing board 224 determines that, for example, a cigarette odor has been detected, it may display on the video display unit an image of the character 1100 making a smoky movement. At this time, the processing board 224 may display an image imitating smoke on the video display unit. Furthermore, the processing board 224 may execute both the audio output operation and the display output operation as described above. For example, when some odor component is detected by the odor sensor, the processing board 224 may display on the video display unit an image of the character 1100 making a sniffing movement, while outputting an audio such as "I smell something" from the speakers 2551, 2552 as the character voice of the character 1100.

Furthermore, the processing board 224 may control the character 1100 to change its response depending on the concentration of the odor components detected by the odor sensor (also called detection intensity). For example, the processing board 224 may selectively display one of a number of stepwise movement images (e.g., "grimacing (making a disgusted expression)," "breaking into a cold sweat," "turning pale," "fainting," etc.) depending on the concentration of the odor components.

The display device 1 may be capable of detecting that the user is smoking by combining, for example, a smoke sensor that detects smoke, a camera image (e.g., an image captured by the image capture unit 227), an image from a thermal camera, or another sensor as a method of distinguishing between tobacco odor and odors (odors) other than tobacco odor.

When the processing board 224 detects that the user is smoking, it may display an image simulating smoke (a picture recreating the appearance of smoke billowing out) superimposed on the character 1100 (in front of the character 1100). At this time, the processing board 224 may push the face of the character 1100 forward, close up the face, and make the character 1100 look displeased. The processing board 224 may display the degree of dislike of the character 1100, which varies according to the intimacy, which is the degree of intimacy between the user and the character 1100. For example, when the intimacy is equal to or greater than a predetermined value, the processing board 224 makes the character 1100 look as if he or she is forgiving the user because he or she loves the user, and when the intimacy is less than the predetermined value, the processing board 224 makes the character 1100 look unpleasant. For example, the processing board 224 may determine the degree of intimacy according to the user's usage history of the display device 1 (for example, the past usage time or the content of past conversations). The algorithm for determining the degree of intimacy is not limited to this.

<About communication function based on user location information>
Also, for example, the display device 1 may have a GPS (Global Positioning System) receiver 60 (see FIG. 33 ), and the following control may be executed based on the user's position information. Then, for example, the processing board 224 may acquire position information of the display device 1 held (carried) by the user as the user's own position information based on a signal from the GPS receiver 60, and execute the following control based on the acquired position information.

For example, the system may be controlled so that a voice corresponding to the frequency of the user's stay (also called the frequency of visit) at a location identified based on the user's location information is output as the character's voice. FIG. 35 is a flowchart for reproducing communication related to frequency of stay.

For example, in step S21, the processing board 224 determines whether the user's location information has been received from the GPS receiver 60. When the user's location information is received, in step S22, the processing board 224 determines whether the number of stays (also called the number of visits, etc.) that the user has stayed in a location specified based on the user's location information is a predetermined value (e.g., "2") or more. The frequency of stay is the number of times that the user has stayed in a location during a predetermined period (e.g., one day or one week, etc.). For example, the processing board 224 searches for and identifies the user's stay location using a network, etc., based on the location information (e.g., the user's current location information) obtained from the GPS receiver 60. Then, the processing board 224 calculates the user's frequency of stay based on the number of stays that the user has stayed in the specified location, and determines whether the user's frequency of stay is a predetermined value or more. For example, if the user's visit frequency is equal to or greater than a predetermined value, in step S23, the processing board 224 controls so that a predetermined character voice preregistered in relation to the user's visit frequency is output from the speakers 2551 and 2552. For example, if the user goes to the same place twice on the same day, the processing board 224 determines that the user's visit frequency is equal to or greater than a predetermined value, and outputs a character voice such as "Oh, I was there earlier too."

The processing board 224 may also be controlled to output a voice corresponding to the location identified based on the acquired position information as the character voice of the character 1100. Figure 36 (A) is a flowchart for reproducing communication related to the place of stay, and Figure 36 (B) is a diagram showing a voice management table of the character voice related to the place of stay.

For example, in step S31, the processing board 224 determines whether or not the user's location information has been received from the GPS receiver 60. When the user's location information is received, in step S32, the processing board 224 searches for and identifies the user's location using a network or the like based on the user's location information. Then, in step S33, the processing board 224 outputs a character voice corresponding to the identified location from the speakers 2551 and 2552 based on the voice management table of FIG. 36 (B). For example, when the user goes to a supermarket, the processing board 224 outputs a character voice such as "What should we make for dinner tonight?", or when the user goes to a hospital, the processing board 224 outputs a character voice such as "Are you OK? Are you feeling unwell?" as the character voice of the character 1100. In addition, when the user goes to an amusement park, the processing board 224 outputs a character voice such as "You had a lot of fun."

The processing board 224 may also be controlled to output, as the character voice of the character 1100, a voice corresponding to the number of stays (also called the number of visits) that the user has stayed in a location identified based on the acquired location information. Figure 37 (A) is a flowchart for reproducing communication related to the number of stays, and Figure 37 (B) is a diagram showing a voice management table of character voices related to the number of stays.

Steps S41 and S42 in FIG. 37(A) are similar to steps S31 and S32 in FIG. 36(A), and therefore will not be described. In step S43, the processing board 224 acquires the number of times the user has stayed at the location identified in step S42. Then, in step S44, the processing board 224 outputs character voices according to the number of times the user has stayed from the speakers 2551 and 2552 based on the voice management table in FIG. 37(B). For example, when the user goes to a certain game center for the first time, the processing board 224 outputs character voices such as "The game was fun." Then, when the user goes to the certain game center again, the processing board 224 outputs character voices such as "I wanted to go again." Furthermore, when the user goes to the certain game center a predetermined number of times or more (for example, 10 times or more), the processing board 224 outputs character voices such as "Let's play again."

The processing board 224 may also be controlled to output, as the character voice of the character 1100, a voice corresponding to the user's stay time at a location identified based on the acquired position information. Fig. 38 (A) is a flowchart for reproducing communication related to stay time, and Fig. 38 (B) is a diagram showing a voice management table of character voices related to stay time.

Steps S51 and S52 in FIG. 38(A) are similar to steps S31 and S32 in FIG. 36(A), and therefore will not be described. In step S53, the processing board 224 acquires the user's stay time at the location specified in step S52. Then, in step S54, the processing board 224 outputs character voices according to the user's stay time from the speakers 2551 and 2552 based on the voice management table in FIG. 38(B). For example, when the user's stay time at the convenience store is within 5 minutes, the processing board 224 outputs character voices such as "You were quick. I'm glad I was waiting." Also, when the user's stay time at the convenience store is 5 minutes or more and less than 10 minutes (5 to 10 minutes), the processing board 224 outputs character voices such as "What did you buy?" Furthermore, when the user's stay time at the convenience store exceeds 10 minutes, the processing board 224 outputs character voices such as "You were late. I was worried."

The processing board 224 may perform the above control based on the user's location information acquired using the GPS receiver 60 or the like. In this way, the user can get the feeling that the character is responding to the user's actions, which increases the user's attachment to the character.

The display device 1 may be installed, for example, in the user's home, and the display device 1 (specifically, the communication control circuit 706) may communicate with the user's smartphone (a smartphone equipped with a GPS receiver) or the like to obtain location information of the user while out (location information of the smartphone carried by the user). Then, when the user communicates with the character 1100 after returning home, etc., the processing board 224 may be controlled to execute the above-mentioned operations. The content of the character's voice and movements at this time may be changed as appropriate.

<Communication features related to user behavior patterns>
Furthermore, when it is determined that the user's behavior pattern matches a predetermined behavior pattern, control may be performed so that a voice corresponding to the predetermined behavior pattern is output as the character voice of the character 1100. Fig. 39(A) is a flowchart for reproducing communication related to behavior patterns, and Fig. 39(B) is a diagram showing a voice management table of character voices related to behavior patterns.

For example, in the processing board 224, the day of the week (e.g., Monday to Friday), the time of commuting (e.g., 8:30), and the specific route used when commuting are registered in advance by the user as a "commute behavior pattern". Then, for example, when the user's location information is received from the GPS receiver 60 (step S61), the processing board 224 determines whether the user's behavior pattern matches a predetermined behavior pattern based on the user's location information (step S62). For example, if the user passes through a specific route at 8:30 on Tuesday, the processing board 224 determines that the user's behavior pattern matches the "commute behavior pattern". Then, in step S63, the processing board 224 outputs the voice (e.g., "Work hard") registered in association with the "commute behavior pattern" from the speakers 2551 and 2552 as the character voice of the character 1100 based on the voice management table of FIG. 39 (B). In this way, the character's reaction to the user's actions can increase the user's attachment to the character. Note that it is not essential that the user pre-registers a specific behavior pattern (e.g., a behavior pattern when going to work); for example, the processing board 224 may have a learning function and register a specific behavior pattern by learning the user's behavior patterns.

<Communication function for detecting suspicious individuals>
Furthermore, when a person detected in the detection target space is determined to be a suspicious person, a predetermined operation may be executed. Then, when a user of the display device 1 is detected after the predetermined operation is executed, a sound related to the detection of the suspicious person may be output as a character sound of the character 1100. Fig. 40 is a flowchart for realizing the operation related to the detection of the suspicious person.

For example, in the display device 1, as shown in FIG. 32, a photographing unit 227 (e.g., a camera) that photographs an observer (user, etc.) is provided on the front side of the display device 1, and the photographing unit 227 (more specifically, a sensor that the photographing unit 227 has) may be used as a detection means for detecting the presence of a person in the detection target space. The "detection target space" may be, for example, the field of view range (photographable range) of the photographing unit 227 in the space in which the display device 1 is placed (e.g., the user's home). The photographing unit 227 includes, for example, a lens and an image sensor (e.g., a CCD or CMOS) and photographs a multi-color image. The lens of the photographing unit 227 is provided at a position where it can photograph the observer (particularly the face). In the example shown in FIG. 32, the lens of the photographing unit 227 is provided above the observation area SC of the front panel 102 arranged on the front side of the housing 100, near the center in the width direction of the display device 1. The image capturing unit 227 outputs image signals for each color component, for example, R, G, and B, to the processing board 224. The processing board 224 processes the image signals to generate a captured image.

The processing board 224 monitors the detection target space using the photographing unit 227 (step S71). Then, when the processing board 224 detects the presence of a person in the detection target space, in step S72, it performs user authentication such as face recognition, and in step S73, it judges whether the detected person is a registered user of the display device 1. For example, when the processing board 224 judges that the person photographed by the photographing unit 227 is a registered user of the display device 1, it judges that the person is not a suspicious person. On the other hand, when it is judged that the person photographed by the photographing unit 227 is an unregistered user, in step S74, the processing board 224 judges that the person is a suspicious person and executes a predetermined operation. Examples of the "predetermined operation" include an operation of starting recording by the photographing unit 227, an operation of sounding an alarm at a relatively loud volume, an operation of reporting to a security company registered in advance, and the like. Alternatively, the "predetermined operation" may be an operation in which the character 1100 calls out to the suspicious person (for example, "Who!? Tell me the password!"). Then, if there is no reply from the person determined to be suspicious, or if the password is incorrect, the above-mentioned recording start operation, etc. may be executed. If the password is incorrect, the processing board 224 may output a voice such as "That's not right! Say it again" as the character voice of the character 1100, and when it is determined again that the password is incorrect, the above-mentioned recording start operation, etc. may be executed. By doing this, for example, it is possible to cause the suspicious person to retreat from the detection target space, thereby improving crime prevention in the detection target space.

Then, when the user of the display device 1 is detected after the execution of a predetermined operation, the processing board 224 may output a voice regarding the detection of a suspicious person as the character voice of the character 1100. An example of a "voice regarding the detection of a suspicious person" is "That was scary." In this way, it is possible to increase the user's attachment to the character displayed on the video display unit.

In addition, when the display device 1 is used to monitor a space to be detected, the following operations may be performed in addition to detecting suspicious individuals.

For example, the display device 1 may be placed in a company's reception space, and the character 1100 may be displayed as performing reception duties as a so-called receptionist. Specifically, when a microwave Doppler sensor (an example of a detection means) detects the approach of a person in the detection target space (e.g., the reception space), the processing board 224 may output a voice such as "Welcome. Which department would you like to serve?" as the character voice of the character 1100. Then, based on the response from the detected person (visitor), the processing board 224 may perform control to make a call to the department desired by the visitor.

<Communication function regarding the startup time of the display device 1>
Furthermore, the processing board 224 may control the language used in the character voice of the character 1100 so that the content of communication changes according to the startup time (e.g., cumulative startup time) of the display device 1 from a predetermined point in time (e.g., the time of the first startup). For example, as described below, the processing board 224 may control the character 1100 to speak to the user in more familiar language as the startup time of the display device 1 becomes longer.

For example, for the various character voices as described above, three patterns of character voices, a polite tone, a tone of speech that sounds like a friend to the user, and a tone of speech that sounds like a lover to the user, may be registered in advance in the voice management table. Then, for example, when the startup time of the display device 1 from a predetermined point in time is less than 100 hours, the processing board 224 outputs a character voice with a polite tone from among the three patterns of character voices registered in the voice management table. Also, when the startup time is more than 100 hours and less than 1000 hours, the processing board 224 outputs a character voice with a tone of speech that sounds like a friend to the user from among the three patterns of character voices registered in the voice management table. Furthermore, when the startup time exceeds 1000 hours, the processing board 224 outputs a character voice with a tone of speech that sounds like a lover to the user from among the three patterns of character voices registered in the voice management table.

In this way, the user can sense a change in the character's relationship with the user (e.g., intimacy level) as a result of the change in the character's way of speaking to the user.

The processing board 224 may control the character 1100 to decrease its intimacy with the user depending on the time that has elapsed since the previous startup. For example, even if the cumulative startup time of the display device 1 exceeds 100 hours, if one week has passed since the previous startup, the processing board 224 may output the character's voice in a polite tone rather than an equal tone with the user. In this way, the user will interact with the character displayed on the video display unit as quickly as possible in order to maintain the intimacy with the character, further increasing the user's attachment to the character.

In addition, the processing board 224 may output a greeting such as "Nice to meet you" as the character's voice when the character is first started, and when a predetermined period of time (e.g., a convenient period such as one month) has passed since the first start, the processing board 224 may output a voice indicating an anniversary (e.g., "It's been one month since we met") as the character's voice. In this way, the attachment to the character 1100 can be further increased.

<About communication functions that link with other devices>
Also, for example, when the display device 1 is installed in a user's home or the like, the following operation may be executed by cooperation with other devices. For example, when a user visits a place where a device capable of communicating with the display device 1 (hereinafter referred to as the other device) is installed, the other device may identify the user as described below and automatically guide the user to information suitable for the user (for example, gourmet information or event information). For example, a face image of a user who uses a cooperation system between a display device and other devices and a display device owned by the user (for example, an IP address, etc.) are associated and registered in an external server or the like in advance. When a person is detected by a sensor of a photographing unit provided in the other device, the control unit of the other device compares the face image of the person detected in the photographed image with the face image of the user registered in the external server to identify the user of the display device 1. Then, the control unit of the other device identifies the display device 1 (for example, the IP address of the display device 1, etc.) registered in association with the user in the external server, and transmits information indicating that the user of the display device 1 has been detected to the display device 1. The processing board 224 of the display device 1 has "information suitable for the user" (e.g., gourmet information and event information) registered in advance, and the processing board 224 transmits the registered "information suitable for the user" to the other device in response to receiving a signal from the other device. Then, the control unit of the other device outputs (e.g., outputs audio) the "information suitable for the user" received from the display device 1. When providing the "information suitable for the user", the other device may display the character 1100 on its display screen. Also, the information provided to the user may be transmitted from the other device to the display device 1 and may be stored as user information in the processing board 224 of the display device 1. Then, for example, when the display device 1 is installed at the user's home, when the user's return is detected, the processing board 224 may output a voice such as "You went to **, how was it?" as the character voice of the character 1100. In this way, the user's attachment to the character displayed on the video display unit can be increased.

<Other communication features>
Furthermore, the following control may be performed based on the detection results from various sensors.

For example, a wind detection means (e.g., a wind sensor) capable of detecting wind (e.g., wind speed) may be provided on the front panel 102 of the display device 1, and the following control may be performed based on the detection result by the wind sensor. The wind may be, for example, artificially generated by the user, for example, by moving a body part such as waving a hand toward the display device 1, or by blowing a breath (exhalation). For example, when the wind sensor detects wind with a wind speed equal to or greater than a predetermined value, the processing board 224 may display and output an image of the fluttering hair (e.g., hair) or clothes (e.g., skirt) of a character (e.g., character 1100) displayed on the video display unit. Also, for example, when an image of a lit candle is displayed on the video display unit, if the wind sensor detects wind, the processing board 224 may switch to an image of the candle going out. Furthermore, when the detected wind is stronger than a predetermined threshold (wind speed is greater than a predetermined threshold), the processing board 224 may switch the image according to the strength of the detected wind (e.g., wind speed), for example, by making the clothing of the character 1100 flutter more.

Also, for example, an object detection means (e.g., an ultrasonic sensor) capable of detecting the presence or absence (existence) of an object and the distance to the object may be installed in the display device 1, and the following control may be performed based on the detection result by the ultrasonic sensor. For example, when the processing board 224 determines that the observer's hand is touching the character 1100 recognized as a three-dimensional image based on the detection result by the ultrasonic sensor, the processing board 224 may display and output an image showing the reaction of the character 1100. Also, the processing board 224 may control the display so that the character 1100 shows a different reaction depending on the part of the character 1100 touched by the observer. The part of the character 1100 touched by the observer may be a body part exemplified by the head, chest, abdomen, buttocks, or legs. Also, the part of the character 1100 touched by the observer may be the clothing (e.g., Western clothes, skirt) or decoration (e.g., accessories) of the character 1100.

In addition, for example, the display device 1 may be provided with an earthquake detection means (e.g., a seismic sensor) capable of sensing an earthquake and measuring the seismic intensity, and the processing board 224 may issue an earthquake warning when an earthquake is detected by the seismic sensor.

For example, a weather sensor capable of measuring the weather may be provided in the display device 1, and the processing board 224 may notify the user of changes in the weather, weather forecasts, etc. based on the measurement results of the weather sensor. The weather sensor may have, for example, an air pressure sensor, a temperature sensor, and a humidity sensor, and may be configured to be able to measure the weather based on the detection results of each sensor.

In addition, the processing board 224 may display an image in which the facial expression and movement (also called motion) of a character displayed on the image display unit changes according to the temperature detected by the temperature sensor and/or the humidity detected by the humidity sensor.

In addition, the processing board 224 may display an image in which the facial expression and movement of a character displayed on the image display unit changes according to the detection results (e.g., the distance between the display device 1 and the observer) of a human presence sensor such as a microwave Doppler sensor.

The processing board 224 may also use a humidity sensor to determine whether or not the user has blown on the display. Furthermore, the processing board 224 may also use a temperature sensor to determine the type of breath blown on the display. These sensors may be provided in the display device 1 as, for example, the second sensor 2332.

Here, there are two types of breath blown by the user: a breath blown with a "huff" sound, and a breath blown with a "ha" sound. Generally, when blowing cold breath, one blows quickly with a "huff," so here, a breath blown with a "huff" sound is referred to as a "cold breath." Generally, when blowing warm breath, one blows slowly with a "ha" sound, so here, a breath blown with a "ha" sound is referred to as a "warm breath."

Regarding these two types of breath, the inventor of the present application has obtained the following findings through experiments, etc. In detail, first, regarding humidity, when a user blows "blow" toward a temperature and humidity sensor (for example, a sensor in which a temperature sensor and a humidity sensor are integrated) at a position a certain distance from the temperature and humidity sensor, the humidity rises sharply from the steady state immediately after the blowing. In addition, the humidity increases monotonically while the user is blowing, and when the user stops blowing, it takes several tens of seconds for the humidity to return to the original steady state. The same measurement result was obtained when the user blew "blow" toward the temperature and humidity sensor under the same conditions. On the other hand, regarding temperature, the measurement result was obtained that when the user blew "blow", the temperature slightly drops from the steady state or remains almost flat, and when the user blew "blow", the temperature rises. The breath blown by the user toward the display device 1 is an example of wind that hits the display device 1.

Taking such knowledge into consideration, for example, in a display device 1 provided with a humidity sensor, when the humidity sensor detects a sudden increase in humidity from a steady state, the processing board 224 may determine (also called a decision) that the observer has blown on the display. Furthermore, when a monotonically increasing humidity is subsequently no longer detected, the processing board 224 may determine that the blowing has stopped. Alternatively, when a predetermined time (for example, the time it normally takes to exhale (for example, about 3 seconds)) has elapsed since the sudden increase in humidity was detected, the processing board 224 may determine that the blowing has stopped, regardless of the humidity detection result at that time.

Furthermore, when a sudden increase in humidity is detected by the humidity sensor, and the temperature sensor detects (for example, at approximately the same time) that the temperature has increased from the steady state, the processing board 224 may determine that the type of breath blown by the observer is "warm breath." Conversely, when a sudden increase in humidity is detected by the humidity sensor, and the temperature sensor detects that the temperature has not changed from the steady state or has decreased from the steady state, the processing board 224 may determine that the type of breath blown by the observer is "cold breath."

The processing board 224 may then output video and audio in response to the detection of the blowing of breath as the movement and voice of the character displayed on the video display unit. The processing board 224 may also output video and audio in response to the type of breath blown as the movement and voice of the character displayed on the video display unit. For example, if the type of breath blown by the observer is "cold breath", the processing board 224 may display and output an image of the character's clothing fluttering, and if the type of breath blown by the observer is "warm breath", the processing board 224 may display and output an image of the character making a movement to avoid the breath.

The inventors of the present application also obtained the following findings using various gas sensors. In detail, the measurement results showed that when a person blows a breath toward a gas sensor at a position a certain distance away (e.g., about 10 cm) from the gas sensor, the output value of the gas sensor drops immediately thereafter, and when the person stops blowing, the output value of the gas sensor starts to rise immediately thereafter. Conversely, the measurement results showed that when a person blows a breath toward the gas sensor, the output value of the gas sensor rises immediately thereafter, and when the person stops blowing, the output value of the gas sensor starts to fall immediately thereafter. The measurement results showed that when the wind from a blower (e.g., a fan) hits the gas sensor, the output value of the gas sensor drops, and when the wind from the fan no longer hits the gas sensor, the output value of the gas sensor rises.

Taking into consideration such findings, for example, in a display device 1 provided with a gas sensor, when the output value of the gas sensor drops sharply from a steady state, it is preferable to determine that the type of breath blown by the observer is "cold breath." Conversely, when the output value of the gas sensor rises sharply from a steady state, it is preferable to determine that the type of breath blown by the observer is "warm breath."

A wind sensor can also be used to detect breath. The inventors of this application have confirmed that the wind sensor can sometimes detect a relatively weak wind, such as that caused by holding a piece of paper up in the air, and that it quickly peaks when you blow on the wind sensor, providing good responsiveness when the wind begins to blow. However, if you continue to blow, the wind will quickly drop from its peak. It is thought that a strong wind, such as a quick whoosh, will cool the heater, and it will take time for it to recover.

In relation to a configuration that uses a wind sensor to detect breath, the following configuration may be used.

When the wind sensor continuously detects a predetermined amount of wind, the processing board 224 may process the direction in which breathing is not being done. Processing the direction in which breathing is not being done may be, for example, processing to determine that breathing is not being done. In this case, the processing board 224 may process the direction in which breathing is being done when the time from when wind detection by the wind sensor begins to when it ends is within the time in which breathing is possible.

In this case, the wind sensor may be a sensor that detects wind by electrically measuring the heat removed by receiving the wind. In this sensor, the wind received by the sensor corresponds to the volume of air blown on the sensor. This sensor is set so that when the wind received by the sensor continues for a period of time equivalent to the time the breath was blown on the sensor, the volume of air drops to a value less than the actual volume of air during that period of time, and is preferably set so that this drop is unlikely to occur when the wind received by the sensor is smaller than the volume of air blown on the sensor.

In the display device 1, a flow path for introducing wind into the multiple wind sensors, which share a common breath inflow path, may be provided as the wind sensor. The flow path may be configured so that wind enters the different wind sensors at a predetermined ratio that is not 1:1. The processing board 224 may determine the magnitude and duration of the breath based on the ratio and the differences in the characteristics of the multiple wind sensors. The display device 1 may include multiple wind sensors that have different timings at which the volume of air drops to a value lower than the actual volume of air during the duration, and the processing board 224 may determine the magnitude and duration of the breath based on the output of the multiple wind sensors. The display device 1 may include at least a wind sensor that is not a gas sensor and a gas sensor as the multiple wind sensors.

It is also preferable to use a gas sensor for detecting a specific gas as the wind sensor. It is also preferable to use a gas sensor for detecting a specific gas that is equipped with a heater as the wind sensor. In this case, it is preferable to use a gas sensor equipped with a heater that has the characteristic of outputting a larger amount of detected gas when the degree of blowing is relatively small (as a result of detecting the gas contained in the breath), and outputting a smaller amount of detected gas when the degree of blowing is relatively large (due to the cooling of the heater by the wind from the breath).

When the processing board 224 determines that breathing is being done, if the strength of the breath is relatively strong (for example, if the air volume is above a threshold), an animation that moves the clothes, hair, etc. of the character 1100 is displayed, whereas if the strength of the breath is relatively weak, a different animation from the animation that moves the clothes, hair, etc. of the character 1100 is displayed.

When a gas sensor is used to detect breath, the gas sensor responds to components contained in human breath (e.g., carbon dioxide) and outputs the above-mentioned output value. The reason why the output value of the gas sensor differs between when a person blows a "blow" and when a person blows a "ha" is thought to be that when a person blows a "blow," the speed of the breath is high and it tends to diffuse into the surrounding area immediately after hitting the gas sensor, whereas when a person blows a "ha" the speed of the breath is low and the components of the breath tend to remain in place even after hitting the gas sensor.

Furthermore, the processing board 224 may use, for example, the humidity detection result by the humidity sensor to determine whether the detected wind is due to the observer's breathing or the wind from a fan. For example, in the display device 1, when the output value of the gas sensor drops sharply from the steady state and the humidity rises sharply from the steady state, the processing board 224 may determine that the user is breathing. On the other hand, when the output value of the gas sensor drops sharply from the steady state but the humidity remains at the steady state, the processing board 224 may determine that the wind is being blown by the fan. Furthermore, the processing board 224 may use the temperature detection result by the temperature sensor to determine the type of breath when it is determined that the user is breathing. Also, when the output value of the gas sensor rises and falls repeatedly within a relatively short fixed period of time, the processing board 224 may determine that the wind from a swinging fan is blowing.

Note that the wind sensor, humidity sensor, temperature sensor, temperature and humidity sensor, and gas sensor used to detect wind are examples of wind detection means. The display device 1 may also have an air pressure sensor as a sensor for detecting wind. This is because changes in air pressure can occur depending on the occurrence and strength of wind.

Various types of control such as those described above can be executed. Note that the above-mentioned control is not limited to character 1100, and may also be performed on communication functions related to other characters (e.g., character 1200).

7. Specific examples of images displayed by the display device 1
Next, specific examples of images displayed by the display device 1 will be described with reference to Figures 41 to 55. Figures 41 to 55 are diagrams showing examples of characters (images including the character 1200 described in Figures 1B and 1C) displayed on the display device 1. Each image shown in Figures 41 to 55 is an image that is displayed on the display device 1 and can be observed by an observer via the observation area SC. Although stage image 2200 is displayed in Figures 41 to 55, character 1200 may be displayed without displaying this.

Figures 41 to 47 show displays that the display device 1 performs during a period when it waits for the observer's actions based on the communication function. The display device 1 (specifically, the processing board 224) performs the displays described below by switching between regular and random. For example, as shown in Figure 41 (a), the character 1200 is displayed in a standing posture facing the front side (observation side) of the display device 1. The display device 1 may use this posture as a basic posture and display the period when communication based on the communication function is not performed. For example, as shown in Figure 41 (b), the character 1200 performs a motion of licking its front paws. For example, as shown in Figures 41 (c) and 42 (a), the character 1200 performs a motion of looking alternately to the left and right. For example, as shown in Figure 42 (b), the character 1200 performs a motion of facing the ground. For example, as shown in Figure 42 (c), the character 1200 performs a motion of stretching while sitting. For example, as shown in FIG. 43(a), character 1200 performs an action of peering down at the ground. Furthermore, as shown in FIG. 43(b) and FIG. 43(c), character 1200 performs an action of walking forward. When character 1200 performs a walking action, display device 1 may not display a stage image, but may display an image in which character 1200 steps in place, with the overall position of character 1200 remaining almost constant. When character 1200 performs an action other than walking, display device 1 may display a stage image at the feet of character 1200.

For example, as shown in FIG. 44(a), the character 1200 makes a motion as if peering into its own abdomen. For example, as shown in FIG. 44(b), the character 1200 makes a motion as if tilting its head. For example, as shown in FIG. 44(c), the character 1200 makes a motion as if rubbing its own eyes. For example, as shown in FIG. 45(a), the character 1200 makes a motion as if yawning. For example, as shown in FIG. 45(b), the character 1200 makes a motion as if licking its own front paws while sitting. For example, as shown in FIG. 45(c), the character 1200 makes a motion as if looking diagonally forward and to the left. As shown in FIG. 46(a) and (b), the character 1200 makes a motion as if stretching. For example, as shown in FIG. 46(c), the character 1200 makes a motion as if peering into the ground. As shown in FIG. 47(a), the character 1200 faces diagonally forward to the right of the display device 1, and then faces forward as shown in FIG. 47(b). As shown in FIG. 47(c), the character 1200 licks its front paws.

Figures 48 to 55 are examples of the display of character 1200 performed by display device 1 (more specifically, processing board 224) during dialogue processing in the communication function. The character strings shown in Figures 48 to 55 are character strings that can be recognized by voice recognition, and are displayed above character 1200. This is preferably done so that a phrase (sentence) recognized by voice recognition is displayed for a predetermined time after voice recognition, and is made invisible after the predetermined time has elapsed. Then, it is preferably controlled so that character 1200 begins speaking after the phrase (voice recognition result) is made invisible.

Fig. 48(a) is displayed when the display device 1 recognizes the observer's utterance of "Come on, come on", and in response, a sound is output. Figs. 48(b) to 49(b) are displayed when the display device 1 recognizes the observer's utterance of "Sit", and in response, a sound is output. Fig. 49(c) is displayed when the display device 1 recognizes the observer's utterance of "I wonder if I'm hungry", and in response, a sound is output. Fig. 50(a) is displayed when the display device 1 recognizes the observer's utterance of "Policeman", and in response, a sound is output. Figs. 50(b) and (c) are displayed when the display device 1 recognizes the observer's utterance of "It's snack time", and in response, a sound is output. Fig. 51(a) is displayed when the display device 1 recognizes the observer's utterance of "Shake your hand", and in response, a sound is output. 51(b) to 52(b) are displayed when the display device 1 recognizes the observer's utterance of "cute," and sound is output in response. When the display device 1 displays the character 1200 at a position other than the center of the stage image (e.g., a position toward the front), when rotating the character 1200, it is preferable to rotate the stage image 2200 in the same manner as the character 1200. In addition, the display device 1 may display a stage image 2200 with a shape that is relatively far away in the depth direction, and display an image of the character 1200 moving inside the stage image 2200 along the outer edge of the stage image 2200 (e.g., an image of a cat character walking along the outer edge of a circular stage).

Fig. 52(c) is displayed when the display device 1 recognizes the observer's utterance of "Hello," and audio is output in response. Fig. 53(a) is displayed when the display device 1 recognizes the observer's utterance of "Eat dinner," and audio is output in response. Figs. 53(b) and (c) are displayed when the display device 1 recognizes the observer's utterance of "Maybe it's a girl," and audio is output in response. Fig. 54(a) is displayed when the display device 1 recognizes the observer's utterance of "Wait," and audio is output in response. Fig. 54(b) is displayed when the display device 1 recognizes the observer's utterance of "Good boy," and audio is output in response. Figs. 54(c) and 55 are displayed when the display device 1 recognizes the observer's utterance of "Yunho," and audio is output in response.

8. More detailed description of the character 1200
The character 1200 described in FIG. 1B, 1C and FIG. 41 to FIG. 55 will be described in more detail. In the following description, the description in which the display device 1 is the processing subject may be regarded as processing performed by the processing board 224. It is particularly preferable that the character 1200 is drawn with approximately 3:1 ratio. It has been found that doing so particularly emphasizes the sense of depth. The display device 1 may draw the character 1200 so that the head occupies approximately one-third of the height, the torso one-third, and the legs one-third of the height in the vertical direction when viewed from the side (side). The display device 1 may draw the character 1200 so that the head occupies approximately one-third of the height, and the torso two-thirds of the height in the horizontal direction when viewed from the side, except for the tail.

The display device 1 should have a function for drawing a tail that alternates between being visible and not being visible when the face is in front, and the time when the tail is visible should be shorter than the time when the tail is not visible. The tail will be drawn further back than the head, but by occasionally seeing the tail, the sense of three-dimensionality can be enhanced, and the constant sense of three-dimensionality can prevent eye and head fatigue. In particular, the display device 1 should perform such drawing processing when in standby for voice recognition. The display device 1 should have a function for drawing the tail so that it appears from the top of the head when the tail changes to being visible. In this way, the viewer's line of sight is usually at the eyes of the face, and the tail suddenly pops out above the head from the top center, providing a sense of surprise and interest.

The display device 1 may provide a flat portion on the top of the head of the character 1200 when viewed from the front, and in particular, may draw the head portion between the left and right ears as flat when viewed from the front. In this way, a three-dimensional tail suddenly appears on top of the head from the flat top of the head, which has a weak three-dimensional effect, and this can provide more surprise and interest. The display device 1 may also draw the tail so that it appears from above the head, and then draw the part near the tip of the tail moving. In this way, the viewpoint can be focused on the tail part behind the face, and a more three-dimensional feeling can be felt. The display device 1 may also draw the tail so that it appears from the left and right sides of the body in the same way. When the tail appears from the left and right, the display device 1 may shorten the length and/or time for which the tail appears compared to when the tail appears from above the head. When the tail appears from the left and right, the display device 1 may draw the tail so that it appears briefly. The display device 1 may draw the line of sight to move occasionally to the back, which can provide a more three-dimensional feeling. The display device 1 may render the tail appearing from above the head during voice recognition, and may render the tail appearing from either the left or right side when the cat outputs a voice. The display device 1 may provide portions of the tail at predetermined intervals along its length that have a rendering style different from other portions. For example, a ring-shaped or arc-shaped pattern with a predetermined width may be applied at regular intervals on the cylindrical surface of the tail. The inventors have found that by doing this, the movement of the tail appears to move more dynamically in three-dimensional space.

The display device 1 may be configured to change its current posture to the same posture when there is a change from a non-voice recognition state to a voice recognition state, so that the posture remains the same during voice recognition. The display device 1 may be configured to say "Let's talk" before voice recognition begins. The display device 1 may be provided with a function to detect the observer's movements (e.g., a sensor or switch), and may be provided with a function to say "What's that?" when it detects a movement from the observer. Both functions may be executed consecutively in the form of "What's that?" and "Let's talk", and then processing may be performed to enter the voice recognition state.

The display device 1 is provided with information on words that are praising, such as "cute," as phrases for speech recognition, and when it recognizes words that are praising, it is advisable to change the posture of the body. It is even better if the cat itself moves, such as by walking. In this way, the observer, wanting to see the change in posture and movement of the cat, will be more likely to use praising words and actions, and excellent therapeutic effects such as improvement of behavior and speech can be obtained.

The display device 1 can be equipped with a voice recognition phrase that corresponds to feeding, and when that phrase is recognized, the display device 1 can be rendered wagging its tail faster than in other cases. In this way, the observer can be given the sensation that the cat is actually alive.

The display device 1 may draw the outline of the cat's eye as a roughly circular shape, and the iris as a vertically long oval shape. The display device 1 may draw the iris with a vertical gradation of light and dark, and may draw it in a lighter color as it goes downward. The display device 1 may provide a roughly circular or oval shape in the center of the iris that is darker than the surrounding area. The display device 1 may draw a crescent or half-moon shaped white area along the outer edge of the iris in the outer part of the face above the iris. In contrast, the display device 1 may draw a circular white area below the iris and closer to the center of the face. The display device 1 may continue drawing this white area even if the position of the character 1200 changes (such as a change in posture) or the position of the camera or light changes. Usually, highlights are added by calculation as a reflection of light, etc. However, by doing this, the individuality of the cat can be emphasized in the part of the eyes that is most visible when talking to the character using voice recognition, and the cat can be made to leave an impression of being unique in the short time that voice recognition is performed.

The display device 1 can be used with the face facing forward and facing downward, and when the face is facing forward, the outline of the eyes is drawn in a roughly circular shape, whereas when the face is facing downward, the eyes are drawn in a horizontally elongated oval shape.

The display device 1 may take a state in which the top of the face is above the top of the torso and a state in which the top of the face is below the top of the torso, and when the top of the face is above the top of the torso, the outline of the eyes may be drawn in a roughly circular shape, whereas when the top of the face is below the top of the torso, the eyes may be drawn in a horizontally long oval shape. The display device 1 has a function for drawing a state in which the mouth is open and yawning, and when drawing this, the eyes may be drawn closed.

The display device 1 may prepare a number of patterns of standby motions for the character 1200, and may repeatedly select one of these patterns and draw it. The standby motions may include a foot licking motion, a hand stroking the face, a butt licking motion, a forward facing motion, a look around to the left and right, a ground licking motion, a stretching motion, a yawning motion, a walking motion, etc.

The display device 1 may be arranged so that the ears move after the face. For example, the ears may be arranged so that they remain in position for a while even after the face starts moving. The display device 1 may be provided with a function for simultaneously performing a tail-swinging motion from side to side when performing a licking motion. The display device 1 may be arranged so that the pattern on the body of the character 1200 is asymmetrical when viewed from the front. This makes it easier to remember the character as a unique cat, and makes it easier to distinguish whether the right side or the left side is being viewed. This is particularly useful when providing a scene that is so enlarged that the entire image of the character 1200 cannot be seen. In particular, the pattern may be a calico cat pattern, and more particularly, a tabby cat pattern. The inventors have found that this makes it particularly easy to grasp the three-dimensional effect. In particular, the pattern on the head may be arranged in the front-to-back direction, while the pattern on the body may be arranged in the up-to-down direction, so that the directions of the patterns are different. This makes it easier to recognize the change in the position of the camera or the cat, regardless of the direction in which they move.

The display device 1 may have a function of emitting the cry "meow", and may have a number of intonations for the "meow" cry, with each cry corresponding to a different motion. Among the intonations, a questioning intonation may be included, and as a motion, a head tilting motion may be included.

<9. Communication Functions for Character 1200>
A supplementary explanation will be given regarding the communication function regarding the character 1200. By doing the following, it is possible to provide the observer with a higher quality communication function with the character 1200. Here, an outline of the action story between the person and the character 1200 until the character is tamed will be explained. In this embodiment, as shown in FIG. 32, a photographing section 227 for photographing the observer is provided on the front side of the display device 1.

(9-1) First step: stage of taming Usually, there are various types of cats, such as stray cats, that run away just by seeing a person, or run away when a person approaches. At that time, the cat's behavior appears to be running away while looking back at the person repeatedly. Here, the processing board 224 recognizes the presence of an observer based on the captured image captured by the imaging unit 227, and when the strength (received strength) of the microwave measured by the human sensor 234 exceeds a threshold, it displays an image in which the character 1200 behaves as if it is running away. This is to consider that a person has approached the cat. The imaging unit 227 recognizes the observer in order to show the observer a scene in which the character 1200 runs away. In other words, the character 1200 does not run away unless the observer approaches the display device 1 from the front side.

Next, the person is likely to call out to the cat, "Squat down (lowering the position of the eyes)" or "Come on, come on", etc. Some cats will stop at this time and watch the behavior. However, when the person approaches, the cat will run away. This happens several times. It is likely that the cat will not get used to this type of human behavior. Therefore, if the microwave intensity measured by the human presence sensor 234 during the period when the image of the character 1200 running away is below a certain threshold and the processing board 224 recognizes voice recognition of words such as "Come on", which indicate that the character 1200 is being called out to, displays an image of the character 1200 making an action to stop. Here, the condition that the microwave intensity is below the threshold is set because it is assumed that the human will stop.

Next, a person may feed the cat that has stopped. However, if a person approaches the cat, the cat will run away, so it is considered that the only option is to place food in a place where the cat often comes. Therefore, the display device 1 has a function of displaying an image of food being placed in the image display space. For example, when the processing board 224 receives a predetermined operation using the operation area OT (for example, an operation of the operation unit 2214), or when a predetermined gesture instructing the first sensor 2231 to place food is detected, the processing board 224 displays an image showing food in the image, assuming that food has been placed. This image may be displayed using either the first display unit 310 or the second display unit 330. In this way, the observer will place food, for example, every day. At this time, the processing board 224 may display the character 1200 as if it is watching the situation from a distance. When the processing board 224 does not recognize the presence of the observer using the human sensor 234 and the observer is not photographed by the photographing unit 227, the processing board 224 displays an image of the character 1200 eating food. Here, a situation is reproduced in which a cat does not eat food in front of a person. From the observer's perspective, a situation is created in which the food is gone before the observer notices it. The processing board 224 counts the number of times the observer feeds the cat and stores it. This state continues for a while, but the processing board 224 may display the distance between the observer, the character 1200, and the person (the distance felt by the observer) gradually decreasing depending on the number of times the feed is given. For example, the processing board 224 may start feeding the cat and display the character 1200 larger along with the number of times the feed is given, recreating the distance decreasing (i.e., getting used to it a little). Furthermore, when the number of times the feed is given increases and reaches a predetermined threshold, it is determined that the cat has gotten used to it, and even if the observer is looking at the character, that is, even if the processing board 224 recognizes the presence of the observer using the human sensor 234 and the observer is being photographed by the photographing unit 227, an image of the character 1200 eating food is displayed. Only at this time can the observer observe the character 1200 eating food.

However, people often do not know when a cat will eat. Therefore, the processing board 224 may determine the time period during which the character 1200 will eat (or may alternatively be a single point in time) and have the character 1200 perform the action of eating food during that time period. This is to make the observer aware that the character seems to be eating food during that time period. When the processing board 224 detects that the observer is about to touch the character 1200, it causes the character 1200 to output a predetermined voice content, such as "squeak" or "squeak squeak," and causes the character 1200 to perform the action of running away. The observer's attempt to touch the character 1200 may be identified by the intensity of the microwaves detected by the human sensor 234 exceeding a threshold value when the character 1200 is performing the action of eating food, or by the first sensor 2331 detecting a predetermined gesture.

This situation continues for a while, and the processing board 224 may gradually reduce the distance at which the observer's hand can approach the character 1200. When the character approaches a predetermined distance, the processing board 224 may determine that the character 1200 has become accustomed to the observer and display an image of the character grooming itself or relaxing in public. Specifically, the processing board 224 may increase the threshold for microwave intensity, or increase the threshold for the distance from the display device 1 when a gesture is made. In this way, the processing board 224 may lower the detection sensitivity, making the character 1200 less responsive to the observer's movements and preventing it from making utterances such as "shrimp" or "shrimp shrimp."

Although cats are always on guard, they will begin to groom themselves, lie down, sharpen their claws, wipe their faces, and so on. As cats gradually become accustomed to people and their hands, people will be able to touch them with their hands. Therefore, the processing board 224 may not reflect the microwave detection results of the human sensor 234 in the behavior of the character 1200. The processing board 224 may cause the first sensor 2331 to function as a motion sensor, and may display an image of the character 1200 feeling good when stroked according to the detection results of the observer's gestures in the up, down, left, right, and front and back directions. The processing board 224 may further recognize the observer's voice using the microphone 2333, and display an image of the character 1200 responding to this. For example, the processing board 224 may be configured to make the character meow as the spoken content regardless of what words it recognizes.

The processing board 224 may further display an image of the character 1200 begging for food as time passes. The processing board 224 may recognize the presence of the observer by the image capture unit 227, and when the presence of the observer is detected by the human sensor 234, may determine a time period (e.g., morning, afternoon, or evening) and display an image of the character 1200 making a meowing sound and rubbing its face against the observer. Furthermore, the processing board 224 may reproduce the state in which the character 1200 learns its own name as the observer calls out to it. For example, the processing board 224 may only respond to the word "Yuno" by making a meowing sound, or may display an image of the character behaving in a cute manner or rubbing its face against the observer in response to the words "Yuno" and "Here's some food."

(9-2) Second step: learning language (Japanese) When the processing board 224 recognizes the observer's call of "Yuno" at some point, it outputs a voice indicating a response of "What?". From this point on, the character 1200 enters the stage of learning words. The processing board 224 counts the number of times it is called out, calculates the cumulative average/day, and responds according to a value calculated by multiplying the cumulative average number of times food is given/day by the cumulative average number of times it is called out by a random coefficient (hereinafter referred to as the "accumulated average value"). This call is, for example, a predetermined call such as "Good morning,""Hello,""Goodevening,""How are you?", or "Do you want food?". Depending on the cumulative average value, the processing board 224 stops responding (i.e., reproduces the character 1200's forgetting of words) or increases the number of words it responds with. For example, the processing board 224 may reduce the number of words it responds with as the cumulative average value becomes smaller, and increase the number of words it responds with as the cumulative average value becomes larger. The cumulative average value is calculated based on the number of times the display device 1 is called out from the start of use to the current time, but may also be calculated based on the number of times the display device 1 is called out from a predetermined period of time back from the current time to the current time. The predetermined period may be a fixed period such as one month, or if the display device 1 has not been used for a certain period of time, it may be the period from the time when the display device 1 was used after that period. In the beginning of use of the display device 1, the cumulative average value may vary greatly, so that if the observer does not call out to the character 1200 even once a day, the character 1200 may not respond, but as the number of times the character is called out to increases, the cumulative average value does not change significantly. In this way, it is possible to reproduce the state in which it takes time for the character 1200 to forget words. The random coefficient used in calculating the cumulative average value is a value for reproducing the whimsical nature of a cat, and may be called a whimsical coefficient. The random coefficient may be varied depending on the actions of the character 1200 and the observer, such as giving/not giving food to the character 21200 when it begs for food, or waking up the character 1200 by talking to or touching it when it is sleeping.

The processing board 224 may further arrange for the character 1200 to speak to the observer or respond to questions about specific information such as the weather or the news when the cumulative average value exceeds a certain threshold. Of course, if the observer reduces the number of times the character is fed, the cumulative average value will decrease, and the processing board 224 will reduce the types of responses or stop responding altogether. In other words, if the observer is careless about feeding the character or does not speak to the character appropriately, the character 1200 may not maintain its ability to respond, or even if it does, it may not respond due to "whims." In this way, a higher quality communication function with the character 1200 can be provided to the observer.

[Other embodiments]
The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit and scope of the present disclosure.

The display device 1 may have an operation receiving means in the observation area SC, such as a touch sensor, a proximity sensor, or other operation receiving means for receiving operations from the observer. In this way, the observer can use the observation area SC to perform operations. For example, if such an operation method is adopted in the communication function, it is expected to have the effect of improving the quality of communication with characters that can be observed through the observation area SC.

The first display unit 310 may display an image that is recognized as a three-dimensional image (stereoscopic image). In this way, it is expected that the effect of further improving the sense of depth given to the observer can be achieved. As a display method, a naked-eye stereoscopic method such as a lenticular lens method or a parallax barrier method, or a display method that allows the image to be recognized as a three-dimensional image using glasses may be used.

The display device is not limited to a rectangular parallelepiped shape, and may be a cube, a cylinder, other polyhedrons, a sphere, or other shapes. The shapes, structures, dimensions, colors, patterns, etc. of the components of the display device 1 described above are merely examples. Some of the components of the display device 1 described above may be omitted, or other components may be added. For example, the second display unit 330 may be omitted, or the second display unit 330 that emits the second image light L2 in the first direction side may be arranged in an upright state instead of the background plate 910. Aerial imaging may be realized using an optical system other than that described in FIG. 13. The configuration of the present invention may be applied to a display device of a display type or a projector type, rather than an aerial imaging type. In the case of a display type, a liquid crystal display, an organic EL display, or other display type display is used as the image display unit. For example, a display unit provided inside the image display space and having a display surface facing the direction of the observation area SC (for example, a display surface parallel to the observation area SC) may be used as the image display unit. This display unit may have the same configuration as the first display unit 310 or the second display unit 330, and may be provided on the rear side of the image display space. In the case of a projector system, a projection surface and a projection means for projecting an image onto the projection surface may be used. The dimming member 200 may be disposed on the front side or rear side of the opening 110 to reduce light entering the image display space from the outside. If the image display space S1 is to be maintained in a relatively dark state, a configuration without the dimming member 200 may be considered.

In the above embodiment, voice recognition is enabled in response to pressing the operation unit 2214 to interact with the character displayed on the video display unit, but this is not limited to this. For example, when the presence of a person is detected in the detection target area of a human presence sensor (one example of a detection means) such as a full-circle microwave Doppler sensor, the processing board 224 may enable the voice recognition. In this way, the user does not need to perform an operation to enable voice recognition or speak a trigger word when interacting with a character displayed on the video display unit, and communication can be reproduced more naturally.

At this time, for example, it is preferable that both the voice recognition in the voice recognition application in the display device 1 and the voice recognition in the voice recognition server are enabled. However, as described below, enabling of the voice recognition in the voice recognition application in the display device 1 and enabling of the voice recognition in the voice recognition server may be performed in stages. For example, when the presence of a person is detected in the detection target area of the human presence sensor, etc., the voice recognition in the voice recognition application in the display device 1 may be enabled, and thereafter, when the voice recognition application detects a call or greeting from an observer, the voice recognition in the voice recognition server may be enabled.

In addition, if the display device 1 is equipped with a photographing unit such as a camera (an example of a detection means), a detection target space may be provided on the front side of the display device 1, and the processing board 224 may be configured to enable voice recognition when the presence of a person is detected on the front side of the display device 1. In this way, it is possible to more reliably determine that the observer is trying to speak to the character displayed on the video display unit, and then enable voice recognition.

Furthermore, after a person is detected in the detection target area, the processing board 224 may be configured to activate the voice recognition on condition that user authentication is successful. The authentication method for user authentication may be, for example, an authentication method that does not require manual operation by the user (face recognition, vocal cord recognition, etc.). In this way, for example, when user authentication is performed on a person other than a registered user registered in the display device 1 and the user authentication fails, voice recognition is not activated and it is not possible to interact with the character displayed on the video display unit. Conversely, the character displayed on the video display unit does not react to people other than registered users. This can increase the user's attachment to the character.

In addition, if speech containing words other than those pre-registered in a dialogue server installed on the display device 1 or an external server is detected, speech recognition may not be enabled.

Also, an odor sensor may be used as a means for activating voice recognition. For example, the odor of a registered user of the display device 1 may be registered in advance in the display device 1, and voice recognition may be activated when the odor of the registered user is detected by the odor sensor.

Furthermore, even if voice recognition is enabled in this manner, it is preferable that the voice of a character (e.g., character 1100) displayed on the video display unit is not output immediately, but rather the voice of the character is not output unless the user speaks to it. For example, the processing board 224 does not immediately output the character's voice saying "Welcome back" in response to voice recognition being enabled, but rather outputs the character's voice saying "Welcome back" on the condition that the user's voice saying "I'm home" is detected while voice recognition is enabled. Furthermore, for example, in the dialogue system, a branching scenario may be created to reproduce the following dialogue between an observer and a character (e.g., character 1100). For example, on a weekday,
User: I'm back. Character: Welcome back.
User: Atchi Character: Thank you for your hard work in this hot weather.
It would be good if branching scenarios were created to recreate conversations like the following:User: I'm homeCharacter: Ah, welcome backUser: It's okCharacter: You went out of your way in this heat, so don't complainUser: It's work, workCharacter: I'm sorry if that's true, thank you for your hard work.

In addition, a branching scenario may be created according to the detection result by the odor sensor. For example,
User: The meal was delicious. Character: "Hmm, let me smell it. Sniff sniff. It smells good!" or "Hmm? Isn't that a lot of salt? Take care of yourself!"
A dialogue such as the following may be reproduced.

As another example, when an odor is detected by an odor sensor,
Character: Sniff sniff, what's that smell?
A dialogue such as the following may be reproduced: User: I lit an aroma. Character: Ah, that's why I thought it would be relaxing. If the observer's response to the question about the type of smell is not registered in the branching scenario, a voice such as "Oh, I see!" may be uniformly output as the character's voice.

The processing board 224 may also record the voice for a recent predetermined time (for example, about 15 seconds) at all times, and when the voice of a trigger word for activating the voice recognition is detected, activate the voice recognition. For example, the voice recognition application of the display device 1 may be activated at all times (or in response to a pressing operation of the operation unit 41, etc.), and the voice recognition by the voice recognition server may be activated by the detection of a trigger word. At this time, the processing board 224 may transmit to the voice recognition server voice data of the voice detected by the microphone 2333 (an example of a voice detection means) (for example, the voice detected between the start and end of detection) excluding the voice of the trigger word. For example, it is assumed that a user voice saying "I'm going to Tokyo tomorrow, so can you tell me the weather, Rei-tan?" has been detected. In this case, the nickname of the character 1100, "Rei-tan", is the trigger word. In this case, the processing board 224 transmits the detected voice "I'm going to Tokyo tomorrow, Rei-tan, tell me the weather," excluding the trigger word voice "Rei-tan," to the voice recognition server: "I'm going to Tokyo tomorrow, Rei-tan, tell me the weather."

Here, for example, if a detected voice such as the above, "I'm going to Tokyo tomorrow, so can you tell me the weather, Rei-tan?", is sent as is to the voice recognition server, there is a risk that the character voice corresponding to the voice recognition result will not be accurately extracted in the dialogue server due to the inclusion of "Rei-tan" in the voice recognition result, and dialogue with the user will not flow smoothly. In contrast, by doing as described above, even if a trigger word is uttered in the middle of a sentence, the voice data of the voice excluding the trigger word is sent to the voice recognition server. Therefore, it is possible to prevent dialogue with the user from flowing smoothly due to voice recognition being performed when a trigger word is included in the middle of a sentence.

In addition, when realizing a dialogue between a user and a character displayed on the video display unit through cooperation between a voice recognition server and a dialogue server, the following operations may be performed depending on the configuration of the voice recognition server.

For example, if the voice recognition server is configured to start voice recognition processing after completing reception of the voice data file, the processing board 224 may individually transmit to the voice recognition server each voice data file of the voice before and after the voice of the trigger word among the voices detected by the microphone 2333. For example, among the voices detected between the start and end of detection, the voice data file of the user voice detected before the voice of the trigger word (also called the first half of the user voice) may be transmitted to the voice recognition server prior to the voice data file of the user voice detected after the voice of the trigger word (also called the second half of the user voice).

In this way, the voice recognition server starts voice recognition processing for the first half of the user voice without waiting for the completion of reception of the voice data file of the second half of the user voice. Therefore, voice recognition processing can be performed more efficiently than when a voice data file of a voice that combines the first half and second half of the user voice is sent to the voice recognition server. As a result, the display device 1 can receive the character voice from the dialogue server earlier, and the user's waiting time until a response to the user's voice can be shortened.

For example, if the voice recognition server is configured to stream voice data from the display device 1 and execute voice recognition processing in real time, when the voice data of the first half portion is transmitted to the voice recognition server in real time, the playback speed T1 of the user voice of the first half portion may be controlled to be faster than the standard playback speed T0 (T1>T0). For example, the processing board 224 changes the settings so that the user voice of the first half portion is reproduced 25% faster than the standard playback speed T0 in the voice recognition server and transmits the voice data of the first half portion. In this way, the voice recognition server can end the voice recognition processing of the user voice of the first half portion earlier and start the voice recognition processing of the user voice of the second half portion earlier, compared to when the user voice of the first half portion is reproduced at the standard playback speed T0. As a result, the display device 1 can receive the character voice from the dialogue server earlier, and the user's waiting time until a response to the user's voice can be shortened.

Furthermore, when the voice data of the latter half of the user voice is transmitted to the voice recognition server in real time, it is preferable to control the playback speed T2 of the voice of the latter half to be faster than the standard playback speed T0 and slower than the playback speed T1 of the user voice of the first half (i.e., satisfying the relationship T0<T2<T1). For example, the processing board 224 changes the setting so that the user voice of the latter half is played back 10% faster than the standard playback speed T0 in the voice recognition server, and transmits the voice data of the latter half. In this way, both the user voice of the first half and the user voice of the latter half are stream-played back faster than the standard playback speed T0, and the voice recognition process is performed in the voice recognition server. As a result, the user's waiting time until a response is made to the user's voice can be further shortened. On the other hand, the user voice of the latter half is played back at a speed faster than the standard playback speed T0 but slower than the playback speed T1 of the user voice of the first half. Therefore, the deterioration of the voice recognition accuracy in the voice recognition system can be suppressed. Therefore, the user's waiting time can be further shortened while suppressing the deterioration of the voice recognition accuracy.

When the voice recognition server performs voice recognition processing in real time by playing back a stream of voice data, for example, the user voice detected by the microphone 2333 is transmitted to the voice recognition server in real time, but voice recognition in the voice recognition server is controlled so as not to start until the voice of the trigger word is detected.

<10. Details of communication function related to character 1200>
Next, the details of the communication function related to the character 1200 described above will be described below. In particular, the first step of taming the character and the second step of learning a language will be described below. In addition, the function of the character 1200 that has learned a language in the second step to send and respond to predetermined information such as the weather and news will be described in detail.

In addition, since the communication function, for example, has been improved compared to the above-mentioned display device 1 and therefore goes beyond the scope of the function expressed as a "display device," hereinafter, it will be referred to as system 1 instead of the above-mentioned display device 1. However, since the external configuration, display principles, internal configuration (structure), etc. of system 1 are similar to those of the above-mentioned display device 1, explanations of these will be omitted.

In addition, in the explanation of the display device 1, the person observing the image displayed on the display device 1 is referred to as the "observer." However, as mentioned above, the system 1 goes beyond the scope of a display device, and therefore, hereafter, the term "observer" will be used instead of "user."

(System 1 concept)
First, the concept of the system 1 will be briefly described with reference to Fig. 56. Fig. 56 is a diagram briefly explaining an example of the development concept of the system 1 according to an embodiment.

Similar to the display device 1 described above, characters such as animals are virtually displayed in the image display space S1 of the system 1. In this embodiment, a cat is displayed as an example of a character virtually displayed in the image display space S1 of the system 1. As described above, the name of this cat is Juno (also called JUNO).

As shown in FIG. 56, JUNO is a character who wants to become friendly with, for example, the user who is the owner of the system 1, but is a little nervous at first and is unable to communicate well, and his movements are realized by software control. For example, JUNO is virtually and pseudo-reproduced as having instincts such as innate qualities and characteristics, just like a real life form such as an animal. Through such software control, JUNO has an animal-like quality and performs animal-specific behaviors in response to changes in the environment, etc. In the following description, when it is expressed that JUNO has a virtual instinct function, it means that JUNO is realized by software control so that he behaves as if he has an instinct. The same is to be understood for various other actions performed by JUNO (for example, speech and other behaviors). Software control may or may not involve artificial intelligence technology, and there is no question about the specific implementation method.

For example, in the initial settings, Juno is set to be highly wary (i.e., low intimacy with the user) and unable or difficult to communicate with the user, and in some cases may be configured to behave in a threatening manner towards the user. Then, when the user takes an action such as feeding Juno every day, Juno's wariness may begin to ease and the intimacy between the user and Juno may increase. As the intimacy between the user and Juno increases, Juno's behavioral patterns may change and Juno may behave in a way that surprises the user. As the intimacy between the user and Juno increases, Juno may be able to have everyday conversations with the user, for example, allowing the user to spend relaxing and healing time with Juno.

In this way, for example, by setting the initial settings to make the character more wary, unable to communicate well with the user, or threatening to the user, it is possible to realistically express the instinctual functions of a living creature. As the intimacy between the user and JUNO increases, it becomes possible to communicate with JUNO as if it had a virtual instinctual function. JUNO is a character that imitates a living creature (for example, a living organism such as an animal, and in the embodiment of this system, a cat). In this way, it is possible to enjoy daily life with a character 1200 (virtual animal) that is as realistic as if it actually exists. Note that the character 1200 can be, for example, a human, a non-human animal, a fantasy creature, a non-existent character, etc., as long as it can be expressed with a level of realism as if it actually exists.

(Character behavior patterns change depending on intimacy)
First, an example of JUNO's behavioral pattern according to the intimacy between the user and JUNO will be briefly described. The intimacy between the user and JUNO can be expressed, for example, by the sense of distance between the user and JUNO. If the intimacy between the user and JUNO is low, JUNO will not approach the user as a virtual instinct function, and will likely keep a certain distance between the user and JUNO. On the other hand, if the intimacy between the user and JUNO increases, JUNO will become less wary of the user and may approach the user. Therefore, in the present system 1, JUNO's behavioral pattern is configured to be a behavioral pattern in which the distance between the user and JUNO can change according to the intimacy between the user and JUNO.

The distance between the user and JUNO can be changed, for example, by changing the display position of JUNO in the three-dimensional image display space S1. The display position of JUNO in the image display space S1 will be described below with reference to FIG. 57. FIG. 57 is a diagram showing an example of the display form in the image display space S1 of system 1 according to one embodiment. By allowing JUNO to move freely within the three-dimensional image display space S1, it is possible to create a sense of presence as if the virtually displayed JUNO actually exists.

As shown in FIG. 57, in the image display space S1 of the system 1, a plurality of areas are displayed, for example, in the depth direction, as a three-dimensional virtual space image. The plurality of areas are displayed so that the sense of distance when viewed from the front varies depending on the area. In this embodiment, for example, the plurality of areas capable of displaying JUNO are divided into Area 1 on the front side, which is the side closest to the user, Area 5 on the back side, which is the side furthest from the user, and Areas 2 to 4 which are intermediate between Area 1 and Area 5. Of Areas 2 to 4, Area 2 is closest to the user in order of Area 3, and Area 4. It is also possible to display Area 0 closer to the user than Area 1, which is not normally displayed but is used, for example, when performing a zoom display, which will be described later. However, in this embodiment, Area 0 is not an area capable of displaying JUNO, but JUNO may be displayed therein.

Of the multiple areas displayed in the depth direction of the video display space S1 of the system 1, an image of, for example, a character is displayed largest in area 1, which is the side closest to the user (the lower side of the display screen), and an image of, for example, a character is displayed smallest in area 5, which is the side farthest from the user (the upper side of the display screen). The images of characters are then displayed so that they become gradually smaller from area 1 to area 5. This makes it possible to further emphasize the sense of depth.

The processing board 224 is configured to be able to control JUNO to move between areas 1 to 5 in the video display space S1 of the system 1 according to the intimacy between the user and JUNO. In this embodiment, the intimacy is abstract and conceptual, and an experience value is used as a parameter that directly or indirectly specifies the intimacy. Intimacy may be used as a parameter instead of the experience value. The experience value is an example of experience information of the present invention. The intimacy is an index of the intimacy between the user and JUNO, and the larger the experience value, the higher the intimacy, and the smaller the experience value, the lower the intimacy. The experience value is an index value of how much JUNO has experienced the action (behavior) performed by the user on JUNO, and changes, for example, depending on the performance of the action. The performance of the action is specified, for example, by the number or frequency of the action, but it is even better if the specific content of the action is also taken into account. Typically, the more actions JUNO experiences from the user on JUNO, the higher the experience value becomes, and the intimacy changes accordingly in a higher direction. In general human relationships and between humans and animals, the more active the communication, the closer the relationship between the two will be, so this can be said to be a pseudo-representation of this. The experience value may not only change in an upward direction, but also in a downward direction, in which case the intimacy level will change in a downward direction. For example, the less JUNO experiences the action from the user toward JUNO, for example, the fewer the number or frequency of the action, the lower the intimacy level. The specific content of the action may be taken into account, and for example, even if the user takes an action toward JUNO, the intimacy level may be lowered if the action is not one that JUNO likes. In this case, it can be understood that there is a lack of experience of the action that JUNO likes, and the intimacy level will be lowered. In general human relationships and between humans and animals, the less good communication is experienced, the closer the relationship between the two will be, so this can be said to be a pseudo-representation of this. In this embodiment, the action toward JUNO is realized by the user inputting information into the system 1. Next, the relationship between the user's action and the experience value or the intimacy level understood therefrom, and the operation of the system 1 related to this will be described.

For example, if the intimacy between the user and JUNO is low, the ratio (i.e. probability) that JUNO will move to an area where the distance to the user is relatively far is high, and as the intimacy between the user and JUNO increases, the probability that JUNO will move to an area where the distance to the user is relatively close increases. By doing this, JUNO can be displayed as if he is actively moving away from or towards the user based on his virtual instinct function, making it possible to make JUNO even more realistic.

(About intimacy)
The processing board 224 adds a predetermined value to the experience value when, for example, JUNO has a specific experience as a preset condition. When this experience value increases, the processing board 224 controls JUNO to perform a behavior pattern as if the intimacy between the user and JUNO, which is an abstract concept, has increased. In this embodiment, for example, the number of times (number of days) that the user has fed JUNO is used as a parameter of the experience value. When the number of times (number of days) that the user has fed JUNO is used as a parameter of the experience value, the experience value is added by 1 when the user feeds JUNO. In this embodiment, the upper limit of the experience value added per day is 1. In other words, even if the user feeds JUNO multiple times in a day, the experience value added is only 1. In this way, it is possible to have the user use the system 1 for a long period of time. However, the upper limit of the experience value added by the user feeding JUNO is not limited to 1 as long as it is within a range of values that allow the user to use the system 1 for a long period of time, and may be, for example, 2 or 3, or the experience value may be added by the number of times the user has fed JUNO. Moreover, when the number of days that the user does not feed JUNO exceeds a preset number of days, the experience value is subtracted by a predetermined value (for example, 1). The process of adding and subtracting the experience value is performed by a cloud server described later, but may be performed by the processing board 224. The method of feeding JUNO by the user will be described later. The food is an example of an item given to JUNO, and may be replaced with or combined with a ball, a toy, or other items. The number of times (number of days) that the user fed JUNO is used as a parameter of the experience value is one example, and the number of times the user took an action, such as the time the user used the system, the number of times it was used, or the number of times JUNO was spoken to, may be used as a parameter of the experience value, or these may be combined. In addition, when an unfriendly parameter, for example, is used as a parameter of the experience value, the intimacy level decreases when the unfriendly parameter is added. In addition, the experience value is an example of the experience information of the present invention as described above, but it may be a parameter related to the usage record of the communication function between the user and JUNO as a higher concept than the experience information.

In light of the fact that some users may wish to increase their experience points in a short period of time depending on their preferences, the upper limit of experience points that are added by the user feeding JUNO may be set to a relatively large value, such as 5 or 10, or there may be no upper limit and the experience points may increase in proportion to the number of times the user feeds JUNO. Furthermore, if the user is allowed to set the upper limit of experience points that are added by feeding JUNO, it is possible to provide a system 1 that allows the user to set the speed at which experience points increase according to the user's preferences.

In this embodiment, for example, if the number of times (number of days) the user has fed JUNO is between 0 and less than 30, the intimacy level is defined as low, if the number of times (number of days) the user has fed JUNO is between 30 and less than 60, the intimacy level is medium, and if the number of times (number of days) the user has fed JUNO is 60 or more, the intimacy level is high.

(Character area movement)
As described above, when the intimacy between the user and JUNO is low, the probability that JUNO will move to an area where the distance between the user and the user is relatively far increases, and as the intimacy between the user and JUNO increases, the probability that JUNO will move to an area where the distance between the user and the user is relatively close increases. Therefore, the probability that JUNO will move to each area depending on the intimacy will be described with reference to Figures 58A to 62.

Fig. 58A is a table showing an example of a formula etc. for calculating the probability that JUNO (character) will move to areas 1 and 2 depending on the number of times (number of days) that a user has fed JUNO in a system 1 according to one embodiment. Fig. 58B is a table showing an example of a formula etc. for calculating the probability that JUNO will move to areas 3 to 5 depending on the number of times (number of days) that a user has fed JUNO in a system 1 according to one embodiment. Fig. 59 is a table showing a formula etc. for calculating the probability that JUNO will move to each area (areas 1 to 5) depending on the number of times (number of days) that a user has fed JUNO in a system 1 according to one embodiment. The formulas shown in Figs. 58A, 58B, and 59 are tables that show the same basic formula in different ways. Specifically, Figs. 58A and 58B are tables that show the basic formula for calculating the probability of moving to each area and the formula for calculating the coefficients a, b, and c included in the basic formula for each level of intimacy. Meanwhile, Figure 59 is a table showing the basic formula for calculating the probability of moving to each area, and the formula obtained by substituting the formula for calculating the coefficients a, b, and c for each intimacy level into the basic formula.

Fig. 60(A) is a table showing an example of values substituted for coefficients X (X1, X2, X3) included in the formula for calculating coefficients a, b, and c in system 1 according to one embodiment, and used to classify intimacy into low, medium, and high intimacy. In this embodiment, as described above, low intimacy is defined as the number of times (number of days) that the user has fed JUNO is 0 or more and less than 30, medium intimacy is defined as the number of times (number of days) that the user has fed JUNO is 30 or more and less than 60, and high intimacy is defined as the number of times (number of days) that the user has fed JUNO is 60 or more. Therefore, as shown in Fig. 60(A), in any of the cases of low, medium, and high intimacy, the upper limit of the number of times (number of days) that the user has fed JUNO is set to, for example, 30 days. Therefore, for example, if the number of times (days) the user has fed JUNO is 0 or more and less than 45 days, the intimacy level is low, the number of times (days) the user has fed JUNO is 45 or more and less than 90 days, the intimacy level is medium, and the number of times (days) the user has fed JUNO is 90 days or more, the intimacy level is high, then X1 = 45, X2 = 45, and X3 = 45 as shown in FIG. 60(A). Note that, for convenience of calculation, in the case of high intimacy, the upper limit of the number of times (days) the user has fed JUNO is set to, for example, 30 days, but even if the number of times (days) the user has fed JUNO exceeds 30 days, the probability of moving to each area is determined using the same formula as in the case of high intimacy.

Figure 60 (B) is an example of a table in which upper and lower limits are set for the probability of moving to each area at each level of intimacy (low intimacy, medium intimacy, high intimacy) in a system 1 according to one embodiment. For example, the probability of moving to Area 1 at medium intimacy has an upper limit of, for example, 0.3 (30%) and a lower limit of, for example, 0.05 (5%).

Figure 61 shows tables indicating coefficients included in a basic formula indicating the probability of moving to Area 1 depending on intimacy (low intimacy, medium intimacy, high intimacy) in a system 1 according to one embodiment, and includes (A) an example of a table indicating coefficient a, (B) an example of a table indicating coefficient b, and (C) an example of a table indicating coefficient c. For example, as shown in Figure 61 (A), the value of coefficient a for high intimacy is set to 10.9091.

Figure 62 shows tables indicating coefficients included in a basic formula indicating the probability of moving to areas 2 to 4 depending on intimacy (low intimacy, medium intimacy, high intimacy) in system 1 according to one embodiment, where (A) is an example of a table indicating coefficient A, and (B) is an example of a table indicating coefficient B. For example, as shown in Figure 62(A), the value of coefficient A included in the basic formula indicating the probability of moving to area 3 when intimacy is medium is set to 0.20. Also, as shown in Figure 62(B), the value of coefficient B included in the basic formula indicating the probability of moving to area 2 when intimacy is high is set to 0.05.

By doing this, the more times (number of days) the user feeds JUNO, the higher the probability that JUNO will move to area 1 (including the probability of staying in area 1), and JUNO will be displayed as coming closer to the user. However, if the user behaves in a way that JUNO does not like, it is also possible to increase the probability that JUNO will move to area 5, for example, or decrease the probability that JUNO will move to area 1, for example (including the probability of staying in area 1).

Next, the concept of the probability of moving to Area 1 will be explained. For example, when keeping an animal that is not used to humans at all, such as a stray cat, one would first start by getting the animal used to humans. Even if an animal starts to receive food, it is thought that it probably does not get used to humans linearly, but rather gets used to humans little by little. Therefore, in this embodiment, in the case of low intimacy, the rate of change of JUNO as it gets used to humans is not constant, and the more time passes, the greater the probability of moving to Area 1 becomes. In other words, when the number of times (number of days) that a user feeds JUNO is used as a parameter of experience value, in the case of low intimacy where the number of times (number of days) that JUNO has been fed is less than 30 days, for example, the amount of change in the probability of moving to Area 1 relative to the increase in the number of times (number of days) that JUNO has been fed increases. In this way, not only can the user feel a sense of closeness to JUNO, but the user can also embody the process of JUNO's accustoming to humans accelerating through the rate at which JUNO moves to Area 1. The formula for low intimacy shown in the Area 1 column of Figure 59 (hereinafter referred to as "first formula") shows that the change in the probability of moving to Area 1 with respect to the increase in the number of times (number of days) feeding increases as the number of times (number of days) feeding increases. Note that the change in the probability of moving to Area 1 with respect to the increase in the number of times (number of days) feeding increases as the number of times (number of days) feeding increases, but this is just one example, and if an aloofness parameter is used as the experience value parameter, the change in the probability of moving to Area 1 with respect to a decrease in the experience value can also be made to be greater.

However, for example, animals do not get used to something forever, but rather hit a plateau at some point and the rate of change in habituation is thought to approach zero (approaching the asymptote of y = 1). In other words, at a high level of intimacy, where the number of times (number of days) the user has fed JUNO is, for example, 60 days or more, the amount of change in the probability of movement to Area 1 relative to the increase in the number of times (number of days) the user has fed JUNO decreases as the number of times (number of days) the user has fed JUNO increases. The formula that expresses this is the formula for high intimacy shown in the Area 1 column in Figure 58A (hereinafter, this formula will be referred to as the "third formula"):

For example, when an animal begins to get used to something, it is thought that the changes it undergoes will become greater, but at some point the rate of change will reach a maximum, after which the rate of change will decrease, leading to the third equation. In other words, it is necessary to smoothly connect the first and third equations.

In this embodiment, as shown in Figures 58A, 59B, and 59, the probability of moving to Area 1 at high intimacy is the same as the first formula that indicates the probability of moving to Area 1 at low intimacy. However, for ease of explanation, hereinafter, the formula that indicates the probability of moving to Area 1 at high intimacy is referred to as the second formula.

In this embodiment, we will explain how to easily, i.e., without placing a burden on the software, use the above first, second, and third formulas to vary the probability that JUNO will move to Area 1 depending on the intimacy level. In terms of not placing a burden on the software, it is preferable not to use trigonometric functions, exponential functions, or logarithmic functions, and it is preferable to use only the four basic arithmetic operations. Furthermore, rather than changing the formulas for the above first, second, and third formulas for the three cases, we express them as one formula, and make the value of the differential coefficient equal at "x = any specified point", so that each formula can be smoothly connected. In this way, it is possible to simplify the program. If the coefficients are determined by the number of times (number of days) the user has fed JUNO in each of the three cases of low intimacy, medium intimacy, and high intimacy, all the coefficients can be determined by simply entering the three parameters 30/30/30, such as 30 days/30 days/30 days. This is to make debugging easier.

Next, the probability of JUNO moving to each area according to the intimacy level, calculated by the four arithmetic operations described with reference to Figs. 58A to 62, is graphed and tabulated as shown in Figs. 63 and 64. Fig. 63 is a graph showing the relationship between the passage of time, and thus the number of times (number of days) that a user has fed JUNO, and the probability of movement to each area, in a system 1 according to one embodiment. Fig. 64 is an example of a table showing the data of the graph shown in Fig. 63, i.e., the number of times (number of days) that a user has fed JUNO and the data of the probability of movement to each area, for each level of intimacy (low intimacy, medium intimacy, high intimacy) in a system 1 according to one embodiment. Note that the 0 to 30 shown in the vertical column of "number of days" for each level of intimacy in Fig. 64 is not the total number of days that a user has fed JUNO, but the number of times (number of days) that a user has fed JUNO at each level of intimacy. For example, if the "number of days" in the intimacy level is "20", the number of times (number of days) the user has fed Juno during the intimacy level is 20 days, and the total number of times (total number of days) the user has fed Juno is 50 days. Below, the amount of change in the probability of moving to each area depending on the number of times (number of days) the user has fed Juno will be explained with reference to Figures 63 and 64.

(Change in probability of moving to each area when intimacy is low)
As shown in FIG. 63 and FIG. 64, in the low intimacy level where the number of times (number of days) the user fed JUNO is 0 to 30 days, the change in the probability of moving to areas 1 to 4 relative to the number of times (number of days) the user fed JUNO increases in the positive direction as the number of times (number of days) the user fed JUNO increases (hereinafter, the change in the positive direction is referred to as the "positive change"). However, the positive change in the probability of moving to areas 1 to 4 relative to the number of times (number of days) the user fed JUNO is largest in the order of area 4, area 3, area 2, and area 1 among areas 1 to 4. In other words, among the positive change in the probability of moving to areas 1 to 4 relative to the number of times (number of days) the user fed JUNO, the positive change in the probability of moving to area 1 is the smallest. On the other hand, the change in the probability of moving to area 5 relative to the number of times (number of days) the user fed JUNO increases in the negative direction as the number of times (number of days) the user fed JUNO increases (hereinafter, the change in the positive direction is referred to as the "negative change"). In this way, at low intimacy, as the number of times (number of days) the user feeds JUNO increases, the positive change increases in the order of area 4, area 3, area 2, and area 1, while the negative change increases in the probability of moving to area 5. The probability of moving to each area at low intimacy is highest in the order of area 5, followed by area 4, area 3, area 2, and area 1. In this way, the position at which JUNO is displayed allows the user to embody the fact that JUNO is not used to the user in the initial state and is not attached to the user. Then, the character is not used to the user for a while from the initial state, and as the number of times (number of days) the user feeds JUNO increases, the user can embody the process of the character becoming less wary and becoming used to the user through the ratio, i.e., the proportion, of the positions at which the character stays.

(Change in probability of moving to each area when intimacy is high)
Next, when the number of times (number of days) the user has fed JUNO is between 30 and 60 days of intimacy, the amount of change in the probability of moving to areas 1 to 3 relative to the number of times (number of days) the user has fed JUNO increases in the positive direction as the number of times (number of days) the user has fed JUNO increases. However, unlike the case of low intimacy, the amount of positive change in the probability of moving to areas 1 to 3 relative to the number of times (number of days) the user has fed JUNO is larger in the order of area 1, area 2, and area 3 among areas 1 to 3. In other words, among the amount of positive change in the probability of moving to areas 1 to 3 relative to the number of times (number of days) the user has fed JUNO, the amount of positive change in the probability of moving to area 1 is the largest. Also, the amount of change in the probability of moving to area 4 relative to the number of times (number of days) the user has fed JUNO is 0. Furthermore, the amount of change in the probability of moving to area 5 relative to the number of times (number of days) the user has fed JUNO is still larger in the negative direction. The probability of moving to each area during the intimacy level is highest in the following order from the first half to the middle of the intimacy level: the probability of moving to area 5, the probability of moving to area 4, the probability of moving to area 3, the probability of moving to area 2, and the probability of moving to area 1, and highest in the second half of the intimacy level: the probability of moving to area 1, the probability of moving to area 4, the probability of moving to area 3, the probability of moving to area 2, and the probability of moving to area 5. As the number of times (number of days) that the user feeds JUNO increases, the difference between the probability of moving to area 2, the probability of moving to area 3, and the probability of moving to area 4 becomes smaller, and at the boundary between the medium intimacy level and the high intimacy level (for example, the number of days that the user has fed JUNO is 60 days), the probability of moving to area 2, the probability of moving to area 3, and the probability of moving to area 4 become the same or almost the same (for example, 0.20 in this embodiment).

(Change in probability of moving to each area when intimacy is high)
Next, at a high level of intimacy where the number of times (number of days) the user has fed JUNO is 60 days or more, the change in the probability of moving to areas 1 and 2 relative to the number of times (number of days) the user has fed JUNO increases in the positive direction as the number of times (number of days) the user has fed JUNO increases. However, the positive change in the probability of moving to areas 1 and 2 relative to the number of times (number of days) the user has fed JUNO decreases as the number of times (number of days) the user has fed JUNO increases. Also, the change in the probability of moving to areas 3 to 5 relative to the number of times (number of days) the user has fed JUNO increases in the negative direction as the number of times (number of days) the user has fed JUNO increases, and the negative change in the probability of moving to areas 3 to 5 relative to the number of times (number of days) the user has fed JUNO decreases as the number of times (number of days) the user has fed JUNO increases. In other words, at a high level of intimacy, the rate of change in the probability of moving to any of areas 1 to 5 slows down as the number of times (number of days) the user has fed JUNO increases. In particular, the graph showing the probability of moving to Area 5 approaches the asymptote of y = 1. By doing so, Juno's movements can be made more realistic. The probability of moving to each area when the intimacy level is high is highest in the following order: Area 1, Area 2, Area 3, Area 4, and Area 5.

In this way, by changing the amount of change in the probability of movement to each of areas 1 to 5 depending on the number of times (number of days) the user has fed JUNO, it is possible to control JUNO's behavioral patterns to change depending on whether the intimacy level is low, medium, or high. In this way, it is possible to create a virtual instinct function in which JUNO as character 1200 acts as if it were acting instinctively.

<11. Functions of this system>
Next, the functions (eg, state, behavior pattern, etc.) of the character 1200 (in this embodiment, JUNO modeled after a cat) according to the intimacy level (low intimacy level, medium intimacy level, high intimacy level) will be described.

In the system 1 according to the present embodiment, JUNO as the character 1200 is displayed in a behavioral pattern as if he has feelings according to the intimacy level described above. For example, at the initial setting, JUNO stays in the position farthest from the user (for example, area 5). Also, when the intimacy level is low, including at the initial setting, the character 1200 is wary (hereinafter referred to as "virtual wariness"). Therefore, at a low intimacy level, as described above, the processing board 224 controls the behavioral pattern so that JUNO does not approach the user (for example, does not approach area 1) as if he does not trust the user, and JUNO often stays in, for example, area 5, away from the user. Also, the processing board 224 controls the behavioral pattern so that, for example, JUNO recognizes area 5 as his territory is displayed. Also, even when JUNO is displayed in areas 1 to 4, the processing board 224 controls the behavioral pattern so that JUNO has a virtual wariness is displayed.

(11-1. Low intimacy level)
Fig. 65A is an example of a diagram for explaining the state and behavior pattern of JUNO when the intimacy level is low in the system 1 according to one embodiment. Also, Fig. 65B is an example of an image displayed in the video display space S1 of the system 1 when the intimacy level is low in the system 1 according to one embodiment. Note that hereinafter in this specification, the expression of the character 1200 will be unified into the description of JUNO.

As shown in FIG. 65A, a fixed location is set as JUNO's course of action. When the intimacy level is low, JUNO's fixed location is, for example, area 5, which is the farthest from the user, and JUNO is mainly in area 5. JUNO is also in area 5 regardless of whether or not there is human detection, which will be described later. JUNO is mainly located in area 5, and is on guard when moving to or in other areas.

As shown in FIG. 65B, a food bowl is displayed in area 1. A claw-sharpening board is displayed in area 3. Furthermore, JUNO is displayed in area 5. Area 5 is an area that is not subject to external influences. External influences include human detection, which will be described later. The food bowl and claw-sharpening board will also be described later.

Figure 66A is an example of an explanatory diagram for moving to another area when the intimacy level is low in a system 1 according to an embodiment. Figure 66B is an example image for explaining moving to another area when the intimacy level is low in a system 1 according to an embodiment.

The processing board 224, for example, performs a movement lottery at a predetermined cycle, and JUNO may move to another area based on the result of this movement lottery, or JUNO may move to another area based on event detection.

As shown in Figures 66A and 66B, when JUNO moves to another area based on the result of a movement lottery held at a predetermined cycle, the movement lottery held at the predetermined cycle is held, for example, at the movement probability shown in Figures 63 and 64 described above. The above-mentioned predetermined cycle is held, for example, at a cycle of about 1 to 10 minutes. Furthermore, when JUNO who won the movement lottery moves to another area, the processing board 224 decides by lottery the cycle for holding the movement lottery after the movement. In this case, the areas to which JUNO can move are, for example, any of areas 1 to 5. In this way, JUNO can move to an area based on the result of a movement lottery held at a predetermined cycle or on event detection. In other words, JUNO can move to an area at random timing, and JUNO can be displayed as if he is moving of his own volition.

Also, as shown in FIG. 66B, when intimacy is low, the probability that JUNO will move to Area 1 based on the results of the movement lottery is 0-5%. Similarly, the probability that JUNO will move to Area 2 is, for example, 0-10%, the probability that JUNO will move to Area 3 is, for example, 0-15%, the probability that JUNO will move to Area 4 is, for example, 0-20%, and the probability that JUNO will move to Area 5 is, for example, 100-50%.

Next, movement to other areas based on event detection will be described with reference to Figures 67A to 69B. Event detection includes, for example, detection of a feeding event, detection of a claw scratching event that may occur randomly, detection of a human detection event, etc.

(If a feeding event is detected)
Fig. 67A is an example of an explanatory diagram of movement to another area when a feeding event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 67B is an example image showing movement from area 5 to area 1 when a feeding event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 67C is an example image showing movement from area 1 or 2 to area 5 when a feeding event is detected when the intimacy level is low in a system 1 according to an embodiment.

Figure 67A describes, for example, the conditions for starting movement, the conditions for escaping to area 5, JUNO's behavior after escaping to area 5, and JUNO's behavior after finishing eating the food. In particular, when JUNO is escaping, by having it escape from area 1, which is closest to the user, to area 5, which is farthest from the user, it becomes possible to clearly understand that JUNO has moved away from the user.

When the user feeds (i.e., when food is generated), a feeding event is detected. When a feeding event is detected, the processing board 224 calculates the probability that JUNO will eat the food. Although Fig. 67A states "calculate the eating probability from the intimacy sheet", the intimacy sheet uses, for example, the probability of moving to Area 1 shown in Fig. 64. For example, if the number of times (number of days) that the user has fed JUNO is the 50th day, the processing board 224 determines whether or not to establish the movement start condition with a probability of 0.1500 (15%), which is the probability listed in the "20" column in the intimacy corresponding to the 50th day in the table shown in Fig. 64.

When the movement start condition is met, the processing board 224 decides that JUNO will start to take action to eat food. Then, when it is decided that JUNO will start to take action to eat food, the processing board 224 displays a behavior pattern in which JUNO starts to move vigilantly in order to eat food, and for example, as shown in FIG. 67B, JUNO moves from Area 5 to Area 1 where a food bowl is placed.

The condition for JUNO to run back to Area 5 from an area other than Area 5 (e.g. Area 1 or Area 2) may be, for example, an external detection (e.g., a proximity detection (i.e., a person is detected because a user approaches JUNO), a gesture detection, a call detection, or an external detection such as a loud sound of a certain decibel or more) is detected while JUNO is moving to Area 1 or while JUNO is eating food. For example, if the above-mentioned proximity detection occurs while JUNO is moving to Area 1 or while JUNO is eating food, the processing board 224 displays a behavior pattern in which JUNO moves to run back from an area other than Area 5 to Area 5, for example, as shown in FIG. 67C. In this way, by having JUNO move to run back to an area relatively far from the user, such as Area 5, when a proximity detection occurs, JUNO's movements can be made more realistic.

After JUNO escapes to Area 5, the processing board 224 plays a fear motion that indicates that JUNO is afraid, and then displays a behavior pattern as if it is wary of its surroundings, while starting a timer. After completing its movement to Area 5, the processing board 224 determines the period for conducting the above-mentioned movement lottery. The period for conducting the movement lottery may be, for example, about 1 to 10 minutes. Then, when the period for conducting the movement lottery arrives, the processing board 224 conducts the movement lottery.

On the other hand, when JUNO has finished eating the food, the processing board 224 displays a behavior pattern in which JUNO remains vigilant of its surroundings, and also conducts a movement lottery at a predetermined interval. If this movement lottery is won, the processing board 224 displays a behavior pattern in which JUNO moves to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is conducted, the processing board 224 displays a behavior pattern in which JUNO stays in Area 1.

(If a scratching event is detected)
Fig. 68A is an example of an explanatory diagram of movement to another area when a scratching event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 68B is an example image showing movement from area 5 to area 3 when a scratching event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 68C is an example image showing movement from area 5 to area 3 when a scratching event is detected when the intimacy level is low in a system 1 according to an embodiment.

Figure 68A shows, for example, the conditions for starting movement, the conditions for Juno to escape to Area 5, Juno's behavior after escaping to Area 5, Juno's behavior after scratching the claw post, etc.

The scratching event occurs randomly, for example. When a scratching event is detected, a movement start condition is met. The scratching event can be generated randomly, for example, by starting a timer based on an external detection (for example, when an external detection such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or the detection of a loud sound of a predetermined decibel or more is detected), and generating the event when no proximity detection is detected for a predetermined time (for example, five minutes). Note that the scratching event may be generated randomly, for example, by a lottery or the like.

When the movement start condition is met, the processing board 224 decides that JUNO will start the action of sharpening its claws. Then, when it is decided that JUNO will start the action of sharpening its claws, the processing board 224 displays a behavior pattern in which JUNO starts moving to sharpen its claws while remaining alert. For example, as shown in FIG. 68B, JUNO moves from area 5 to area 3 where the scratching board is located.

A condition for JUNO to run back to Area 5 may be met, for example, when an external detection (such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more) is detected while moving to Area 3 or while scratching the claws as shown in FIG. 68C. For example, when the above-mentioned proximity detection occurs while moving to Area 3 or while scratching the claws, the processing board 224 displays a behavior pattern in which JUNO moves to run back to Area 5.

After JUNO escapes to Area 5, the processing board 224 will play back a motion in which JUNO shows actions such as getting angry and turning away. Also, it is a good idea to prevent JUNO from scratching its claws on that day.

On the other hand, when JUNO finishes scratching the claws, the processing board 224 displays a behavior pattern of being alert to the surroundings and conducts a movement lottery at a predetermined interval. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is conducted, the processing board 224 displays a behavior pattern of JUNO staying in Area 3.

(When a human detection event is detected)
Fig. 69A is an example of an explanatory diagram for moving to another area when a human detection event is detected when the intimacy level is low in a system 1 according to an embodiment. Fig. 69B is an example image for explaining moving from area 1 or area 3 to area 5 when a human detection event is detected when the intimacy level is low in a system 1 according to an embodiment.

Figure 69A shows, for example, the conditions for starting movement, Juno's actions after returning to Area 5, and his actions after escaping to Area 5.

A human detection event occurs when JUNO is in any of areas 1 to 4 and, for example, a human is detected. When a human detection event is detected, a movement start condition is met. When the movement start condition is met, the processing board 224 displays a behavior pattern in which JUNO moves to area 5 (runs away). Human detection corresponds to, for example, external detection (such as proximity detection (i.e., when a human is detected because a user approaches JUNO), gesture detection, calling detection, or external detection such as detection of a loud sound above a certain decibel).

As shown in FIG. 69B, after JUNO returns to Area 5, the processing board 224 plays back motions that show actions such as getting angry or looking away, and also conducts a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in FIG. 63 and FIG. 64 above. Note that until the movement lottery is conducted, the processing board 224 displays a behavior pattern of JUNO staying in Area 5.

(Communication Function)
Next, a communication function of JUNO when the intimacy level is low will be described. Figures 70A and 70B are examples of diagrams for explaining a communication function of JUNO when the intimacy level is low in the system 1 according to an embodiment.

As shown in Figure 70A, Juno does not respond to calls from the user. Also, when the user calls out to Juno and Juno is in areas 1 to 4, Juno will move to area 5 (run away). Note that when Juno moves to area 5, the motion that is played will differ depending on the situation, etc.

As shown in FIG. 70B, when the intimacy level is low, JUNO will not speak spontaneously. Area movement is based on the results of a movement lottery (period determination) that is performed at a predetermined cycle. In addition, the home appliance control function, alarm function, and timer function described below cannot be used.

Below, we will briefly explain, with reference to Figure 70A, an example of Juno's motion that is played when Juno moves from any of areas 1 to 4 to area 5 when the intimacy level is low.

If an event (e.g., a call) is detected while JUNO is waiting in any of areas 1 to 4 (i.e., while waiting motion is being played), the processing board 224 displays JUNO's behavior pattern in the following order: interrupt the waiting motion, play a motion in which the ears respond, play a 180-degree turn motion, and move (run, walk, etc.) toward area 5. Furthermore, when JUNO reaches area 5, the processing board 224 makes a 180-degree turn and starts the waiting motion, and as a behavioral derivative, turns only his head while walking and moves to area 5. Then, the processing board 224 displays a behavior pattern in which when JUNO reaches area 5, he makes a 180-degree turn and starts the waiting motion.

In addition, when JUNO is in one of areas 1 to 4, for example, if a movement start condition is met and an event (e.g., a call) is detected during playback of a short motion, such as immediately before moving to another area, the processing board 22 displays JUNO's behavior pattern in the following order: completion of short motion playback, playback of an ear response motion, playback of a 180-degree turn motion, and movement (running, walking, etc.) toward area 5. In addition, when JUNO reaches area 5, the processing board 224 makes a 180-degree turn and starts a standby motion, and as a behavioral derivative, while walking, it turns its head only and moves to area 5. The processing board 224 then displays a behavior pattern in which when JUNO reaches area 5, it makes a 180-degree turn and starts a standby motion.

In addition, for example, if a movement start condition is met while JUNO is in one of areas 1 to 4 and an event (e.g. a call) is detected while moving between areas, the processing board 224 stops the area movement, plays an ear reaction motion, plays a 180-degree turn motion, and displays JUNO's behavior pattern in the following order: movement (running, walking, etc.) toward area 5. In addition, when JUNO reaches area 5, the processing board 224 makes a 180-degree turn and starts a standby motion, and as a behavioral derivative, turns only his head while walking and moves to area 5. The processing board 22 then displays a behavior pattern in which, when JUNO reaches area 5, he makes a 180-degree turn and starts a standby motion.

Even if an event (such as a call) occurs while JUNO is in Area 5, the processing board 224 will display a behavior pattern indicating that JUNO is unresponsive.

At low intimacy, JUNO will not speak spontaneously. The processing board 224 also performs a movement lottery at a predetermined cycle, and displays JUNO's area movement based on the result of this movement lottery. In addition to the movement lottery performed at a predetermined cycle, the processing board 224 can also display a behavior pattern of moving to another area when the above-mentioned feeding event is detected (when the user feeds JUNO), when a randomly occurring scratching event is detected, or when a human detection event is detected (when a human is detected). At low intimacy, other functions described below (home appliance control function, alarm function, timer function) cannot be used.

In this way, the processing board 224 is able to display behavioral patterns as if JUNO had a virtual instinct function when the intimacy level is low.

(Other examples of motion in low density areas)
The motion of JUNO is as described above, but the motion of JUNO is not limited to the above example, and various motions may be reproduced. Examples of the reproduction motion of JUNO are shown in Figs. 71 to 76.

Figure 71 is a diagram showing an example of a motion played back when JUNO runs away when an event is detected in system 1 according to one embodiment. JUNO's specific actions are as shown in Figure 71, so a detailed description will be omitted.

Figure 72 is a diagram showing an example of a motion played back when JUNO turns and escapes while moving between areas in system 1 according to one embodiment. JUNO's specific movements are as shown in Figure 72, so a detailed description will be omitted.

Figure 73 is a diagram showing an example of a playback motion when JUNO turns and escapes while waiting in system 1 according to one embodiment. JUNO's specific movements are as shown in Figure 73, so a detailed description will be omitted.

Figure 74 is a diagram showing an example of a motion played back when an event is detected while JUNO is eating food in system 1 according to one embodiment. Specific operations of JUNO are as shown in Figure 74, so a detailed description will be omitted.

Figure 75 is a diagram showing an example of a motion played back when an event is detected while JUNO is scratching its claws in the system 1 according to one embodiment. The specific operation of JUNO is as shown in Figure 75, so a detailed description will be omitted.

Figure 76 is a diagram showing an example of a motion played back when JUNO runs away after an event is detected while sleeping in system 1 according to one embodiment. JUNO's specific actions are as shown in Figure 76, so a detailed description will be omitted.

Note that the examples of playback motions shown in Figures 71 to 76 should be used not only for low intimacy levels, but also for medium and high intimacy levels.

(11-2. In the case of intimacy level)
Fig. 77A is an example of a diagram for explaining Juno's state, behavior pattern, etc. when the intimacy level is high in the system 1 according to an embodiment. Fig. 77B is an example of an image displayed in the video display space S1 of the system 1 according to an embodiment when the intimacy level is high.

As shown in FIG. 77A, JUNO's state and emotions at medium intimacy are as follows. That is, JUNO is beginning to get used to the environment and understand where things are. Furthermore, the processing board 224 displays JUNO's behavioral pattern as if he is able to move freely within the area, regardless of whether or not he detects people. In other words, unlike in the case of low intimacy, there is no fixed position where JUNO recognizes Area 5 as his territory. Furthermore, at medium intimacy, JUNO will not run away based on the occurrence of an external event. However, because JUNO is still not used to people, the processing board 224 controls JUNO so that he is often seen to ignore calls from the user.

As shown in Figure 77A, JUNO's behavioral policy is set so that he does not remain in a fixed position. During the intimacy level, JUNO moves freely between areas 1 to 5 regardless of whether or not a person is detected, which is a behavioral pattern that indicates that he is gradually becoming accustomed to the area.

As shown in FIG. 77B, a food bowl is displayed in area 1, and a scratching board is displayed in area 3. During intimacy, JUNO can move freely between areas 1 and 5 without being detected by humans. Area 5 is an area that is not subject to outside influences.

Figure 78A is an example of an explanatory diagram for moving to another area when intimacy is at a high level in a system 1 according to an embodiment. Figure 78B is an example image for explaining moving to another area when intimacy is at a high level in a system 1 according to an embodiment.

As in the case of low intimacy, the processing board 224 may, for example, perform a movement lottery at a predetermined interval, and JUNO may move to another area based on the results of this movement lottery, or JUNO may move to another area based on event detection.

Also, as shown in Figures 78A and 78B, when JUNO moves to another area based on the results of a movement lottery held at a predetermined cycle, the movement lottery held at the predetermined cycle is held, for example, with the movement probabilities shown in Figures 63 and 64 as described above, just like when the intimacy level is low. Note that the above-mentioned predetermined cycle is held, for example, at a cycle of about 1 to 10 minutes. Also, when JUNO who won the movement lottery moves to another area, the processing board 224 holds a lottery to determine the cycle for holding the movement lottery after the movement. In this case, the areas to which JUNO can move are, for example, any of areas 1 to 5.

As shown in Figure 78B, when the intimacy level is medium, the probability that JUNO will move to Area 1 based on the results of the movement lottery is, for example, 5-30%. Similarly, the probability that JUNO will move to Area 2 is, for example, 10-20%, the probability that JUNO will move to Area 3 is, for example, 15-20%, the probability that JUNO will move to Area 4 is, for example, 20%, and the probability that JUNO will move to Area 5 is, for example, 50-10%.

Next, movement to other areas based on event detection will be described with reference to Figures 79A to 80C. As with low intimacy, event detection can include, for example, detection of a feeding event that can occur when the user feeds the robot, and detection of a scratching event that can occur randomly. Note that, during intimacy, as with low intimacy, it is advisable to prevent movement to other areas even if a human detection event is detected that can occur when a human is detected.

(If a feeding event is detected)
79A is an example of an explanatory diagram of movement to another area when a feeding event is detected during intimacy in a system 1 according to an embodiment. Fig. 79B is an example image showing movement from area 5 to area 1 when a feeding event is detected during intimacy in a system 1 according to an embodiment.

Figure 79A shows, for example, the conditions for starting movement, the conditions for escaping to Area 5, and Juno's behavior after finishing eating food.

When the user feeds (i.e., when food is generated), a feeding event is detected. When a feeding event is detected, the processing board 224 calculates the probability that JUNO will eat the food. Although it is written in Figure 79A that "the eating probability is calculated from the intimacy sheet", the intimacy sheet is the same as in the case of low intimacy, for example, the probability of moving to Area 1 shown in Figure 64 is used.

When the movement start condition is met, the processing board 224 decides that JUNO will start taking action to eat food. Then, when it is decided that JUNO will start taking action to eat food, the processing board 224 displays a behavior pattern in which JUNO starts moving while being vigilant in order to eat food, and for example, as shown in FIG. 79B, JUNO moves from Area 5 to Area 1 where a food bowl is placed. Note that, during intimacy, no conditions are set for escaping to Area 5. Also, JUNO will not escape even if an external event is detected.

When JUNO has finished eating the food, the processing board 224 displays a behavior pattern in which JUNO is alert to its surroundings, and also conducts a movement lottery at a predetermined interval. If this movement lottery is won, the processing board 224 displays a behavior pattern in which JUNO moves to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is conducted, the processing board 224 displays a behavior pattern in which JUNO stays in Area 1.

(If a scratching event is detected)
Fig. 80A is an example of an explanatory diagram of movement to another area when a scratching event is detected during intimacy in a system 1 according to an embodiment. Fig. 80B is an example of an image showing movement from area 5 to area 3 when a scratching event is detected during intimacy in a system 1 according to an embodiment.

Figure 80A describes, for example, the conditions for starting movement, the conditions for Juno to escape to Area 5, Juno's behavior after scratching the claw post, etc.

As with low intimacy, scratching events occur, for example, randomly. When a scratching event is detected, a movement start condition is met. A scratching event can be generated randomly, for example, by starting a timer based on an external detection (for example, when an external detection such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more is detected), and generating the event when no external detection is detected for a predetermined time (for example, 5 minutes). Note that scratching events may be generated randomly, for example, by drawing lots. Note that during intimacy, there is no condition set for escaping to Area 5. Also, JUNO will not escape even if an external event is detected.

When the movement start condition is met, the processing board 224 decides that JUNO will start the action of sharpening its claws. Then, when it is decided that JUNO will start the action of sharpening its claws, the processing board 224 displays a behavior pattern in which JUNO starts moving to sharpen its claws while remaining alert. For example, as shown in FIG. 80B, JUNO moves from area 5 to area 3 where the scratching board is located.

When JUNO finishes sharpening its claws, the processing board 224 will conduct a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 will display a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is conducted, the processing board 224 will display a behavior pattern of JUNO staying in Area 3.

(When a human detection event is detected)
FIG. 80C is an example of an explanatory diagram of movement to another area when a human detection event is detected when the intimacy level is at medium in a system 1 according to an embodiment.

As shown in FIG. 80C, even if a human detection event is detected when the intimacy level is medium, JUNO will not move to Area 5 (will not escape), unlike when the intimacy level is low.

(Communication Function)
Next, the communication function of Juno when the intimacy level is medium will be described with reference to Fig. 81A and Fig. 81B. Fig. 81A and Fig. 81B are examples of diagrams for explaining the communication function of Juno when the intimacy level is medium in the system 1 according to one embodiment.

During intimacy, Juno can respond to calls from the user. However, as shown in Figure 81A, Juno's response differs depending on the area he is in.

For example, if JUNO is in area 1, he will respond to the user's Japanese. Also, when the user calls out to him, JUNO will respond, for example, with motions only, without speaking. Also, if JUNO is in areas 2 to 4, JUN will respond, for example, with motions.

When the user calls out, the processing board 224 performs a response lottery. The probability that JUNO will respond to the user's call is set according to the area in which JUNO is staying, as shown in FIG. 81B, for example. If JUNO is in area 1, for example, the probability of being in area 1 (i.e., the probability of moving to area 1 shown in FIG. 64, which is a 5-30% probability) is used, and if JUNO is in area 2, for example, the probability of being in area 2 (i.e., the probability of moving to area 2 shown in FIG. 64, which is a 10-20% probability) is used. Note that if the response lottery is lost, the processing board 224 displays a behavior pattern in which JUNO will ignore the user's call.

When the above-mentioned motion response is made, the processing board 224 displays a behavior pattern in which JUNO responds, for example, with its ears. For example, as shown in FIG. 81B, when JUNO is in areas 1 to 4, the processing board 224 displays a behavior pattern in which JUNO responds to a call from the user by twitching its ears.

Even if an event (such as a call) occurs while JUNO is in Area 5, the processing board 224 will display a behavior pattern indicating that JUNO is unresponsive.

(Other, Summary)
Next, other overviews of intimacy will be described with reference to Fig. 82. Fig. 82 is an example of a diagram for explaining other states and actions of JUNO when intimacy is in progress in the system 1 according to one embodiment.

As shown in FIG. 82, JUNO will not spontaneously speak during intimacy. The processing board 224 also performs a movement lottery at a predetermined cycle, and displays JUNO's area movement based on the result of this movement lottery. In addition to the movement lottery performed at a predetermined cycle, the processing board 224 can also display a behavior pattern of moving to another area when the above-mentioned feeding event is detected (when the user feeds) or when a randomly occurring scratching event is detected. Note that during intimacy, other functions (home appliance control function, alarm function, timer function) described below cannot be used.

In this way, the processing board 224 is able to display behavioral patterns as if JUNO had a virtual instinct function during intimacy.

(11-3. High Intimacy)
Fig. 83A is an example of a diagram for explaining JUNO's state, behavior pattern, etc. when the intimacy level is high in the system 1 according to one embodiment. Fig. 83B is an example of an image displayed in the video display space S1 of the system 1 according to one embodiment when the intimacy level is high.

As shown in FIG. 83A, JUNO's state and feelings when the intimacy level is high are as follows. That is, JUNO trusts the owner (user) and is completely accustomed to the surrounding environment, and is no longer wild. The processing board 224 displays JUNO's behavior pattern as being not wary and mostly staying in the front of the display area. In this embodiment, as shown in FIG. 64 above, in the case of high intimacy, from 60 to 90 days (shown as 30 days from 0 to 30 in FIG. 64 for convenience), the ratio of JUNO being in area 1 or area 2 (i.e., the combined probability of moving to area 1 and the probability of moving to area 2) is, for example, 50% or more, and when the number of days the user has fed JUNO exceeds 90 days, the ratio of JUNO being in area 1 (i.e., the probability of moving to area 1) exceeds, for example, 50%. Also, at high intimacy level, JUNO trusts the owner (user) and begins to act proactively. However, the processing board 224 may set, for example, a whimsical coefficient, so that JUNO's reaction may be displayed as, for example, low intimacy or low intimacy.

As shown in FIG. 83A, a fixed location is set as JUNO's course of action. At low intimacy, JUNO's fixed location is, for example, area 1, which is closest to the user, and JUNO is most likely to be in area 1 among areas 1 to 5. JUNO is mainly in area 1, but even if JUNO is in an area other than area 1, it moves to area 1 in response to human detection. In this way, by having JUNO move to area 1, which is closest to the user, it is possible to embody the process of JUNO getting used to the user and becoming attached to the user. In particular, when moving from area 5, which is farthest from the user, to area 1, which is farthest from the user, JUNO's movement distance is large, and it is possible to clearly grasp that JUNO is getting closer to the user, i.e., that JUNO is becoming attached to the user.

As shown in FIG. 83B, a food bowl and JUNO are displayed in area 1, and a scratching board is displayed in area 3. Area 5 is an area that is not subject to external influences.

Figure 84A is an example of an explanatory diagram for moving to another area when the intimacy level is high in a system 1 according to one embodiment. Figure 84B is an example image for explaining moving to another area when the intimacy level is high in a system 1 according to one embodiment.

Also, as in the cases of low and medium intimacy, the processing board 224 may, for example, perform a movement lottery at a predetermined cycle, and JUNO may move to another area based on the results of this movement lottery, or JUNO may move to another area based on event detection.

Also, as shown in Figures 84A and 84B, when JUNO moves to another area based on the results of a movement lottery held at a predetermined cycle, the movement lottery held at the predetermined cycle is held, for example, with the movement probabilities shown in Figures 63 and 64 as in the cases of low and medium intimacy. Note that the above-mentioned predetermined cycle is held, for example, at a cycle of about 1 to 10 minutes. Also, when JUNO who won the movement lottery moves to another area, the processing board 224 decides by lottery the cycle for holding the movement lottery after the movement. In this case, the areas to which JUNO can move are, for example, any of areas 1 to 5.

As shown in Figure 84B, when intimacy is high, the probability that JUNO will move to Area 1 based on the results of the movement lottery is, for example, 30-50%. Similarly, the probability that JUNO will move to Area 2 is, for example, 20-30%, the probability that JUNO will move to Area 3 is, for example, 20-10%, the probability that JUNO will move to Area 4 is, for example, 20-7%, and the probability that JUNO will move to Area 5 is, for example, 10-3%.

Next, we will explain movement to other areas based on event detection. Event detection includes, for example, feeding event detection that can occur when the user feeds the robot, scratching event detection that can occur randomly, human detection event detection that can occur when a human is detected, etc.

(If a feeding event is detected)
85A is an example of an explanatory diagram of a movement to another area when a feeding event is detected when the intimacy level is high in the system 1 according to an embodiment. Fig. 85B is an example of an image showing an example when a feeding event is detected when the intimacy level is high in the system 1 according to an embodiment.

Figure 85A shows, for example, the conditions for starting movement, the conditions for escaping to Area 5, and Juno's behavior after finishing eating food.

When the user feeds (i.e., when food is generated), a feeding event is detected. When a feeding event is detected, the processing board 224 calculates the probability that JUNO will eat the food. Although it is written in Figure 85A that "the eating probability is calculated from the intimacy sheet", the intimacy sheet is the same as in the cases of low intimacy and medium intimacy, for example, the probability of moving to Area 1 shown in Figure 64 is used.

When the movement start condition is met, the processing board 224 decides that JUNO will start taking action to eat food. Then, when it is decided that JUNO will start taking action to eat food, the processing board 224 displays a behavior pattern in which JUNO starts moving while being vigilant in order to eat food, and for example, as shown in FIG. 85B, JUNO moves from the area it is currently in to Area 1 where a food bowl is placed. However, at high intimacy, JUNO is often in Area 1, and if a feeding event occurs while in Area 1, it will start eating food in Area 1. Note that at high intimacy, there are no conditions set for escaping to Area 5. Also, JUNO will not escape even if an external event is detected.

After JUNO has finished eating the food, the processing board 224 will hold a movement lottery at a predetermined interval. If this movement lottery is won, the processing board 224 will display a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is held, the processing board 224 will display a behavior pattern of JUNO staying in Area 1.

(If a scratching event is detected)
Fig. 86A is an example of an explanatory diagram of movement to another area when a scratching event is detected when the intimacy level is high in a system 1 according to an embodiment. Fig. 86B is an example of an image showing movement from area 1 to area 3 when a scratching event is detected when the intimacy level is high in a system 1 according to an embodiment.

Figure 86A describes, for example, the conditions for starting movement, the conditions for Juno to escape to Area 5, Juno's behavior after scratching the claw post, etc.

As with low and medium intimacy, scratching events occur randomly, for example. When a scratching event is detected, a movement start condition is met. A scratching event can be generated randomly, for example, by starting a timer based on an external detection (for example, when an external detection such as a proximity detection (i.e., when a person is detected because a user approaches JUNO), a gesture detection, a call detection, or a loud sound of a predetermined decibel or more is detected), and generating the event when no external detection is detected for a predetermined time (for example, five minutes). Note that scratching events may be generated randomly, for example, by a lottery. Note that at high intimacy, there is no condition set for escaping to Area 5. Also, JUNO will not escape even if an external event is detected.

When the movement start condition is met, the processing board 224 decides that JUNO will start the action of sharpening its claws. Then, when it is decided that JUNO will start the action of sharpening its claws, the processing board 224 displays a behavior pattern in which JUNO starts moving to sharpen its claws while remaining alert, and for example, as shown in FIG. 86B, JUNO moves from area 1 to area 3 where a scratching board is placed.

When JUNO finishes sharpening its claws, the processing board 224 will conduct a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 will display a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is conducted, the processing board 224 will display a behavior pattern of JUNO staying in Area 3.

(When a human detection event is detected)
87A is an example of an explanatory diagram for moving to another area when a human detection event is detected when the intimacy level is high in a system 1 according to an embodiment. Fig. 87B is an example of an image for explaining moving from an area other than area 1 to area 1 when a human detection event is detected when the intimacy level is high in a system 1 according to an embodiment.

Figure 87A shows, for example, the conditions for starting movement and Juno's actions after moving to Area 1.

As shown in Figures 87A and 87B, a human detection event occurs when JUNO is in any of areas 1 to 4, for example, when an external presence is detected. When a human detection event is detected, a movement start condition is met. When the movement start condition is met, the processing board 224 displays a behavior pattern in which JUNO moves (approaches) to area 1. External detection corresponds to proximity detection (i.e., when a person is detected because, for example, a user approaches JUNO), gesture detection, calling detection, or external detection such as detection of a loud sound above a certain decibel is detected), etc.

After JUNO moves to area 1, the processing board 224 performs a movement lottery at a predetermined cycle. If this movement lottery is won, the processing board 224 displays a behavior pattern of moving to another area. The destination area is determined based on the movement probabilities shown in Figures 63 and 64 above. Note that until the movement lottery is performed, the processing board 224 displays a behavior pattern of JUNO staying in area 1.

(Communication Function)
Next, the communication function of JUNO when the intimacy level is high will be described with reference to Fig. 88A and Fig. 88B. Fig. 88A is an example of a diagram for explaining a call response, which is an example of the communication function of JUNO when the intimacy level is high, in the system 1 according to one embodiment. Fig. 88B is an example of an image for explaining area movement when a call is made in the case of a high intimacy level, in the system 1 according to one embodiment.

As shown in FIG. 88A, when the response condition is met, the processing board 224 can be controlled so that JUNO responds in Japanese to a call from the user, for example. The processing board 224 can also be controlled so that the content of the response differs depending on the area where JUNO is located. The response condition corresponds to, for example, when a call from the user is detected.

As shown in FIG. 88A, the human detection event is such that, when JUNO is in any of areas 1 to 4, the processing board 224 can control JUNO to respond to a call from a user, for example, in Japanese or with a motion. Also, when JUNO is in areas 2 to 4, the processing board 224 can control JUNO to respond to a call from a user, for example, with a motion.

If JUNO is in Area 1 and has not yet mastered Japanese, the processing board 224 performs control as follows. That is, if an event such as a call from the user is detected, the processing board 224 controls JUNO to respond with a motion in response to the detected event.

Furthermore, if JUNO is in area 1 and has learned Japanese, the processing board 224 performs control as follows. That is, for example, when an event such as a call from a user is detected, if JUNO remembers the Japanese that is being called out to (i.e., if he has already learned the Japanese that is being called out to), the processing board 224 controls JUNO to respond in Japanese. On the other hand, if JUNO is in area 1 and has learned Japanese, but does not remember the Japanese that is being called out to (i.e., if he has not already learned the Japanese that is being called out to), the processing board 224 controls JUNO to respond with a motion.

In addition, when JUNO is in areas 2 to 4, if an event such as a call from the user is detected, the processing board 224 controls JUNO to respond with a motion in response to the detected event.

In addition, when JUNO is in Area 5, even if an event such as a call is detected, the processing board 224 controls JUNO so that it does not respond. As mentioned above, Area 5 is an area that is not subject to external influences.

In this embodiment, even if JUNO learns Japanese and becomes able to speak, it is controlled so that it will not speak spontaneously. Therefore, unless an event such as the user calling out is detected, JUNO is controlled so that it will not speak to the user. However, this is not limited to this, and even if an event such as a user calling out is not detected, for example, if an event such as human detection occurs, the processing board 224 may control JUNO to speak spontaneously.

Also, as shown in FIG. 88B, when Juno is called and the intimacy level is high, there is a 30-50% chance that he will move to Area 1, a 20-30% chance that he will move to Area 2, a 20-10% chance that he will move to Area 3, a 20-7% chance that he will move to Area 4, and a 10-3% chance that he will move to Area 5. Note that the above probabilities include not only the chance that he will move, but also the chance that he will stay in the same place as before.

In this embodiment, the explanation is given on the assumption that the language that JUNO (character) will learn is Japanese, but the language to be learned is not limited to Japanese, and may be other languages such as English, Chinese, French, German, and Korean, or multiple languages may be learnable. It is also preferable to learn the language spoken by the user.

(summary)
Next, a summary of Juno's actions when the intimacy level is high will be described with reference to Fig. 89. Fig. 89 is an example of a diagram for explaining a summary of Juno's actions when the intimacy level is high in the system 1 according to one embodiment.

As described above, the processing board 224 performs a movement lottery at a predetermined cycle and displays JUNO's area movement based on the result of this movement lottery. In addition to the movement lottery performed at a predetermined cycle, the processing board 224 can also display a behavior pattern of moving to another area when the above-mentioned feeding event, scratching event, or human detection event is detected.

In this way, when the intimacy level is high, the processing board 224 is able to display behavioral patterns as if JUNO had a virtual instinct function.

<12. Functions unique to high intimacy>
By the way, there are functions unique to the high intimacy level that are not available to the low intimacy level and the medium intimacy level. The functions unique to the high intimacy level will be described below with reference to Figures 90A, 90B, 90C, 90D, and 91.

Figure 90A is an example diagram for explaining functions specific to high intimacy in system 1 according to one embodiment.

As shown in FIG. 90A, functions specific to a high level of intimacy include, for example, home appliance control functions, alarm functions, and timer functions.

The alarm function and timer function are similar functions, but in this embodiment, from the viewpoint of reliably informing the user of the passage of time, the alarm function is a function that notifies the user of the passage of time multiple times (e.g., five times), and the timer function is a function that notifies the user of the passage of time only once.

As shown in FIG. 90A, the ROM 702 of the processing board 224 stores a number of call word groups in which a number of call words having the same function are stored, and JUNO response words corresponding to the call word groups. The JUNO response words store content confirmation words that confirm the content, and implementation words that inform the user that implementation (or time has passed).

In this embodiment, multiple call word groups having the same function are stored, such as, but not limited to, a timer call word group, an alarm call word group, and a home appliance control call word group.

Figure 90B is an example of a diagram for explaining functions specific to high intimacy levels in system 1 according to one embodiment, showing an example of a group of call words and JUNO response words (character response words) for the timer function.

As shown in FIG. 90B, for example, a plurality of call words such as "timer for ** minutes," "set timer for ** minutes," and "set timer in ** minutes" are stored in the call word group for the timer function. In addition, for example, a content confirmation word such as "I'll set the timer for ** minutes" and an implementation word such as "The time's up" are stored as JUNO response words corresponding to the call word group for the timer function.

Figure 90C is an example of a diagram for explaining functions specific to high intimacy in system 1 according to one embodiment, and shows an example of a group of call words and JUNO response words (character response words) for the alarm function.

As shown in FIG. 90C, for example, a plurality of call words such as "Alarm at ** minutes," "Set alarm at ** minutes," and "Set alarm at ** minutes" are stored in the call word group for the alarm function. In addition, for example, a content confirmation word such as "I'll set the alarm in ** hours and ** minutes" and an implementation word that repeats "It's time" five times are stored as JUNO response words corresponding to the call word group for the alarm function.

Figure 90D is an example of a diagram for explaining functions specific to high intimacy in system 1 according to one embodiment, and shows an example of a group of call words and JUNO response words (character response words) for home appliance control functions.

As shown in FIG. 90D, for example, a plurality of call words such as "Turn on the TV," "Turn on the TV," and "TV on" are stored in the call word group for the home appliance control function. In addition, for example, content confirmation words such as "I'll turn on XX/I'll turn off XX" and implementation words such as "I tried it" are stored as JUNO response words corresponding to the call word group for the home appliance control function.

Returning to FIG. 90A, when an external event such as a call from the user is detected, the processing board 224 determines whether the called word is stored in any of the call word groups. If the called word is stored in any of the call word groups, the processing board 224 controls JUNO to issue a JUNO response word (content confirmation word) corresponding to the call word group in which the called word is stored. The processing board 224 then performs the content corresponding to the call word and controls JUNO to issue a response word (implementation word).

For example, when a call from a user is detected and the called word is a setting word for a home appliance control function, the processing board 224 judges whether the called word has been stored, and if the called word has been stored, controls JUNO to utter a content confirmation word such as "I'll turn on XX" or "I'll remove XX", and then carries out that content, and then controls JUNO to utter "I tried it".

Figure 91 is an example diagram for explaining the timer function and alarm function in system 1 according to an embodiment. As shown in Figure 91, if a timer or alarm has been set, the information can be cancelled by voice. For example, when a call word such as "cancel alarm" is detected, the processing board 224 controls JUNO to say "I'll cancel the alarm," and also cancels the content so that it is not carried out.

In addition, if a timer or alarm is already set and an additional timer or alarm is set, it will be overwritten. For example, if the user says words such as "Alarm at 3:15," JUNO will say "I'll set the alarm at 3:15," overwriting the previous setting. Note that only one timer and one alarm can be set in total, but this is not limited to the above, and it may be possible to set one timer and one alarm each, or to set multiple timers and alarms at least.

<13. Japanese language learning function>
The Japanese language learning function of JUNO will be described with reference to Figures 92A to 101D. The Japanese language learning function is a function that is unlocked during intimacy (i.e., activated during intimacy), for example.

Figures 92A and 92B are diagrams illustrating an example of how a character learns Japanese in system 1 according to one embodiment.

As shown in FIG. 92A, JUNO can learn Japanese by having the user persistently speak to him in Japanese. It is advisable to set a daily upper limit (described below) for when he is spoken to in Japanese, and furthermore, to reduce the daily increase upper limit for how much Japanese he can learn on days when he is not spoken to in Japanese. In this way, JUNO will not be able to learn Japanese unless he studies it every day. Also, even if he does learn it once, if several days pass without speaking to JUNO, JUNO will no longer respond in Japanese.

When JUNO is in Area 1 and is called out by a user, the processing board 224 counts the number of times he is called out. However, there is a limit to how much the count can increase per day (the increase limit mentioned above).

As shown in FIG. 92B, before mastering Japanese, the robot passes through several states, such as not understanding Japanese, trying to learn Japanese, and speaking Japanese. When in the state of not understanding Japanese or trying to learn Japanese, the robot will only respond with motions, for example, even if called upon by the user. When speaking Japanese, the robot will, for example, speak in Japanese.

(13-1. When you don't understand Japanese)
When the user does not understand Japanese, if the words spoken by the user include a registered word (described later) that has been registered in advance in the ROM 702 of the processing board 224, the number of times that group, described later, is counted up. For example, if the user says "Good morning," the number of times the "Good morning group" is counted up by one. This number of times that the user has spoken is stored in the character-specific saved information, described later.

When a user speaks a word that has not been preregistered in the ROM 702 of the processing board 224, JUNO will not react. For example, even if a user speaks to JUNO saying "Good morning," JUNO will not react if "Good morning" is not a registered word. In addition, the number of times the user has spoken to JUNO will not be counted.

For example, it is not necessary to set the number of times a registered word (described later) is spoken by the user or the upper limit of increase per day. Also, depending on the settings and other circumstances, the number of times until Japanese is memorized may be changed.

Next, as mentioned above, JUNO's responses to calls at each level of intimacy will be explained again. Figure 93A is an example of a diagram for explaining the response of JUNO (character 1200) at a low level of intimacy in a system 1 according to one embodiment. Figure 93B is an example of a diagram for explaining the response of JUNO (character) at a medium level of intimacy in a system 1 according to one embodiment. Figure 93C is an example of a diagram for explaining the response of JUNO (character) at a high level of intimacy in a system 1 according to one embodiment.

As shown in Figure 93A, when the intimacy level is low, JUNO will make a motion to run away to Area 5, for example, when called by the user.

As shown in Figure 93B, if Juno is in Areas 1 to 4 and is at a medium level of intimacy, he will respond with a motion if the user calls out to him.

As shown in FIG. 93C, in the case of high intimacy, if JUNO is called by the user while in Areas 2 to 4, he will respond with motions. On the other hand, if the user calls out to him while he is in Area 1, he will respond in Japanese, such as "Good morning." However, he will only respond in Japanese if he is called out using a registered word that he has memorized. If he is called out using a registered word that he has not memorized, he will only respond with motions.

Next, we will explain the recognition words. Figure 94 is an example of a diagram for explaining the recognition words, particularly the group of parrot-like words, in system 1 according to one embodiment.

As shown in FIG. 94, the recognition words include, for example, customary greetings, dialogue words, and echo words. FIG. 94 also shows a echo word table, which is stored in the ROM 702 of the processing board 224.

As shown in FIG. 94, the echo word group table contains JUNO's response words that return the same message as the user's call, such as "Good morning" or "Good evening." For example, if the user says "Good morning" to JUNO, JUNO will echo (i.e., repeat) "Good morning." In this way, JUNO repeating the user's call can give the user a sense of closeness to JUNO. In particular, having JUNO repeat what the user says in area 1, which is close to the user, can give the user an even greater sense of closeness to JUNO. Moreover, because JUNO only repeats when the intimacy level is high, the bond between the user and JUNO can be further deepened.

In addition, the call word group in the echo word group table includes a number of word groups, such as the "good morning group," "good afternoon group," "good evening group," and "good night group." The "good morning group" includes call words (registered words), such as "good morning." The "hello group" includes registered words, such as "hello." The "good evening group" includes call words (registered words), such as "good evening." The "good night group" includes call words (registered words), such as "good night."

Figure 95 is an example of a diagram for explaining dialogue-related word groups in system 1 according to an embodiment. Figure 95 shows a dialogue-related word group table consisting of word group names, call-out words, and JUNO response words. The dialogue-related word group table is stored, for example, in ROM 702 of the processing board 224. Note that Figure 95 shows only a portion of the dialogue-related word group table, and details of the dialogue-related word group table are shown in Figures 96A to 97E.

Figures 96A and 96B are diagrams showing examples of word group names and call word groups in the dialogue word group table shown in Figure 95. Note that the word group names and call word groups are linked.

Figures 97A and 97B are diagrams showing examples of word group names and JUNO response words (character response words) in the first, second, and third stages in the dialogue word group table shown in Figure 95. Note that the word group names and JUNO response words are linked.

Figures 97C to 97E are diagrams showing examples of word group names and JUNO response words (character response words) in the fourth stage in the dialogue word group table shown in Figure 95. Note that the word group names and JUNO response words are linked.

As shown in FIG. 95, JUNO's response words are set in multiple stages (for example, stage 1 to stage 4) in response to words called out by the user. The called out words are counted up, and when a predetermined number of times is reached, the stage increases. The stages are ranked in the order of stage 1, stage 2, stage 3, and stage 4, for example. For example, in stage 1, even if the user calls out to JUNO as "JUNO," JUNO will simply respond with "Hmm?".

Although not shown in the drawings, the dialogue word group table contains address words and dictionary registration words. For example, the address words include words that are spoken in the hope that JUNO will respond (address words). Also, for example, the dictionary registration words include words that are spoken in the hope that JUNO will reply (dictionary registration words).

The group of call words corresponds to, for example, the word groups "Yuno (1)", "Yuno (2)", "Question", and "Call" shown in FIG. 96A. Registered words such as "Yuno" are set for "Yuno (1)". Registered words such as "Kitty" are set for "Yuno (2)". Registered words such as "Hello" are set for "Question". Registered words such as "Come on over" are set for "Call".

The dictionary registered word groups include various groups of words that are spoken in the hope that JUNO will respond, such as the "I'm leaving group" shown in FIG. 96A, the "hungry group" and the "meal time group" shown in FIG. 96B. The "I'm leaving group" includes registered words such as "I'm going to the convenience store." The "hungry group" includes registered words such as "I'm hungry." The "meal time group" includes registered words such as "It's meal time."

As shown in Fig. 97C to Fig. 97E, the conversation word group table includes the address word group and the dictionary registration word group, and the response words of JUNO associated with each word group are set. As described above, JUNO's response words are divided into the first, second, third, and fourth stages. Compared to the first and second stages, many more emotional response words are set from the third stage onwards. JUNO's emotions include happy, sad, and angry. For example, a response word expressing JUNO's emotions when he is happy is set to "Oh, you're lonely." Also, for example, a word expressing JUNO's emotions when he is sad is set to "I'm right next to you, but can't you see me?" Also, for example, a response word expressing the emotions of the character 1200 when he is angry is set to "Please don't call me when I have no need to."

Details will be given later, but the first, second, third, and fourth stages of JUNO's response words are unlocked in sequence as JUNO learns Japanese. After unlocking the first stage of Japanese language learning, the number of times each call word in the dialogue word group table is used will begin to be recorded. When the number of times each call word in the dialogue word group table is used exceeds a specified number of times, the response words will be unlocked in stages. Then, when the fourth stage is unlocked, all registered information will be unlocked.

(13-2. Trying to learn a language (e.g. Japanese))
Next, a state in which a language (such as Japanese) is to be learned will be described with reference to Figures 98A to 98C. Figures 98A to 98C are examples of diagrams for explaining a state in which JUNO is trying to learn a language in system 1 according to one embodiment. In this example, the language that JUNO is trying to learn will be described as Japanese.

The number of times the user calls out in Japanese language learning starts to be counted from the intimacy level. On the other hand, responses (in Japanese) using learned words start from the high intimacy level. In other words, even if the user learns a word during the intimacy level, responses in Japanese will not start until the intimacy level is high.

As shown in FIG. 98A, when JUNO memorizes a group of parrot-like words, it does so in response to the user's prompts for registered words. For example, when JUNO is prompted for a registered word 20 times or more, it can memorize that word. For example, when prompted for "Good morning," JUNO will count the number of times "Good morning" has been said as one. The number of times this prompt has been received is stored in the character-specific saved information, which will be described later.

Also, when the intimacy level is high and Juno is in Area 1, if the user asks Juno about a word he has memorized, Juno will repeat it back to the user.

For example, if a user says "Good morning" to JUNO, the number of times "Good morning" has been said will be counted as one. Then, if the number of times "Good morning" has been said reaches, for example, 20 or more times, JUNO will repeat "Good morning." Note that if the user asks JUNO a word that JUNO does not remember, such as "Good morning," JUNO will assume that JUNO has a virtual feeling, such as "I don't know!", and will not respond in Japanese.

Also, as shown in Figure 98B, in the echo word group, the registered words that are called out are managed individually, and the number of times other registered words are called out is not updated. For example, if someone calls out "Good morning," the number of times "Good morning" has been called out is counted once. In that case, even if the registered word is from the same group, such as "Good morning," the number of times "Good morning" has been called out does not change. Also, even if no registered words are called out at all for an entire day, the number of times they have been called out does not decrease. For example, even if no one calls out for a week, the setting information does not change because it is saved in the character-specific saved information described below.

Also, as shown in FIG. 98C, when JUNO memorizes three or more registered words in the echo word group, the first stage of the dialogue word group is unlocked. After the first stage of the dialogue word group is unlocked, the number of response types increases depending on the number of times each individual word group is asked. For example, when each individual word group is asked a cumulative total of 20 times or more, each stage is unlocked.

Also, for example, when the user has asked a registered word 20 times, JUNO will memorize that word. For example, if the user has called out each registered word, such as "good morning," "hello," and "good night," 20 or more times, JUNO will have memorized three or more words. Then, the first stage of the dialogue word group is unlocked. When the first stage is unlocked, JUNO at a high level of intimacy will respond to the call words from the dialogue word group with response words associated with each word group (for example, see Figures 96A to 97E).

For example, if the user says "Yuno", Juno will respond with "What's that?". Also, if the user says "Come here", Juno will respond with "What's wrong?". And if the user says "I'm leaving", Juno will respond with "Have a nice day".

Figures 99A and 99B are diagrams showing an example of a response flow in system 1 according to one embodiment. When Juno is asked a question about a word he has learned while his intimacy level is high and he is in area 1, he will repeat it back to him.

As shown in FIG. 99A, for example, if the number of times "Good morning" has been asked is less than 20, JUNO will respond with motion only, regardless of the intimacy level, even if the user asks "Good morning." However, the number of times "Good morning" has been asked will be counted up.

Also, for example, even if the number of times "Good morning" has been asked is 20 or more, if the intimacy level is medium, JUNO will respond with only a motion even if the user asks "Good morning." In this case, the number of times "Good morning" has been asked will be counted up.

In contrast, if the user has said "Good morning" 20 or more times and the intimacy level is high, then when the user says "Good morning," JUNO will repeat "Good morning." Note that when the intimacy level is high, JUNO will always move to Area 1 when an external event occurs and will, for example, repeat the message.

Also, as shown in Figure 99B, if the intimacy level is high and JUNO has learned three or more words while in Area 1, JUNO will respond to questions from the dialogue word group with words from the first stage (see, for example, Figures 96A to 97B).

Although JUNO will react to the dialogue-type words, he will not respond in Japanese, and the number of times he has been asked will not be counted up. The first stage will be unlocked when JUNO has memorized three or more words from the echo-type words, and the number of times he has used the dialogue-type words will start to be recorded.

Figure 100 is an example diagram for explaining a case where a response is made in Japanese in the system 1 according to an embodiment. As shown in Figure 100, when the intimacy level is high and JUNO has learned three or more words from the group of parrot-like words, it will respond according to each level.

For example, as shown in FIG. 100, in the first stage, if the user asks "Yuno", JUNO will reply with a curt "Hmm?". In the second stage, if the user asks "Yuno", JUNO will reply normally with "Hmm?" and then "What?". In the third stage, if the user asks "Yuno", JUNO will reply in a friendly manner with "Hmm?", "What?" and then "Yes". In the fourth stage, if the user asks "Yuno", JUNO will reply in a friendly manner with "Hmm?", "What?" and "Yes", and all functions specific to high intimacy levels, for example those described below, will be unlocked.

In addition, each stage is unlocked depending on the number of times the dialogue-based call words are used. For example, if any of the dialogue-based call words are used 20 times or more, the second stage is unlocked. For example, if any of the dialogue-based call words are used 40 times or more, the third stage is unlocked. For example, if any of the dialogue-based call words are used 60 times or more, the fourth stage is unlocked.

When any word group is unlocked up to the fourth level, functions specific to high intimacy are unlocked. These functions include weather and news inquiries, home appliance control functions, timer functions, alarm functions, etc.

Figures 101A to 101D are examples of diagrams for explaining special call words in system 1 according to one embodiment. As shown in Figures 101A to 101D, special call words for setting the weather, news, home appliance control, timer, and alarm are set in ROM 702 of processing board 224. The home appliance control function, timer function, and alarm function are similar to those described above, so detailed explanations are omitted. For inquiries about the weather and news, "when (e.g., today)," "where (e.g., where)," "keyword (e.g., weather)," "verb (e.g., tell me)," etc. are set. A detailed explanation of the weather and news will be given later.

Juno will be able to speak Japanese once his intimacy level reaches high. If you move from high intimacy to medium intimacy, Juno will no longer be able to speak Japanese. Intimacy will decrease if you do not feed Juno for several days.

<14. Priority of response words>
The priority of response words in response to user calls will be described with reference to Figs. 102A to 105C. Figs. 102A to 102C are exemplary diagrams for explaining an overview of response word priorities in system 1 according to an embodiment. Fig. 103 is an exemplary diagram for explaining a call system and a dictionary registration word system in system 1 according to an embodiment. Figs. 104A to 105C are exemplary diagrams for explaining response word priorities in system 1 according to an embodiment.

As shown in Figures 102A to 102C, one set of words is determined as a response to one voice input (short sentence), but for multiple voice inputs (short sentences), the dialogue may become unnatural depending on which response is given. In this system 1, when one registered word is detected, a response word linked to that registered word is selected. For example, when the word "Yuno" is detected, the response word "Hmm?" linked to "Yuno" is selected. In that case, it is thought that there is no sense of incongruity.

However, if multiple registered words are detected, selecting all of the response words associated with those registered words will sound strange. For example, if the three words "JUNO", "Good morning", and "I'm hungry" are detected, the response words "Hmm?", "Good morning", and "I'm hungry", which are associated with "JUNO", "Good morning", and "I'm hungry", respectively, will be selected. In this way, creating and replying to all of the response sentences for the input voice will make the sentences themselves sound strange.

Therefore, we set priorities for the input words and narrow down the response words to, for example, one. By setting priorities for the response words that have been set, Juno's response will sound natural.

As shown in FIG. 103, the dialogue word group (see FIG. 96, 97) includes calling-out words and dictionary registration words. For example, calling-out words are set as words that are spoken in the hope that JUNO will respond. Also, for example, dictionary registration words are set as words that are spoken in the hope that JUNO will reply.

As shown in Figs. 104A to 105C, for example, if a word such as "JUNO" is detected as an address word, then a response word such as "Hmm?" is selected as the response word associated with the address word. Also, for example, if a word such as "I'm hungry" is detected as an address word, then a response word such as "I didn't eat just now" is selected as the response word associated with the dictionary registration word.

On the other hand, if multiple words are detected, a priority is set for the response words. In that case, dictionary registered words are given priority over address words. For example, if two words, "JUNO," which is an address word, and "I'm hungry," which is a dictionary registered word, are detected, the response word "I didn't eat just now," which is linked to the dictionary registration word, is selected.

In addition, if the same word family is duplicated, the registered word detected last will be given priority. For example, if two words, the greeting word "JUNO" and the greeting word "hey hey," are detected, the response word "what" linked to the greeting word "hey hey" detected last will be selected.

For example, if two words, the dictionary registered word "I'm hungry" and the dictionary registered word "It's time for dinner," are detected, the response word "Yuno wants to eat too," which is linked to the dictionary registered word "It's time for dinner" detected later, is selected.

Furthermore, three words may be detected. As shown in Figures 105A to 105C, for example, if three words from the same word family are detected, such as the greeting word "JUNO," the greeting word "Hey hey," and the greeting word "Hey," the response word "What's wrong?" linked to the last detected greeting word "Hey" is selected.

For example, if three words are detected from the same word family, such as the dictionary registration word "I'm hungry," the dictionary registration word "It's time for dinner," and the dictionary registration word "I'm going to the convenience store," the response word "See you, meow," which is linked to the last dictionary registration word detected, "I'm going to the convenience store," is selected.

For example, if three words are detected from different word systems, such as the dictionary registration word "I'm hungry," the calling word "JUNO," and the calling word "Hey, hey," the response word "I didn't eat just now," which is linked to the dictionary registration word "I'm hungry," which takes priority over the calling word, is selected.

In this way, by setting the priority of response words, the conversation between the user and JUNO becomes natural and unnatural. Note that the number of registered words that can be detected at one time is, for example, up to three words.

<15. Character-specific saved information>
The character-specific stored information will be described with reference to Fig. 106. Fig. 106 is an example of character-specific stored information in the system 1 according to an embodiment.

As explained above, in this system 1, when a user feeds a pet, the experience points related to increasing intimacy increase, but in reality, this does not increase the experience points on that day. The experience points increase as a result of feeding a pet after the system is updated the following day. In other words, information that food has been fed once is sent to the cloud server, which then tallys it up, and after the system is updated, this is reflected in system 1 as experience points.

As shown in FIG. 106, not only information that the animal has been fed, but also information that the animal has not been fed is sent from the system 1 to the cloud server. If the animal has not been fed for several days (e.g., three days), the experience points are deducted.

In addition, information related to calls is also stored in the cloud server. The information stored includes the number of times each registered word has been called out, the number of times each registered word group has been called out, the Japanese language acquisition status and number of responses for each registered word group, etc. Note that updates such as intimacy level and learning of Japanese are made, for example, once a day.

<16. Feeding>
Feeding JUNO will be described with reference to Figures 107A to 107C. Figures 107A to 107C are examples of diagrams for explaining feeding in system 1 according to one embodiment.

As shown in Figs. 107A to 107C, there are several ways to feed JUNO, including feeding by voice, feeding by an application on an external terminal (e.g., a smartphone, etc.) described below, and feeding by a function button. Of these multiple feeding methods, the user can feed JUNO in any way he desires. However, this is not limited to this, and for example, the system may be designed so that feeding can only be done by operating the function button. Note that in this system 1, for example, the operation unit 2214 (see Fig. 3, for example) is used as the function button.

When feeding by voice, food is generated in response to voice input such as "Here's food," "Put the food in," or "Eat the food." When feeding by application on an external device (such as a smartphone), food is generated in response to tapping the food icon button on the application screen. When feeding by function button, food is generated and collected in response to pressing the operation unit 2214 installed on the top cover 105 of the system 1 (see FIG. 3, for example).

Food can only be given once per day. In other words, the number of times food is given will only be counted once per day.

By inputting an input operation by the user, a food container is displayed, and food, for example dry food, can be placed in this container.

Food can be collected in response to pressing the operation unit 2214. Even when food is displayed, new food can be generated by pressing a button on the operation unit 2214 or an application on an external terminal (e.g., a smartphone, etc.). Also, for example, if food remains by the time of a system update, the remaining food will be collected at the time of the system update.

As shown in FIG. 107B, the amount of food displayed in the food bowl is displayed in five levels, for example. Then, for example, the amount of food decreases each time JUNO eats food, and when JUNO has eaten food four times, the food bowl becomes empty. Once the food bowl becomes empty, it is no longer displayed.

As shown in FIG. 107B, when a specified time has elapsed from the start of playback of the eating motion showing JUNO eating food, the amount of food displayed in the food bowl decreases. The specified time is, for example, 3 seconds. When the food bowl becomes empty, it disappears. Also, if the conditions for JUNO to run away while eating are met, JUNO will run away to Area 5.

17. Sleep
Next, JUNO's sleep will be described with reference to Fig. 108A to Fig. 110. Fig. 108A to Fig. 108D are exemplary diagrams for explaining JUNO's behavioral patterns during sleep transition in system 1 according to one embodiment. Fig. 109 is an exemplary diagram for explaining Japanese responses during sleep in system 1 according to one embodiment. Fig. 110 is an exemplary diagram for explaining JUNO's behavioral patterns upon waking up in system 1 according to one embodiment.

As shown in Figures 108A to 108D, Juno's sleep time can be set via an application on an external device (such as a smartphone) described later (initial setting: for example, 10pm to 7am). If Juno is awake at the set time, Juno will go to sleep. Note that the area where Juno sleeps will differ depending on the level of intimacy.

For example, when the intimacy level is low, Juno will sleep in Area 5. Also, if Juno is in any of Areas 1 to 4, he will move from his current location to Area 5 before sleeping. When the intimacy level is medium or high, Juno will sleep in the area he is in when he starts to sleep. Note that when the intimacy level is low, Juno cannot be woken up by an external event (such as being called) while he is sleeping.

If an external event is detected while sleeping (for example, being called), Juno will behave differently depending on the intimacy level. For example, when the intimacy level is low, Juno will not wake up because he is in Area 5. When the intimacy level is medium and Juno is sleeping in Area 1, a motion will be played. When the intimacy level is high and Juno is sleeping in Area 1, Juno will respond in Japanese while remaining in a lying position.

As shown in Figure 109, when the intimacy level is high, if JUNO detects registered words (echo words, dialogue words) while sleeping, it will respond in Japanese according to the release stage of the dialogue words (see Figures 96 and 97).

For example, if the first stage is unlocked, when the user calls out to Juno with words like "Yuno," "Good morning," or "Hey, hey," Juno will respond with something like "Hmm? What's that?"

For example, if the second stage is unlocked, when the user calls out to Juno with words such as "Yuno," "Good morning," or "Hey, hey," Juno will respond with words such as "Hmm? What's that?" or "I was sleeping..."

For example, if the third stage is unlocked, when the user calls out to Juno with "Yuno," "Good morning," or "Hey, hey," Juno will respond with "Hmm? What's that?", "I was sleeping," "Hmm, good morning," etc.

Furthermore, for example, if the fourth stage is unlocked, when the user calls out to Juno with "Yuno," "Good morning," or "Hey hey," Juno will respond with "Huh? What's that?", "I was sleeping," "Hmm, good morning," "Hmm, Zzzzz," etc.

The response motion is the wake-up motion. After JUNO wakes up, he goes into standby.

In addition, a sleep induction check (for example, success rate 50%) is performed every 10 minutes, and if this sleep induction check is successful, JUNO performs the sleep action again.

Also, as shown in Figures 108D and 110, JUNO will wake up after the set time has passed.

By the way, there may be cases where an alarm or other setting is made during JUNO's sleep time. In this case, when the set time for the alarm arrives, JUNO will wake up and move to Area 1, and will call out to you, for example, "It's time." After the alarm goes off and stops, a sleep determination will be made again. If it is determined that JUNO is within its sleep time, it will go back to sleep.

In order to prevent JUNO from waking up due to an external event being detected while sleeping, the processing board 224 should be designed not to control JUNO to react even if an external event is detected. JUNO will react, for example, when the intimacy level is high and the character is sleeping in Area 1.

<18. Zoom display>
Next, a zoom display of JUNO will be described with reference to Fig. 111A to Fig. 111C. Fig. 111A to Fig. 111C are examples of diagrams for explaining a zoom display of JUNO in the system 1 according to an embodiment. In this example, for example, when JUNO speaks Japanese, the zoom display is performed.

As shown in Figures 111A to 111C, one method of zoom display is to move JUNO forward and enlarge the display. In this case, when JUNO speaks, it is necessary to move it to Area 0. Note that Area 0 is not normally displayed, but is displayed when JUNO is zoomed in, and is closer to the user than Area 1.

For zoom display, for example, there is a method of moving the camera position to enlarge the display. In this case, it does not depend on Juno's position. However, if Japanese speech occurs in the back, such as in Area 4, it will feel unnatural because the camera will have to move a long distance.

Therefore, in this embodiment, when JUNO speaks Japanese, he only does so in Area 1, the camera moves when he speaks, and the camera moves away after he finishes speaking. By doing this, it is possible to zoom in and display in a natural way without moving JUNO to Area 0 or causing any awkwardness with the camera movement.

<19. Linking with external devices (e.g. smartphones, etc.)>
Next, cooperation with an external terminal (e.g., a smartphone, etc.) will be described. In this embodiment, a smartphone will be described as an example of an external terminal. Note that an external terminal that can cooperate with the system 1 according to an embodiment is not limited to a smartphone, and may be, for example, an external terminal such as a mobile terminal such as a tablet PC or a notebook PC.

112A and 112B are diagrams showing an example of a screen transition displayed on an Android (registered trademark) version smartphone that can be linked with system 1 according to an embodiment. Also, FIGS. 113A, 113B, and 113C are diagrams showing an example of a screen transition displayed on an iOS (registered trademark) version smartphone that can be linked with system 1 according to an embodiment.

As shown in FIG. 112A, the login screen (see FIG. 112A (a)) displayed on the screen of the Android (registered trademark) version smartphone is a screen on which, for example, one of the "login" button, the "password reissue" button, and the "new registration" button can be operated. Thereafter, as shown in FIG. 112A and FIG. 112B, the screen displayed on the smartphone changes according to the operated button. Then, as shown in FIG. 112B, when the setup is completed via the setup in progress screen (see FIG. 112B (b)), etc., the screen transitions to the menu screen (see FIG. 112B (c)). The menu screen is a screen on which, as shown in FIG. 112B (c), for example, one of the "alarm" button, the "home appliance operation" button, the "settings" button, the "notification" button, and the "logout" button can be operated. When the user operates one of the buttons on the menu screen, the user can proceed to the menu corresponding to the operated button.

As shown in FIG. 113A, the login screen (see FIG. 113A (a)) displayed on the screen of the iOS (registered trademark) smartphone is a screen on which, for example, any of the "login" button, the "reissue password" button, and the "new registration" button can be operated, as in the case of the Android (registered trademark) smartphone. After that, as shown in FIG. 113A, FIG. 113B, and FIG. 113C, when the setup is completed via the connection waiting screen (see FIG. 113B (b1)) and the setup in progress screen (see FIG. 113C (b2)), the screen transitions to the menu screen (see FIG. 113C (c)). As shown in FIG. 113C (c), the menu screen is a screen on which, for example, any of the "alarm" button, the "home appliance operation" button, the "settings" button, the "notification" button, and the "logout" button can be operated, as in the case of the Android (registered trademark) smartphone. When the user presses a button on the menu screen, the user can proceed to the menu that corresponds to the button pressed.

Figure 113D shows an example of an enlarged view of a screen displayed on a smartphone screen, where (A) is an enlarged view of a login screen, and (B) is an enlarged view of a menu screen. Note that in this embodiment, the basic design of the login screen and menu screen is the same for the Android (registered trademark) version and the iOS (registered trademark) version, but is not limited to this.

Figure 113E shows an example of an enlarged view of a screen displayed on a smartphone, in which (A) is an enlarged view of the setup screen, and (B) is an enlarged view of the connection waiting screen. Note that in this embodiment, the basic design of the setup screen and the connection waiting screen is the same for the Android (registered trademark) version and the iOS (registered trademark) version, but is not limited to this.

As shown in Figures 112 and 113, system 1 can be linked to a compatible smartphone. When system 1 and a smartphone are linked, various operations including initial settings of system 1 can be performed, for example, on the smartphone.

When the initial settings of system 1 are performed using a smartphone, the items required for the initial settings include, for example, a compatible smartphone, a MyYupiteru account, and a wireless LAN environment. The application shown in FIG. 112 or FIG. 113 is installed on the smartphone that will link with system 1. Then, following the instructions of the application, system 1 and the smartphone are connected. After that, various settings and requests can be made following the instructions of the application.

When a connection request is received from a smartphone that is linked to system 1, the processing board 224 can establish a communication connection with the smartphone that made the connection request.

When the system 1 and the smartphone are connected for communication, the processing board 224 can perform various controls, including JUNO's operation, based on instruction requests from the smartphone. For example, a feeding event can be generated by tapping the food icon button on the smartphone's application screen. When a feeding event occurs, the processing board 224 can start a behavioral pattern operation, such as JUNO eating food.

20. Control of the System
The functions of this system have been explained above, but here, the control of the system will be explained with reference to the flowcharts. An example of the control executed by, for example, the processing board 224 will be explained with reference to Figures 114 to 127. Note that in the flowcharts shown in Figures 114 to 127, the expression "JUNO" is not used, but is illustrated as "character."

(Area movement processing)
First, the control of JUNO's area movement will be briefly described with reference to Fig. 114. Fig. 114 is a flow chart showing an example of area movement processing executed by the processing board 224 in the system 1 according to an embodiment. For example, the area movement processing may be performed at regular intervals. The regular interval here is, for example, about 10 minutes.

As shown in FIG. 114, the processing board 224 first determines whether "0≦experience value<30" (step S101). That is, it determines whether the experience value is, for example, within a range of 0 or more and less than 30. In step S101, if it is determined that the experience value is, for example, within a range of 0 or more and less than 30 (YES in step S101), the processing board 224 moves the process to step S102. Note that in this invention, the number of times (number of days) that the user has fed JUNO is used as a parameter for the experience value, so the experience value and the number of times (number of days) that the user has fed JUNO are synonymous.

In step S102, the coefficient for low intimacy is used to calculate the probability of movement to each area. As explained above, in step S102, the coefficient corresponding to low intimacy and the value of the experience points at this time are substituted into each basic formula (see, for example, Figures 58 and 59) that calculates the movement rate for areas 1 to 5 to calculate the probability of movement to each area. After the processing board 224 calculates the probability of movement to each area using the coefficient for low intimacy, it moves the process to step S106.

On the other hand, if it is determined in step S101 that the experience value is not within the range of, for example, 0 or more and less than 30 (NO in step S101), that is, if the experience value is 30 or more, the processing board 224 transfers the process to step S103.

In step S103, the processing board 224 determines whether, for example, "30≦empirical value<60". That is, it determines whether the empirical value is within a range, for example, of 30 or more and less than 60. In step S103, if it is determined that the empirical value is within a range, for example, of 30 or more and less than 60 (YES in step S103), the processing board 224 moves the process to step S104.

In step S104, the processing board 224 uses the coefficients in the intimacy to determine the probability of movement to each area. As explained above, in step S104, the coefficients in the intimacy and the experience value at this time are substituted into each basic formula that determines the movement rate to areas 1 to 5. After the processing board 224 calculates the probability of movement to each area using the coefficients in the intimacy, it moves the process to step S106.

On the other hand, if it is determined in step S103 that the experience value is not within the range of, for example, 30 or more and less than 60 (NO in step S103), that is, if it is determined that the experience value is 60 or more, the processing board 224 transfers the process to step S105.

In step S105, the processing board 224 uses the high intimacy coefficient to determine the probability of movement to each area. An experience value of 60 or more indicates high intimacy. As explained above, in step S105, the coefficient corresponding to high intimacy and the value of the experience value at this time are substituted into each basic formula that determines the movement rate to areas 1 to 5. After the processing board 224 calculates the probability of movement to each area using the high intimacy coefficient, it moves to step S106.

In step S106, the processing board 224 randomly determines the destination by drawing lots. Here, for example, a random drawing is performed based on each movement rate to determine the destination area for JUNO. That is, if the experience level is low, the destination is randomly determined based on the movement rate according to the number of days (i.e., experience level) shown in the low intimacy column in FIG. 64 above. If the intimacy level is medium, the destination is randomly determined based on the movement rate according to the number of days (i.e., experience level) shown in the medium intimacy column in FIG. 64 above. If the intimacy level is high, the destination is randomly determined based on the movement rate according to the number of days (i.e., experience level) shown in the high intimacy column in FIG. 64 above. After the processing board 224 randomly determines the destination by drawing lots, the processing proceeds to step S107.

In step S107, the processing board 224 displays the behavior pattern of JUNO moving to the destination area. Here, for example, if the area determined as the destination is area 1, the movement motion from the current area in which JUNO is displayed to the determined area 1 is displayed. Note that the area in which JUNO is displayed may also be determined as the destination area. In that case, JUNO does not move. However, a motion of JUNO moving from the current area, a motion of returning to the original area, etc. may be displayed. Also, JUNO's movement motion does not have to be limited to linear movement. For example, a motion of moving to the determined area while moving left and right may be displayed. Also, JUNO's movement motion may be a walking motion or a running motion. It is even better if there is a change for each intimacy level. For example, if the intimacy level is low, it may be a motion of walking slowly while being cautious. Also, if the intimacy level is high, it may be a motion of running up to the user.

The area change process does not have to be performed at a fixed interval. For example, whether or not to perform the area change process may be determined by lottery.

The control of the area movement lottery will be briefly described with reference to FIG. 115. FIG. 115 is a flowchart showing an example of the area movement lottery process executed by the processing board 224 in the system 1 according to one embodiment.

As shown in FIG. 115, the processing board 224 first determines whether or not the area movement lottery has been won (step S100).

As mentioned above, the area change lottery is held at a predetermined interval of about 1 to 10 minutes. In this embodiment, for example, the timing of the next area change lottery is determined at the time of the area change lottery. In this way, it is possible to randomize the timing of the area change lottery. However, this is not limited to this, and it may be held at a predetermined time, such as 1 minute, 5 minutes, or 10 minutes.

In step S100, if it is determined that the area change lottery has been won (YES in step S100), that is, if the area change lottery is held and it is decided to change the area, the processing board 224 performs the area change process from step S101 onwards. Note that the process from step 101 onwards is the same as in FIG. 114, so a detailed description will be omitted.

(Feeding event processing)
Next, the control of the feeding event will be briefly described with reference to Fig. 116. Fig. 116 is a flow chart showing an example of feeding event processing executed by the processing board 224 in the system 1 according to an embodiment. Feeding of JUNO by the user is counted up to once a day, for example.

As shown in FIG. 116, the processing board 224 first determines whether or not a feeding event has occurred (step S111). For example, a feeding event occurs when a feeding action is performed by the user. The feeding action is performed by, for example, pressing a function button. If it is determined in step S111 that a feeding event has occurred (YES in step S111), the processing board 224 proceeds to step S112.

In step S112, the processing board 224 determines whether or not this is the first feeding of the day. If it is determined in step S112 that this is the first feeding of the day (YES in step S112), the processing board 224 moves the process to step S113.

In step S113, the processing board 224 executes a feeding count. The feeding count is a process that counts up, for example, "1" as the number of times the user has fed JUNO. After executing the feeding count, the processing board 224 transfers the process to step S115.

On the other hand, if it is determined in step S112 that this is not the first feeding of the day (NO in step S112), i.e., if the number of feedings has already been counted, the processing board 224 proceeds to step S114.

In step S114, the processing board 224 determines whether or not a bait image is being displayed. If it is determined in step S114 that a bait image is not being displayed (YES in step S114), the processing board 224 moves the process to step S115.

In step S115, the processing board 224 displays a food image. In this embodiment, for example, the food image is displayed in area 1. The type of food displayed is, for example, dry food type food. However, the type of food displayed is not limited to dry food type, and for example, a wet food image or a snack type food image may be displayed. It is also preferable if the user can select the type of food to be displayed. Note that even if a food image is displayed, a new food image may be displayed again, and if a feeding event occurs while a food image is displayed, the display of the food image may be terminated.

(Feeding event processing)
Next, control of a feeding event will be briefly described with reference to Fig. 117. Fig. 117 is a flowchart showing an example of feeding event processing executed by the processing board 224 in the system 1 according to an embodiment.

As shown in FIG. 117, the processing board 224 first judges whether new food has been displayed (step S121). If it is judged in step S121 that new food has been displayed (YES in step S121), that is, if the user has performed a feeding action for the first time today (more specifically, the first time after a system update, which will be described later), the processing board 224 moves the process to step S122. Note that this does not include cases where there is leftover food from a previous time that JUNO ate.

In step S122, the processing board 224 performs a lottery for eating food according to the experience value. Here, for example, a lottery is performed based on the probability that JUNO will eat food according to the experience value of that day, and it is determined whether or not to eat the food. After performing the lottery for eating food according to the experience value, the processing board 224 moves the process to step S123.

In step S123, the processing board 224 determines whether or not the feeding lottery has been won. If it is determined in step S123 that the feeding lottery has been won (YES in step S121), the processing board 224 transfers the process to step S124.

In step S124, the processing board 224 executes a motion related to Juno eating food. Here, it first executes a motion to move Juno to the area where the food bowl is located. If Juno's intimacy level is low, for example, Juno may be shown moving cautiously. After executing the motion to move to the area where the food bowl is located, the processing board 224 executes a motion for Juno to eat food. After executing Juno's eating motion, the processing board 224 moves the process to step 125.

In step S125, the processing board 224 determines whether a predetermined time has elapsed. The predetermined time here is, for example, 3 seconds. If it is determined in step S125 that the predetermined time has elapsed (YES in step S125), the processing board 224 transfers the process to step S126.

In step S126, the processing board 224 executes a display to reduce the amount of food. Here, a display is executed to reduce the amount of food each time a predetermined time elapses during JUNO's food-eating motion. After executing the display to reduce the amount of food, the processing board 224 moves the process to step 129.

On the other hand, if it is determined in step S125 that the predetermined time has not elapsed (NO in step S125), the processing board 224 transfers the process to step S127.

In step S127, the processing board 224 determines whether the conditions for running away are met. If JUNO's intimacy level is low, the conditions for running away are met by detecting an external event. In this case, this applies when moving to the food bowl or when eating food. If it is determined in step S127 that the conditions for running away are met (NO in step S127), the processing board 224 moves the process to step S128.

In step S128, the processing board 224 executes a movement display to Area 5. For example, if the robot is moving to the food bowl, the robot stops the movement and executes a movement motion to Area 5. Also, for example, if the robot is moving to eat food, the robot stops the eating motion and executes a movement display to Area 5.

On the other hand, if it is determined in step S127 that the conditions for fleeing are not met (YES in step S127), the processing board 224 transfers the process to step S129.

In step S129, the processing board 224 determines whether or not to end the display of the food image. The display of the food image is ended when the food container becomes empty due to repeated displays of reducing the amount of food.

If it is determined in step S129 that the display of the bait image is not to be ended (NO in step S129), the processing board 224 returns the process to step S125 and repeats the processes from S125 onwards until it is determined that the display of the bait image is to be ended (until YES is determined in step S129) or until the condition for escaping is met (until NO is determined in step S127).

On the other hand, if it is determined in step S129 that the display of the food image is to be ended (YES in step S129), i.e., if the food bowl is empty, the processing board 224 ends the display of the food image (display of the food bowl). In addition, JUNO's eating motion also ends. Note that after the food-eating event ends, JUNO will continue to stay in Area 1 until the next area movement is performed.

(Claw scratch event processing)
Next, control of the above-mentioned scratching event will be briefly described with reference to Fig. 118. Fig. 118 is a flow chart showing an example of scratching event processing executed by the processing board 224 in the system 1 according to an embodiment. For example, JUNO scratches its claws once a day.

As shown in FIG. 118, the processing board 224 first determines whether a scratching event has occurred (step S131). If it is determined in step S131 that a scratching event has occurred (YES in step S131), the processing board 224 moves the process to step S132.

In step S132, the processing board 224 resets the timer. Here, for example, the timer is set to "0." After resetting the timer, the processing board 224 transfers the process to step S133.

In step S133, the processing board 224 starts timer measurement. For example, the timer is set to 5 minutes and measures for 5 minutes. During this time, JUNO should be alert to its surroundings. After starting the timer measurement, the processing board 224 moves the process to step S134.

In step S134, the processing board 224 determines whether or not an external event has been detected. As described above, an external event is detected when a moving object such as proximity detection (i.e., when a person is detected because a user approaches JUNO), gesture detection, calling out, or external detection such as detection of a loud sound of a predetermined decibel or more is detected). A gesture is, for example, when a gesture such as a wave of the hand is detected by the first sensor 2331 (see, for example, FIG. 5). A call is, for example, when a recognition word (see, for example, FIG. 94, FIG. 96) is detected. A loud sound is, for example, when a voice other than a recognition word or a sound-like object is detected.

If it is determined in step S134 that an external event has been detected (NO in step S134), the processing board 224 returns the process to step S132.

On the other hand, if it is determined in step S134 that an external event has not been detected (YES in step S134), the processing board 224 transfers the process to step S135.

In step S135, the processing board 224 determines whether five minutes have elapsed on the timer. If it is determined in step S135 that five minutes have elapsed on the timer (YES in step S135), that is, if no external event has been detected and it is determined that five minutes have elapsed on the timer, the processing board 224 transfers processing to step S136.

On the other hand, if it is determined in step S135 that the timer has not yet reached 5 minutes (NO in step S135), the processing board 224 returns the process to step S134. The processing board 224 repeats the processes of S134 and S135 until no external event is detected (YES in step S134) and 5 minutes of timer measurement has elapsed (YES in step S135), except when NO is determined in step S134.

In step S136, the processing board 224 executes a motion of JUNO sharpening its claws. Here, it first executes a motion of moving JUNO to the area where the scratching board is placed. The scratching board is displayed in area 3. JUNO is displayed moving vigilantly. After executing the movement motion to the scratching board, the processing board 224 executes a motion of JUNO sharpening its claws. After the processing board 224 executes the motion of JUNO sharpening its claws, it moves the process to step S137.

In step S137, the processing board 224 determines whether or not an external event has been detected. If it is determined in step S137 that an external event has not been detected (YES in step S137), the processing board 224 transfers the process to step S138.

If it is determined in step S138 that scratching has not ended (NO in step S138), the processing board 224 returns the process to step S136 and repeats the processes from S136 onwards until it is determined that scratching has ended (until YES is determined in step S138) unless the intimacy level is low (unless YES is determined in step S139).

Meanwhile, in step S138, the processing board 224 determines whether or not scratching has ended. If it is determined in step S138 that scratching has ended (YES in step S138), the processing board 224 ends the execution of the scratching motion. The scratching event ends, for example, when the scratching motion has been executed a predetermined number of times. The predetermined number here may be, for example, five times, or the number of times scratching may vary depending on the level of intimacy. After scratching has ended, JUNO will continue to stay in area 3 until the next area movement is performed.

Returning to step S137, if it is determined that an external event has been detected (NO in step S137), the processing board 224 transitions to S139.

In step S139, the processing board 224 determines whether or not JUNO's intimacy level is low. If it is determined in step S139 that JUNO's intimacy level is not low (NO in step S139), the processing board 224 transfers the process to step 138.

On the other hand, if it is determined in step S139 that JUNO's intimacy level is low (YES in step S139), the processing board 224 transfers the process to step 140.

In step S140, the processing board 224 executes an escape motion. For example, if JUNO is moving to the scratching board, the movement is stopped and a motion to move to Area 5 is executed. Also, for example, if JUNO is performing a scratching motion, the scratching motion is stopped and a movement display to Area 5 is executed. Also, after executing the area movement display of JUNO, the processing board 224 executes a motion of getting angry, looking away, etc. Furthermore, no further scratching will be performed on that day.

(Human detection event processing)
Next, control of the above-mentioned human detection event will be briefly described with reference to Fig. 119. Fig. 119 is a flowchart showing an example of human detection event processing executed by the processing board 224 in the system 1 according to an embodiment.

As shown in FIG. 119, the processing board 224 first determines whether, for example, a human detection event has occurred (step S151). A human detection event is executed, for example, when JUNO is displayed in areas 1 to 4. If it is determined in step S151 that a human detection event has occurred (YES in step S151), the processing board 224 transfers the process to step S152.

In step S152, the processing board 224 determines whether the intimacy level is low. If it is determined in step S152 that JUNO's intimacy level is low (YES in step S152), the processing board 224 moves the process to 153.

In step S153, the processing board 224 executes a movement display to Area 5. As described above, the movement display to Area 5 executes motions such as anger, turning away, etc. Note that JUNO with low intimacy level may not only move to Area 5, but may also, for example, run away outside the area. In that case, it is advisable not to display JUNO until the next area movement is performed.

On the other hand, in step S152, if the processing board 224 determines that JUNO's intimacy level is not low (NO in step S152), the processing board 224 transfers the process to step 154.

In step S154, the processing board 224 determines whether the intimacy level is high. If it is determined in step S154 that JUNO's intimacy level is not high (NO in step S154), that is, if JUNO's intimacy level is neither low nor high but is medium, movement display due to a human detection event is not performed.

On the other hand, if it is determined in step S154 that the intimacy level is high (YES in step S154), the processing board 224 transfers the process to step S155.

In step S155, the processing board 224 executes a moving display of JUNO to area 1. That is, when the intimacy level is low, JUNO is moved and displayed in area 5 (see S153), whereas when the intimacy level is high, JUNO is moved and displayed in area 1 (that is, moving closer to the user). At this time, it is preferable to execute a motion that makes JUNO seem familiar with the user, such as JUNO running up to the user happily.

(System update process)
Next, control of the above-mentioned system update will be briefly described with reference to Fig. 120. Fig. 120 is a flowchart showing an example of a system update process executed by the processing board 224 in the system 1 according to an embodiment.

As shown in FIG. 120, the processing board 224 first performs the current day's total data transmission process (step 161). In the current day's total data transmission process, data such as the number of times the pet has been fed today (up to a maximum of 1 count), the number of times it has been called out for each registered word (number of times the call word was detected), the Japanese language acquisition status of each registered word group (number of times the call word group was detected), and the number of times it responded using JUNO are collected, and the collected data is transmitted to an external server such as a cloud server. Note that the number of times the pet has been fed today is information that allows one to confirm whether the pet has not been fed yet or has already been fed. After performing the current day's total data transmission process, the processing board 224 moves to step S162.

In step S162, the processing board 224 performs a server data reception process. In the server data reception process, information related to the control of JUNO is received from, for example, a cloud server. After performing the server data reception process, the processing board 224 ends the system update process. Note that, after performing the server data reception process, the system update process may be ended by restarting the main body of the system 1. Also, processing such as compiling and updating JUNO information may be performed by the processing board 224 provided in the system 1, rather than an external server such as a cloud server.

(Server processing)
Next, control of the above-mentioned cloud server will be briefly described with reference to Fig. 121. Fig. 121 is a flowchart showing an example of a server process executed in the cloud server according to one embodiment.

In this embodiment, transmission and reception processing is performed between the processing board 224 included in the system 1 according to one embodiment and, for example, a cloud server, but the server capable of transmitting and receiving with the processing board 224 is not limited to a cloud server. For example, it may be an information processing device (hardware resource) that can transmit and receive with multiple systems 1, has a control device capable of executing calculation processing such as a CPU, ROM, and RAM, performs calculations and management based on information transmitted from the system 1 (for example, whether feeding has been performed, etc.), and transmits result information based on the calculation results (for example, information such as experience points) to the above-mentioned system 1.

As shown in FIG. 121, for example, the cloud server first performs a current day data reception process (step S171). In the current day data reception process, the current day aggregated data transmitted from the above-mentioned system 1 is received. After performing the current day data reception process, the cloud server proceeds to step S172.

In step S172, for example, the cloud server determines whether feeding has been performed. If it is determined in step S172 that feeding has been performed (YES in step S172), for example, if it is determined that the feeding data for that day is "1", the cloud server proceeds to step S173.

In step S173, for example, the cloud server adds experience points. When adding experience points, the cloud server adds, for example, 1 to the experience points managed by the cloud server. After adding the experience points, the cloud server moves the process to step S176.

On the other hand, if it is determined in step S172 that feeding has not occurred (NO in step S172), for example, if it is determined that the feeding data for that day is "0", the cloud server proceeds to step S174.

In step S174, for example, the cloud server determines whether feeding has not occurred for a predetermined period of time. The predetermined period here is, for example, three days. In this case, it may be three consecutive days, or it may be a cumulative three days. This may be changeable as appropriate by using JUNO settings, etc. In step S174, if it is not determined that feeding has not occurred for a predetermined period of time (NO in step S174), i.e., the period during which feeding has not occurred has not reached the predetermined period of time, for example, the cloud server moves the process to step S176.

On the other hand, if it is determined in step S174 that feeding has not been performed for a predetermined period of time (YES in step S174), the cloud server, for example, moves the process to step S175.

In step S175, for example, the cloud server subtracts from the experience points managed by the cloud server. After performing the experience point subtraction process, the cloud server transfers the process to step S176.

In step S176, for example, the cloud server determines whether the call word group has been called out a predetermined number of times. Here, it is determined whether the conditions related to the Japanese language acquisition status, such as the number of times each call word group or each registered word has been called out, are met. In step S176, if it is determined that the call word group has not been called out a predetermined number of times (NO in step S176), the cloud server moves the process to step S178.

On the other hand, if it is determined in step S176 that the call word group has been used a predetermined number of times (YES in step S176), the cloud server, for example, moves the process to step S177.

In step S177, for example, the cloud server performs a process to permit a corresponding response to a call. In the process to permit a corresponding response to a call, if each group of call words and the number of times each registered word has been called meet the above-mentioned predetermined conditions, a call response corresponding to the detected recognition word is permitted (see Figures 94 to 97E). Permitted call responses include, for example, registered words such as echoing, and response words from a group of call words that are released in stages. After the cloud server performs the process to permit a corresponding response to a call, it moves the process to step S178.

In step S178, for example, the cloud server performs a data transmission process. In the data transmission process, information on experience points that have been added or subtracted, information permitting a corresponding call response, and the like are transmitted to the system 1. Note that in the system 1, various information such as experience points is updated by performing an update process. Then, the processing board 224 performs an action according to the updated intimacy level or an action according to the various updated words.

(External connection processing)
Next, the above-mentioned control using a smartphone application will be briefly described with reference to Fig. 122. Fig. 122 is a flowchart showing an example of an external connection process in the system 1 according to an embodiment.

As shown in FIG. 122, the processing board 224 first determines whether or not there is a connection from a specific external terminal (step S181). A connection from a specific external terminal is, for example, a connection from a smartphone application. If it is determined in step S181 that there is a connection from a specific external terminal (YES in step S181), the processing board 224 moves the process to step S182.

In step S182, the processing board 224 determines whether or not a request has been made. A request may be, for example, a request related to settings of the system 1 main body from a connected mobile terminal such as a smartphone, a request related to feeding, or a function specific to a high level of intimacy (an alarm or a function related to the weather). If it is determined in step S182 that a request has been made (YES in step S182), the processing board 224 moves the process to step S183.

In step S183, the processing board 224 executes the setting according to the request. For example, if an alarm setting is requested, the processing board 224 executes the alarm setting.

(Alarm processing)
Next, control relating to the above-mentioned alarm function will be briefly described with reference to Fig. 123. Fig. 123 is a flowchart showing an example of alarm processing in the system 1 according to an embodiment. Setting of an alarm, etc. can be performed by a smartphone application, voice input, etc.

As shown in FIG. 123, the processing board 224 first determines whether it is the alarm setting time (step S191). If it is determined in step S191 that it is the alarm setting time (YES in step S191), the processing board 224 transfers the process to step S192.

In step S192, the processing board 224 displays JUNO in the set manner. For example, when the set alarm time arrives, JUNO is displayed moving to Area 1. Then, JUNO sounds the alarm. The alarm sounds by repeatedly outputting a sound such as "It's time." As mentioned above, functions specific to high intimacy levels, such as the alarm, can be used when JUNO's intimacy level becomes high. In addition, when the fourth stage is unlocked, JUNO is able to speak using response words. In this way, alarm operations that are highly convenient in everyday life can be performed by JUNO's actions, so you can increase convenience while enjoying everyday life with JUNO.

Functions such as alarms may be made available regardless of the level of intimacy or the level of Japanese language acquisition. In this case, it is possible to allow use without restrictions, but it is advisable to impose usage restrictions. For example, alarms may be restricted to be set by means other than voice input. Also, for example, the alarm may be set to sound by means other than JUNO's voice until a certain condition is met. In this way, the user can enjoy the convenience of the functions of system 1 and the changes in JUNO's level of intimacy or level of Japanese language acquisition through external connection.

Note that while FIG. 123 shows an example of alarm processing in system 1 according to one embodiment, timer processing and home appliance control processing are also performed in system 1 according to one embodiment. Functions such as the alarm function, timer function and home appliance control function are highly convenient functions in daily life, and in this system 1, these operations can be executed by JUNO's operations, so convenience can be increased while enjoying daily life with JUNO.

For example, when operating home appliances such as televisions and air conditioners, a dedicated infrared remote control that corresponds one-to-one with each appliance is often used. However, since this system 1 has a smart remote control function, it is possible to control home appliances using, for example, infrared rays emitted from this system 1 alone, without using a dedicated infrared remote control. Note that the smart remote control function is a well-known technology, so a description of it will be omitted here.

(The process of learning Japanese)
Next, the above-mentioned process of learning Japanese will be briefly described with reference to Figs. 124 to 127. As described above, the language that JUNO learns is not limited to Japanese. Fig. 124 is a flowchart showing an example of a greeting learning process in the system 1 according to an embodiment. Fig. 125 is a flowchart showing an example of a greeting response process in the system 1 according to an embodiment. Fig. 126 is a flowchart showing an example of a dialogue learning process in the system 1 according to an embodiment. Fig. 127 is a flowchart showing an example of a dialogue response process in the system 1 according to an embodiment.

(Processing for learning Japanese. Processing for learning greetings)
First, the greeting memorization process will be described. As shown in Fig. 124, the processing board 224 judges whether or not a greeting word has been detected (step S201). The greeting word is a registered word in the echo word group table of Fig. 94 described above. If it is judged in step S201 that a greeting word has been detected (YES in step S201), the processing board 224 moves the process to step S202.

In step S202, the processing board 224 determines whether or not JUNO's intimacy level is low. If it is determined in step S202 that JUNO's intimacy level is low (YES in step S202), the processing board 224 ends the greeting memorization process. In other words, if JUNO's intimacy level is low, the process for memorizing the greeting is not performed.

On the other hand, if it is determined in step S202 that JUNO's intimacy level is not low (NO in step S202), the processing board 224 moves the process to step S203.

In step S203, the processing board 224 determines whether or not JUNO is displayed in area 1. For example, in order to have JUNO memorize greetings, JUNO needs to be displayed in area 1. In step S203, if it is determined that JUNO is displayed in area 1 (YES in step S203), the processing board 224 moves the process to step S204.

In step S204, the processing board 224 counts greetings. Here, the number of times the detected greeting word is used is counted. For example, if the detected greeting word is "Good morning", "Good morning" is counted as "1". The counted information is sent to the cloud server when the system is updated as described above. After being processed by the cloud server, when the data transmission process (see S178) is performed, the character-specific saved information managed by the system 1 is updated.

(Japanese language learning processing. Greeting and response processing)
Next, the greeting response process (echo) will be described. As shown in Fig. 125, the processing board 224 first judges whether or not a greeting word has been detected (step S211). If it is judged in step S211 that a greeting word has been detected (YES in step S211), the processing board 224 moves the process to step S212.

In step S212, the processing board 224 determines whether or not JUNO's intimacy level is high. If it is determined in step S212 that JUNO's intimacy level is not high (NO in step S212), the processing board 224 transfers the process to step S216.

On the other hand, if it is determined in step S212 that JUNO's intimacy level is high (YES in step S212), the processing board 224 transfers the process to step S213.

In step S213, the processing board 224 determines whether or not JUNO is displayed in area 1. Although not shown, in the case of a high level of intimacy, when a greeting word is detected, a process is performed to move JUNO to area 1, so here it is determined that JUNO has been moved to area 1.

If it is determined in step S213 that the item is not displayed in area 1 (NO in step S213), the processing board 224 transfers the process to step S216.

On the other hand, if it is determined in step S213 that the item is displayed in area 1 (YES in step S213), the processing board 224 transfers the process to step S214.

In step S214, the processing board 224 determines whether or not the greeting is remembered. If it is determined in step S214 that the greeting is not remembered (NO in step S214), the processing board 224 transfers the process to step S216.

On the other hand, if it is determined in step S214 that the greeting is remembered (YES in step S214), the processing board 224 transfers the processing to step S215.

In step S215, the processing board 224 refers to the echoing word group table (see FIG. 94) and executes echoing. When the user already remembers the detected greeting word, echoing outputs a response word similar to the detected greeting word.

In step S216, the processing board 224 executes a motion display. Here, even if a greeting word is detected, if the conditions of steps S212 to S214 are not satisfied, the motion display is executed. At high and medium intimacy levels, the executed motion display corresponds to, for example, a motion that responds to a call from the user (hereinafter referred to as a "response motion"), and at low intimacy levels, corresponds to, for example, a motion that flees to Area 5 (hereinafter referred to as a "flee motion").

(Processing for memorizing Japanese language. Processing for memorizing dialogue)
Next, the dialogue memorization process will be described. As shown in Fig. 126, the processing board 224 first judges whether or not a call word has been detected (step S221). If it is judged in step S221 that a call word has been detected (YES in step S221), the processing board 224 proceeds to step S222.

In step S222, the processing board 224 determines whether or not JUNO's intimacy level is low. If it is determined in step S222 that JUNO's intimacy level is not low (NO in step S222), the processing board 224 transfers the process to step S223.

In step S223, the processing board 224 determines whether or not the character is displayed in area 1. If it is determined in step S223 that the character is displayed in area 1 (YES in step S222), the processing board 224 moves the process to step S224. That is, as described above, if the intimacy level is high, the character 1200 is displayed to move to area 1 when an external event is detected, so here, if the intimacy level is high, it is determined that the character is displayed in area 1.

In step S224, the processing board 224 determines whether the conditions are met. In this system 1, the first stage of JUNO's response words described above is released (see Figures 97A and 97B), causing JUNO to learn new dialogue words. In step S224, if it is determined that the conditions are met (YES in step S224), the processing board 224 moves the process to step S225.

In step S225, the processing board 224 refers to the dialogue word group table (see Figures 96 and 97) and counts up the call words. Here, the call word group related to the detected call word is counted. For example, if the detected call word is "JUNO", the call word group "JUNO (1)" is counted by "1".

(Japanese language learning processing, dialogue response processing)
Next, the dialogue response process will be described. As shown in Fig. 127, the processing board 224 first judges whether or not a call word has been detected (step S231). If it is judged in step S231 that a call word has been detected (YES in step S231), the processing board 224 proceeds to step S232.

In step S232, the processing board 224 determines whether the intimacy level is high. If it is determined in step S232 that the intimacy level is not high (NO in step S232), the processing board 224 transfers the process to step S236.

On the other hand, if it is determined in step S232 that the intimacy level is high (YES in step S232), the processing board 224 transfers the process to step S233.

In step S233, the processing board 224 determines whether or not it is displayed in area 1. If it is determined in step S233 that it is not displayed in area 1 (NO in step S233), the processing board 224 transfers the process to step S236.

On the other hand, if it is determined in step S233 that the item is displayed in area 1 (YES in step S233), the processing board 224 transfers the process to step S234.

In step S234, the processing board 224 determines whether or not the step has been released in stages. If it is determined in step S234 that the step has not been released in stages (NO in step S234), the processing board 224 transfers the process to step S236.

On the other hand, if it is determined in step S234 that gradual release has occurred (YES in step S234), the processing board 224 transfers the process to step S235.

In step S235, the processing board 224 refers to the dialogue word group table (see Figures 96A to 97E) and responds according to the stage. Here, the response word linked to the detected call word is output aloud. Also, for example, the response word changes in stages depending on Juno's progress in learning Japanese.

In step S236, the processing board 224 executes a motion display. Here, even if a call word is detected, if the conditions in steps S232 to S234 are not met, for example, a motion display is executed. If the intimacy level is high or medium, a response motion is executed, and if the intimacy level is low, a retreat motion is executed.

Note that the process of memorizing Japanese is not limited to this. For example, greeting words may be counted for each group of calling words. For example, if "Good morning" is detected, the "Good morning group" may be counted by "1". In that case, if the "Good morning group" is counted, for example, 20 times or more, JUNO may be able to memorize all the words in the "Good morning group". Also, conversation words may be counted for each registered word. For example, if "Yuno" is detected, the processing board 224 may be configured to count "Yuno" by "1".

In this way, the processing board 224 responds (S235) when JUNO is displayed in area 1, and does not respond when JUNO is not displayed in area 1. By responding when JUNO is displayed in area 1, communication can be achieved from a position relatively close to the user, giving the user an even greater sense of familiarity with JUNO.

In the above embodiment, the Japanese language learning function is unlocked at medium or higher intimacy levels, but this is not limited to this. For example, it may be possible to allow Juno to learn greeting words (registered words in the parrot-like word group) regardless of his intimacy level. In this case, the process of step S202 is not performed, and if the answer is YES in step S201, the process may proceed to step 203.

It may also be possible for JUNO to memorize greeting words regardless of the area in which he is displayed. In that case, it is advisable not to perform the process of S203. It is also possible for JUNO to memorize greeting words regardless of the area in which he is displayed in his intimacy level. In that case, if the answer is YES in step 201, it is advisable to perform the process of step 204.

In the above embodiment, for example, the experience value addition process (see S173), experience value subtraction process (S175), and permission to respond to a call (S177) are all performed by the cloud server. In this way, even if a malfunction occurs in the system 1, data such as previous experience value information can be prevented from being lost because the data is managed by the cloud server. In addition, for example, experience value updates and response word stages are released when a system update process (see FIG. 120) is performed. In this way, it is possible to prevent the occurrence of unnatural situations such as suddenly being permitted to respond to a call at a specific time during the day (i.e., when a set of call words has been used a certain number of times), and it is possible to create a sense of realism in which the character grows day by day.

In addition, the process of adding experience points (see S173), the process of subtracting experience points (S175), the process of allowing a response to a call (S177), etc. may be performed by the processing board 224. In this way, the process of adding experience points (see S173), the process of subtracting experience points (S175), the process of allowing a response to a call (S177), etc. can be performed without waiting for the system update process (see FIG. 120).

For example, it may be possible to allow JUNO to memorize dialogue words regardless of his intimacy level. In that case, it is advisable to not perform the process of step S222, and if the answer is YES in step 221, to perform the processes from step 233 onwards. Also, it may be possible for JUNO to memorize dialogue words regardless of the area that is displayed. In that case, it is advisable to not perform the process of S223. It is also possible for JUNO to memorize greeting words regardless of the area that is displayed in his intimacy level. In that case, it is advisable to perform the processes from step 224 onwards if the answer is YES in step 221.

JUNO may also be able to respond in Japanese regardless of the area in which it is displayed. If it is responding by repeating a greeting, it is advisable not to perform the process of step S213. If it is responding to a dialogue, it is advisable not to perform the process of step 233. Note that when the intimacy level is high, a human detection event or the like will cause JUNO to move to Area 1, but it is also possible to move to Area 1 only after JUNO responds to a call, etc.

When performing the process of learning Japanese regardless of the area, the process of learning Japanese (including responding) may be performed even in Area 5, which is not influenced from the outside. Also, when JUNO's voice is output, it may be at a constant volume, but the volume may be made to differ depending on the area in which JUNO is displayed. In that case, it is advisable to make JUNO's voice quieter as it is displayed in a position farther from the user, i.e., in the direction of Area 5.

<21. Weather reading function>
The weather reading function of the present system 1 will be described below with reference to Fig. 128A to Fig. 129C. The weather reading function assumes that the user inputs voice such as "when", "where", "tell me the weather", etc., and JUNO will read out the weather based on the voice input.

The weather reading function is unlocked at high intimacy levels. Therefore, at low and medium intimacy levels, even if there is voice input, the processing board 224 displays a behavior pattern that makes it appear as if JUNO is ignoring it. This can encourage the user to feed JUNO.

Fig. 128A is an example of an outline for explaining expected voice input information from a user in system 1 according to one embodiment. Fig. 128B is a diagram showing an example of how defined time periods are called in system 1 according to one embodiment. Figs. 129A to 129C are an example of an outline for explaining a response sentence created (hereinafter sometimes referred to as "generated") based on acquired weather information in system 1 according to one embodiment.

As shown in FIG. 128A, the ROM 702 of the processing board 224 stores, as voice input information corresponding to "when," words such as "today," "tomorrow," and "evening." In addition, as voice input information corresponding to "where," words such as the names of all prefectures and all city, town, and village groups as regions of Japan are stored. Keywords such as "weather" and "tell me" are also stored.

As shown in FIG. 128B, in this embodiment, a day (24 hours) is divided into three-hour intervals, and for example, 0:00 to 3:00 are defined as "pre-dawn," 3:00 to 6:00 are defined as "dawn," 6:00 to 9:00 are defined as "morning," 9:00 to 12:00 are defined as "early afternoon," 12:00 to 15:00 are defined as "early afternoon," 15:00 to 18:00 are defined as "evening," 18:00 to 21:00 are defined as "early evening," and 21:00 to 24:00 are defined as "late night." For example, around 12:00 (for example, 10:30 to 13:30) is defined as "around noon," and 12:00 is defined as "noon." For example, 0:00 to 12:00 are defined as "morning," and 12:00 to 24:00 are defined as "afternoon." For example, 9:00 to 18:00 are defined as "daytime," and 18:00 to 24:00 are defined as "night."

The processing board 224 also acquires weather information provided by, for example, the Japan Meteorological Agency in response to a user request or at any time, creates a comment based on the user request and the weather information acquired from outside, and speaks the created summary.

As shown in FIG. 128, for example, if the voice input for "when" is "today," weather information for that day, etc. is obtained; if it is "before dawn," weather information for 1 a.m. on that day, etc. is obtained; if it is "at dawn" or "early morning," weather information for 4 a.m. on that day, etc. is obtained; if it is "now," data including the current time (weather information, etc.) is obtained; and if it is "over the weekend" or "this week," forecaster comments, etc. are obtained and handled accordingly.

The processing board 224 also creates a phrase for JUNO to respond to based on the acquired weather information. For example, if the voice input for "when" is "today," JUNO's response phrase may be a forecaster's comment. On the other hand, if the voice input for "when" is "in the morning," a phrase for JUNO to respond to (hereinafter also referred to as a summary comment) is generated based on short-term information for, for example, 6:00 to 11:00. Also, if the voice input for "when" is "from the afternoon," a phrase for JUNO to respond to is generated based on short-term information for, for example, 12:00 to 18:00. Note that this short-term information may be replaced with weather information for explanation. Note that the comment is finally created as a hearsay-style sentence.

In this way, if the voice input information from the user contains a word stored in the ROM 702 of the processing board 224, the processing board 224 can have JUNO output the created phrase aloud. In this way, it is possible to respond to the user's requests while communicating between the user and JUNO.

The processing board 224 also creates additional information in addition to the summary comment. The additional information is created in a different context from the summary comment. For example, the summary comment may be written so that it is output in a serious tone, as if a newscaster were speaking, whereas the additional information may be written so that it is output in a soft tone, in a voice that gives a sense of familiarity, as if a family member were speaking.

Weather information that can be acquired from external servers, etc., includes, for example, daily weather information, i.e., weather information from midnight to midnight, and hourly weather information, for example, short-term weather information such as the hourly weather information from 6:00 to 7:00 or the hourly weather information from 7:00 to 8:00.

The weather information for each day includes, for example, at least one or all of the following information: information on weather conditions, information on maximum temperatures, information on minimum temperatures, and information on the probability of precipitation.

The hourly weather information includes, for example, at least one or all of the following: information on weather conditions, information on temperature, and information on precipitation.

In this way, by including at least one or all of the following in the daily and hourly weather information: information on weather conditions, information on maximum temperature, information on minimum temperature, and information on the probability of precipitation, it is possible to provide useful information to the user. For example, the user can decide what to wear that day based on information on the maximum temperature, and can determine whether or not an umbrella is needed that day based on information on weather conditions and the probability of precipitation.

In addition, if weather information obtained from outside indicates, for example, "sunny," the device may respond with a phrase such as, "The weather looks like it's sunny." In addition, hourly weather information is obtained by, for example, acquiring six hours' worth of discrete data as short-term information.

Priorities are set in advance for the six hours of short-term information that can be obtained from outside, and the information with the highest priority among the six hours of short-term information obtained from outside can be used as weather information.

In addition, for example, if six hours' worth of short-term information obtained from outside have the same priority, the short-term information that appears the most frequently from the six hours can be adopted (i.e., determined by majority vote). Furthermore, if not only the priority but also the number of occurrences are the same, the short-term information that appears earlier can be adopted as the weather information. Even if no priority is set for the six hours' worth of short-term information obtained from outside, or if the priorities are set but not the same, the information that appears the most frequently can be adopted.

In this way, if the weather information obtained from outside is discrete weather information, for example, short-term information collected over multiple hours (e.g., six hours), it is possible to provide the user with relatively accurate information for deciding, for example, what to wear that day, whether an umbrella is necessary, etc.

It is also advisable to have the weather information acquired from outside include daily weather information in addition to discrete weather information, such as short-term information collected over multiple hours (e.g., six hours), for each hour. In this way, it is possible to provide the user with general weather information for the day. For example, it is possible to provide the user with either discrete weather information or daily weather information as short-term information, depending on the user's preference, thereby improving user convenience.

The area from which weather information is obtained can be specified by inputting "where (from what area)". However, if no specification is made, the location set in a smartphone application or the like can be used. In this case, the location specification that can be specified can be, for example, only the prefecture and its location.

Next, we will explain how to generate a summary comment that is created when short-term information is acquired from an external source, such as when the voice input for "when" is "in the morning," and how to generate additional information that is created in addition to the summary comment.

As shown in Fig. 129A and Fig. 129B, the processing board 224 generates summary comments indicating the way the rain is falling and the state of the rain from discrete data, such as precipitation data, for six hours, for example, from 6:00 to 7:00, 7:00 to 8:00, 8:00 to 9:00, 9:00 to 10:00, 10:00 to 11:00, and 11:00 to 12:00. Note that if the voice input for "when" is "from the afternoon," etc., a summary comment indicating the way the rain is falling and the state of the rain is generated from precipitation data from 12:00 to 18:00.

For example, if the amount of precipitation is "0" for all time periods, i.e., "0" precipitation from 6:00 to 7:00, "0" precipitation from 7:00 to 8:00, "0" precipitation from 8:00 to 9:00, "0" precipitation from 9:00 to 10:00, "0" precipitation from 10:00 to 11:00, and "0" precipitation from 11:00 to 12:00, the processing board 224 generates phrases such as "It will not rain in the morning."

In addition, if the amount of precipitation is not "0" for all time periods, for example, if the amount of precipitation from 6:00 to 7:00 is "1," from 7:00 to 8:00 is "10," from 8:00 to 9:00 is "7," from 9:00 to 10:00 is "8," from 10:00 to 11:00 is "3," and from 11:00 to 12:00 is "2," the processing board 224 generates phrases such as "It looks like it will rain all morning." It also generates explanatory phrases about the maximum amount of precipitation within the time period, such as "It looks like it will rain particularly heavily from 7:00."

Also, if the rain stops halfway through (the amount of precipitation becomes "0"), i.e., the amount of precipitation from 6:00 to 7:00 is "1", from 7:00 to 8:00 is "10", from 8:00 to 9:00 is "7", from 9:00 to 10:00 is "0", from 10:00 to 11:00 is "0", and from 11:00 to 12:00 is "0", the processing board 224 generates phrases such as "It looks like it will rain from around 6:00 to around 9:00". In this way, explanatory phrases about the maximum amount of precipitation within a time period are also generated.

Also, if it starts raining partway through (the amount of precipitation is not "0" partway through), i.e., the amount of precipitation from 6:00 to 7:00 is "0", from 7:00 to 8:00 is "0", from 8:00 to 9:00 is "0", from 9:00 to 10:00 is "8", from 10:00 to 11:00 is "3", and from 11:00 to 12:00 is "2", the processing board 224 generates a phrase such as "It looks like it will rain from around 9:00 to around 12:00". In such a case, an explanatory phrase about the maximum amount of precipitation within the time period is also generated.

If it rains during the period, i.e., the amount of precipitation from 6:00 to 7:00 is "0", from 7:00 to 8:00 is "0", from 8:00 to 9:00 is "7", from 9:00 to 10:00 is "8", from 10:00 to 11:00 is "3", and from 11:00 to 12:00 is "0", the processing board 224 generates phrases such as "It looks like it will rain from around 8:00 to around 11:00". It also generates explanatory phrases about the maximum amount of precipitation within the period, such as "It looks like it will rain especially heavily from around 9:00".

Also, if it is raining at a specific time, i.e., the amount of precipitation is "0" from 6:00 to 7:00, "0" from 7:00 to 8:00, "1" from 8:00 to 9:00, "0" from 9:00 to 10:00, "0" from 10:00 to 11:00, and "0" from 11:00 to 12:00, the processing board 224 generates phrases such as "It looks like it will be drizzling lightly from around 8:00 to around 9:00."

In addition, if the rain falls in a manner other than those described above, the processing board 224 generates phrases such as "The weather doesn't look good this morning."

In addition, if the amount of precipitation from 6:00 to 7:00 is "1", from 7:00 to 8:00 is "0", from 8:00 to 9:00 is "1", from 9:00 to 10:00 is "0", from 10:00 to 11:00 is "0", and from 11:00 to 12:00 is "0", the processing board 224 will generate phrases such as "It looks like it will be drizzling, especially around 6:00. It looks like the rain will come and go in the morning."

In addition, if the amount of precipitation from 6:00 to 7:00 is "1", from 7:00 to 8:00 is "0", from 8:00 to 9:00 is "2", from 9:00 to 10:00 is "1", from 10:00 to 11:00 is "0", and from 11:00 to 12:00 is "0", the processing board 224 will generate phrases such as "It looks like it will be drizzling, especially around 8:00. It looks like the rain will come and go in the morning."

In addition, if the amount of precipitation from 6:00 to 7:00 is "1", from 7:00 to 8:00 is "0", from 8:00 to 9:00 is "1", from 9:00 to 10:00 is "0", from 10:00 to 11:00 is "1", and from 11:00 to 12:00 is "0", the processing board 224 will generate phrases such as "It looks like it will be drizzling, especially around 6:00. It looks like the rain will come and go in the morning."

In addition, if the amount of precipitation from 6:00 to 7:00 is "1", from 7:00 to 8:00 is "5", from 8:00 to 9:00 is "7", from 9:00 to 10:00 is "0", from 10:00 to 11:00 is "3", and from 11:00 to 12:00 is "0", the processing board 224 will generate phrases such as "It looks like it will rain heavily especially around 8:00. It looks like it will rain most of the morning."

In addition, if the amount of precipitation from 6:00 to 7:00 is "1", from 7:00 to 8:00 is "0", from 8:00 to 9:00 is "7", from 9:00 to 10:00 is "8", from 10:00 to 11:00 is "3", and from 11:00 to 12:00 is "1", the processing board 224 also generates an explanation of the maximum amount of precipitation within the time period, such as "It looks like it will rain heavily especially around 9:00." Note that if the maximum amount of precipitation is the same, a phrase is generated to explain only the first time period.

Next, we will explain how to generate additional information corresponding to the generated phrase. Additional information is information that can give the user a sense of familiarity, such as "don't fall" when the weather is snowing. When the processing board 224 has JUNO speak the additional information, it controls it so that it is output in a familiar voice, distinct from the speech of the summary comment.

As shown in FIG. 129C, if the generated phrase is, for example, "It doesn't seem to be raining," the processing board 224 does not generate additional information.

In addition, if the generated phrase is "It looks like it's raining a little at a time," the processing board 224 generates additional information such as "When you go out, take an umbrella with you."

Also, if the generated phrase is "It looks like it's going to rain," the processing board 224 generates additional information such as "You might want to take an umbrella with you when you go out."

In addition, if the generated phrase is "It looks like it's going to rain heavily," the processing board 224 generates additional information such as "You might want to take an umbrella when you move around."

In addition, if the generated phrase is "It looks like it's going to be raining quite heavily," the processing board 224 will generate additional information such as "You might need an umbrella when you go out."

In addition, if the generated phrase is "It looks like a heavy downpour is coming," the processing board 224 generates additional information such as "Even if you use an umbrella, your feet may get wet."

In addition, if the generated phrase is "It feels like a torrential rain is falling, like a bucket being overturned," the processing board 224 generates additional information such as "Even if you use an umbrella, you might still get wet."

In addition, if the generated phrase is "It looks like it's going to rain very heavily, like a waterfall," the processing board 224 will generate additional information such as "You might want to avoid going outside."

In addition, if the generated phrase is "It looks like it's going to rain very heavily," the processing board 224 generates additional information such as "You might want to avoid going outside."

(Weather event processing)
Next, the control of weather events in this system will be described with reference to Fig. 130. Fig. 130 shows an example of weather event processing executed by the processing board 224 in the system 1 according to one embodiment.

As shown in FIG. 130, the processing board 224 first determines whether a weather event has occurred (step S241). A weather event occurs when, for example, a voice input such as "when," "where," or "tell me the weather" is detected. If it is determined in step S241 that a weather event has occurred (YES in step S241), the processing board 224 transfers the process to S242.

In step S242, it is determined whether the intimacy level is high. As described above, it is preferable that JUNO reads out weather information on the condition that the intimacy level is high and the user can respond in Japanese. If the intimacy level is low or medium, even if a weather event occurs, i.e., if voice input such as "when", "where", or "tell me the weather", is detected, JUNO is displayed as ignoring it. In other words, JUNO's behavior pattern is such that when the intimacy level is relatively low (for example, when the intimacy level is low or medium), JUNO ignores the user's calls, and when the intimacy level is relatively high, JUNO responds to the user's requests, making JUNO a realistic character who acts as if based on instinct. This can also encourage the user to feed JUNO. If it is determined in step S242 that the intimacy level is high (YES in step S242), the processing board 224 moves the process to S243.

In step S243, the processing board 224 acquires various information based on the input information. Here, for example, weather information is acquired from an external server by voice input as shown in FIG. 128. Weather information for each day, hourly weather information for six hours, etc. is acquired. After acquiring various information based on the input information, the processing board 224 transfers the process to step S244.

In step S244, the processing board 224 sets various comments according to the acquired information. Here, for example, daily weather information is acquired. As for setting various comments according to the acquired information, for example, if the acquired weather information indicates "sunny", a comment such as "It looks like the weather will be sunny" is set. Also, when setting a comment from hourly weather information, for example, the highest priority information from six hours of short-term information is adopted as the weather information, and a comment according to that weather information is set. After the processing board 224 sets various comments according to the acquired information, the process proceeds to step S245.

In step S245, the processing board 224 performs an analysis of the acquired discrete data. Here, for example, analysis of weather information, which is discrete data for six hours from 6:00 to 12:00, acquired in response to input information such as "morning", is performed. After analyzing the acquired discrete data, the processing board 224 transfers the process to step S246.

In step S246, the processing board 224 generates and sets a summary comment based on the analysis data. Here, the processing board 224 generates and sets a summary comment based on the data analyzed as shown in FIG. 129. After generating and setting the summary comment based on the analysis data, the processing board 224 transfers the process to step S247.

In step S247, the processing board 224 outputs the set comment by voice on JUNO. Here, JUNO outputs phrases and the like that are set to respond as if it were a human, and to teach the user in a friendly manner in a one-on-one conversation.

If there is a change in weather information, etc., JUNO may automatically read out the latest weather information (e.g., summary comments and additional information, etc.). In other words, even if no weather event is occurring, JUNO may be made to actively read out weather information of its own volition, making JUNO a more realistic character.

<22. News reading function>
Below is a brief explanation of the news reading function provided in the present system 1. The news reading function assumes that the user will input voice such as "when,""where(what)," and "tell me a keyword (e.g., news)," and based on this voice input, the processing board 224 will obtain news information from an external server, etc., and JUNO will read it out.

For example, when a special call word such as "Tell me about today's sports news" from the user is detected, the processing board 224 acquires information on the day's sports news from the news information server and outputs the information through JUNO. For example, JUNO reads out the title of the headline or the like in the first designated area from the acquired information. The reading out may be the part of the title excluding the additional information called a tag that is not related to the content. A tag is, for example, information that indicates the attributes of news information and is information added to the content of the news. A tag may be, for example, a predetermined character string such as "update" or "replacement" that is added when the content of the news information is changed (for example, updated or replaced) after the first news information is distributed. In this case, the system 1 may output the voice reading out the title of the news information excluding the predetermined character string. Furthermore, when a special call word such as "tell me more," is detected, JUNO reads out the content of the sports news in the second designated area corresponding to the headline.

For example, if there are multiple pieces of sports news information, JUNO will read out the titles in the order of the first designated area, the second designated area, and so on. For example, if the user says "Tell me more" in the first designated area, JUNO will read out the content of the sports news corresponding to the second designated area. Note that the timing for acquiring various news information may be set to a fixed timing, such as every hour, or even better, to acquire news information when a special call word is detected.

The voice output in the above-mentioned embodiment may be replaced by or combined with output by display, communication, printing, or other methods. For example, the voice output of a character, exemplified by the character's speech, is an example of output associated with a character. For example, instead of or in combination with the voice output of a character, it may be displayed associated with the character (for example, a speech bubble may be displayed around the character), and text indicating the content that was output as voice in the above-mentioned embodiment may be displayed within the speech bubble. For example, instead of or in combination with the voice output of a character, it may be displayed showing the character holding a board, and text indicating the content that was output as voice on the board. Also, such a display may be made on another device by communicating with that device, or may be printed and output on paper or other recording medium.

Parameters such as experience points and intimacy levels described in the above embodiment may be displayed so that they can be viewed by the user. The processing board 224 may display the parameters, for example, on a screen that displays JUNO or on another screen. The parameters may be displayed using a bar graph, graph, meter, or other display method.

In the above-described embodiment, the process of increasing a parameter may be replaced with a process of decreasing a parameter. In the above-described embodiment, the process of decreasing a parameter may be replaced with a process of increasing a parameter. For example, what is understood to be an increase in intimacy can also be understood to be a decrease in the degree of unrecognition. For example, the specific internal process for reproducing the relationship between the user's input to system 1 and the output of system 1 in the form of voice, display, etc., as in the above-described embodiment is not limited to the process in the above-described embodiment.

The scope of the present invention is not limited to the configurations expressly described in the specification, but includes combinations of various aspects of the invention disclosed herein. The configurations of the invention that are sought to be patented are specified in the accompanying claims, but it is the intention of the present inventors to claim configurations disclosed in this specification that are not currently specified in the claims.

The present invention is not limited to the configurations described in the above-mentioned embodiments. The components of each of the above-mentioned embodiments and variations may be arbitrarily selected and combined. Any components of each of the embodiments and variations may be arbitrarily combined with any components described in the means for solving the invention or any components that embody any components described in the means for solving the invention. We intend to acquire rights to these as well through amendments to this application or divisional applications, etc. Even if there is a description such as "in the case of" or "when", this is not intended to describe a configuration that is limited to that case or time. We also disclose configurations that are not in these cases or times, and we intend to acquire rights to them. Furthermore, the parts that are described in order are not limited to this order. We also disclose configurations in which some parts are deleted or the order is changed, and we intend to acquire rights to them.

In addition, by converting to a design registration application, we intend to obtain rights to the entire design or partial design. If the device displays an image, we intend to obtain rights to the design including the entire or part of that image. The drawings show the entire device in solid lines, but they include not only the entire design but also partial designs claimed for a part of the device. For example, some parts of the device may be partial designs, and the drawings include some parts of the device as partial designs regardless of the parts. A part of the device may be a part of the device, or a part of that part. We intend to obtain rights to not only the entire design, but also partial designs in which any part of the solid line part of the drawing is shown as a broken line part. In addition, all of the modules, parts, and components inside the device's casing shown in the drawings are subject to independent trade, and we intend to obtain rights to them by converting to a design registration application in the same way.

1: Display device, system 100: Housing 101: Right side panel 102: Front side panel 103: Left side panel 104: Rear side panel 105: Top cover portion 106: Bottom cover portion 107: Right side internal panel 108: Front side internal panel 109: Left side internal panel 110: Opening 200: Dimming member 210: First area 220: Second area 221: Operation board 222: Slot 223: USB connector 224: Processing board 226: Holding portion 227: Shooting portion 231: Control board 232: Wiring 233: Sensor board 234: Human presence sensor 310: First display portion 311: First display surface 312: Viewing angle control member 320: Optical member 321: First surface 330: Second display portion 331 : Second display surface 400 : Base 500 : Light blocking member 601 : First plate 602 : Second plate 800 : Wiring section 810 : Power line 820 : Signal line 830 : Signal line 900 : Partition member 910 : Background board 1041 : Back cover 1051 : Opening 1052 : Opening 1100 : Character 1200 : Character 1210 : Character main body 1220 : Collar 1221 : Tag 2100 : Stage image 2200 : Stage image 2211 : Operation unit 2212 : Operation unit 2213 : Operation unit 2214 : Operation unit 2251 : Speaker 2252 : Speaker 2261 : Heat sink 2311 : Power jack 2331 : First sensor 2332 : Second sensor 2333 : Microphone 2334 : Light emitting unit

Claims (6)

A display unit capable of displaying characters;
a control means having a function of changing the experience information of the character and a function of changing the action mode of the character in response to the experience information;
Equipped with
the control means has a storage means for storing a preset language, a language recognition function capable of recognizing a language from an external source, and a counting function for counting the number of times the language is recognized when the language recognized by the language recognition function is a language stored in the storage means;
When the count value counted by the counting function reaches a specified value, a response by the character is permitted.
A system characterized by : 2. The system according to claim 1 , wherein when the count value counted by said counting function reaches a specified value and the value of said experience information reaches a predetermined specified value, a response by said character is permitted. 3. The system according to claim 1 , wherein the control means has a function of causing the character to respond at a position within a range in which the character can be displayed that is relatively close when the user looks at the display unit. The system according to any one of claims 1 to 3, characterized in that the control means has a repetition function that causes the character to repeat the language recognized by the language recognition function if the language is a language stored in the storage means. The system according to claim 4 , wherein the repetition function has a function of making the character repetitively recite the language at a position relatively close to the display unit when the user looks at the display unit within a range in which the character can be displayed. 6. The system according to claim 4 , wherein the control means activates the repeat function when the value of the experience information reaches a predetermined specified value.

Images