JP7567311B2 - Information providing device, information providing method, and program - Google Patents

Description

The present invention relates to an information providing device, an information providing method, and a program.

Recently, information devices have evolved significantly with the advancement of high-speed CPUs, high-resolution screen display technology, small and lightweight battery technology, the spread of wireless network environments, and the expansion of broadband. Such information devices include not only smartphones, which are a representative example, but also so-called wearable devices that are worn by users. For example, Patent Document 1 describes a device that gives the user the sensation that a virtual object is actually present by presenting multiple sensory information to the user. Patent Document 2 describes that the form and timing of information provision by an information provision device are determined so as to maximize the sum of an evaluation function that represents the appropriateness of the timing of information provision.

JP 2011-96171 A JP 2011-242219 A

Here, information providing devices that provide information to users are required to provide information appropriately to users.

In view of the above problems, the present invention aims to provide an information providing device, an information providing method, and a program that can provide appropriate information to a user.

An information providing device according to one aspect of the present invention is an information providing device that provides information to a user, and includes an output unit including a display unit that outputs visual stimuli, an audio output unit that outputs auditory stimuli, and a sensory stimulus output unit that outputs sensory stimuli different from vision and hearing, an environmental sensor that detects environmental information around the information providing device, an output specification determination unit that determines output specifications for the visual stimuli, the auditory stimuli, and the sensory stimuli based on the environmental information, and an output control unit that causes the output unit to output the visual stimuli, the auditory stimuli, and the sensory stimuli based on the output specifications.

An information provision method according to one aspect of the present invention is a method for providing information to a user, and includes the steps of: detecting surrounding environmental information; determining output specifications for visual stimuli, auditory stimuli, and sensory stimuli other than visual and auditory stimuli based on the environmental information; and outputting the visual stimuli, auditory stimuli, and sensory stimuli from an output unit based on the output specifications.

A program according to one aspect of the present invention causes a computer to execute an information provision method for providing information to a user, the method including the steps of: detecting surrounding environmental information; determining output specifications for visual stimuli, auditory stimuli, and sensory stimuli other than visual and auditory stimuli based on the environmental information; and outputting the visual stimuli, auditory stimuli, and sensory stimuli to an output unit based on the output specifications.

The present invention allows appropriate information to be provided to users.

FIG. 1 is a schematic diagram of an information providing device according to the present embodiment. FIG. 2 is a diagram showing an example of an image displayed by the information providing device. FIG. 3 is a schematic block diagram of an information providing device according to the present embodiment. FIG. 4 is a flowchart for explaining the process of the information providing device according to this embodiment. FIG. 5 is a table illustrating an example of the environment score. FIG. 6 is a table showing an example of an environment pattern. FIG. 7 is a schematic diagram for explaining an example of the levels of the output specifications of a content image. FIG. 8 is a table showing the relationship between the environment patterns, the target devices, and the reference output specifications. FIG. 9 is a graph showing an example of a pulse wave. FIG. 10 is a table showing an example of the relationship between the user state and the output specification correction degree. FIG. 11 is a table showing an example of the output restriction necessity information.

The following describes in detail an embodiment of the present invention with reference to the drawings. Note that the present invention is not limited to the embodiment described below.

(Information Providing Device)
FIG. 1 is a schematic diagram of an information providing device according to the present embodiment. The information providing device 10 according to the present embodiment is a device that provides information to a user U by outputting a visual stimulus, an auditory stimulus, and a sensory stimulus to the user U. The sensory stimulus here is a stimulus for a sense different from vision and hearing. In the present embodiment, the sensory stimulus is a tactile stimulus, but is not limited to a tactile stimulus, and may be a stimulus for any sense different from vision and hearing. For example, the sensory stimulus may be a stimulus for taste, a stimulus for smell, or a stimulus for two or more of touch, taste, and hearing. As shown in FIG. 1, the information providing device 10 is a so-called wearable device that is worn on the body of the user U. In the example of the present embodiment, the information providing device 10 includes a device 10A worn on the eye of the user U, a device 10B worn on the ear of the user U, and a device 10C worn on the arm of the user. The device 10A worn on the eye of the user U includes a display unit 26A (described later) that outputs a visual stimulus (displays an image) to the user U, the device 10B worn on the ear of the user U includes a sound output unit 26B (described later) that outputs an auditory stimulus (sound) to the user U, and the device 10C worn on the arm of the user U includes a sensory stimulus output unit 26C (described later) that outputs a sensory stimulus to the user U. However, the configuration in FIG. 1 is an example, and the number of devices and the positions at which they are worn on the user U may be arbitrary. For example, the information providing device 10 is not limited to a wearable device and may be a device carried by the user U, such as a so-called smartphone or tablet terminal.

(Environmental Image)
FIG. 2 is a diagram showing an example of an image displayed by the information providing device. As shown in FIG. 2, the information providing device 10 provides the user U with an environmental image PM through the display unit 26A. As a result, the user U wearing the information providing device 10 can visually recognize the environmental image PM. In this embodiment, the environmental image PM is an image of a landscape that the user U would visually recognize if it were assumed that the user U was not wearing the information providing device 10, and can also be said to be an image of an actual object that is within the range of the user U's field of vision. In this embodiment, the information providing device 10 provides the user U with the environmental image PM by, for example, transmitting external light (ambient visible light) from the display unit 26A. That is, in this embodiment, it can be said that the user U directly visually recognizes an image of an actual landscape through the display unit 26A. However, the information providing device 10 is not limited to allowing the user U to directly visually recognize an image of an actual landscape, and may provide the user U with the environmental image PM through the display unit 26A by displaying an image of the environmental image PM on the display unit 26A. In this case, the user U visually recognizes the image of the scenery displayed on the display unit 26A as the environmental image PM. In this case, the information providing device 10 causes an image captured by a camera 20A (described later) and falling within the visual field of the user U to be displayed on the display unit 26A as the environmental image PM. Note that, although roads and buildings are included as the environmental image PM in FIG. 2, this is merely an example.

(Content image)
As shown in FIG. 2, the information providing device 10 displays a content image PS on the display unit 26A. The content image PS is an image other than an actual scene that is within the visual field of the user U. The content image PS may be any content (content) as long as it is an image that includes information to be notified to the user U. For example, the content image PS may be a distributed image such as a movie or a television program, a navigation image that shows the user U directions, or a notification image that indicates that communication to the user U has been received, such as a phone call or email, or may include all of these. Note that the content image PS may not include advertisements, which are information that notify users of products or services.

In the example of FIG. 2, the content image PS is displayed on the display unit 26A so as to be superimposed on the environmental image PM provided through the display unit 26A. This allows the user U to visually recognize an image in which the content image PS is superimposed on the environmental image PM. However, the manner in which the content image PS is displayed is not limited to superimposition as shown in FIG. 2. The manner in which the content image PS is displayed, i.e., the output specifications described below, are set, for example, based on environmental information, and will be described in detail below.

(Configuration of the information providing device)
3 is a schematic block diagram of the information providing device according to the present embodiment. As shown in FIG. 3, the information providing device 10 includes an environmental sensor 20, a biosensor 22, an input unit 24, an output unit 26, a communication unit 28, a storage unit 30, and a control unit 32.

(Environmental Sensor)
The environmental sensor 20 is a sensor that detects environmental information around the information providing device 10. The environmental information around the information providing device 10 can be said to be information indicating the environment in which the information providing device 10 is placed. In other words, since the information providing device 10 is worn by the user U, the environmental sensor 20 can be said to detect environmental information around the user U.

The environmental sensor 20 includes a camera 20A, a microphone 20B, a GNSS receiver 20C, an acceleration sensor 20D, a gyro sensor 20E, a light sensor 20F, a temperature sensor 20G, and a humidity sensor 20H. However, the environmental sensor 20 may include any sensor that detects environmental information, and may include, for example, at least one of the camera 20A, the microphone 20B, the GNSS receiver 20C, the acceleration sensor 20D, the gyro sensor 20E, the light sensor 20F, the temperature sensor 20G, and the humidity sensor 20H, or may include other sensors.

The camera 20A is an imaging device, and captures the surroundings of the information providing device 10 by detecting visible light around the information providing device 10 (user U) as environmental information. The camera 20A may be a video camera that captures images at a predetermined frame rate. The position and orientation of the camera 20A in the information providing device 10 are arbitrary, but for example, the camera 20A may be provided in the device 10A shown in FIG. 1, and the imaging direction may be the direction in which the face of the user U is facing. This allows the camera 20A to capture an object in the line of sight of the user U, that is, an object within the range of the user U's field of vision. The number of cameras 20A is arbitrary, and may be one or more. Note that when there are multiple cameras 20A, information on the direction in which the camera 20A is facing is also acquired.

The microphone 20B is a microphone that detects sounds (sound wave information) around the information providing device 10 (user U) as environmental information. The position, orientation, and number of microphones 20B installed in the information providing device 10 are arbitrary. Note that, if there are multiple microphones 20B, information on the direction in which the microphones 20B are facing is also obtained.

The GNSS receiver 20C is a device that detects the position information of the information providing device 10 (user U) as environmental information. The position information here is Earth coordinates. In this embodiment, the GNSS receiver 20C is a so-called GNSS (Global Navigation Satellite System) module that receives radio waves from satellites and detects the position information of the information providing device 10 (user U).

The acceleration sensor 20D is a sensor that detects the acceleration of the information providing device 10 (user U) as environmental information, such as gravity, vibration, and impact.

The gyro sensor 20E is a sensor that detects the rotation and orientation of the information providing device 10 (user U) as environmental information, and detects this using the principles of Coriolis force, Euler force, centrifugal force, etc.

The optical sensor 20F is a sensor that detects the intensity of light around the information providing device 10 (user U) as environmental information. The optical sensor 20F can detect the intensity of visible light, infrared light, and ultraviolet light.

The temperature sensor 20G is a sensor that detects the temperature around the information providing device 10 (user U) as environmental information.

The humidity sensor 20H is a sensor that detects the humidity around the information providing device 10 (user U) as environmental information.

(Biosensor)
The biosensor 22 is a sensor that detects bioinformation of the user U. The biosensor 22 may be provided at any position as long as it can detect the bioinformation of the user U. The bioinformation here is preferably not unchanging, such as a fingerprint, but is information whose value changes depending on the state of the user U. Furthermore, the bioinformation here is preferably information related to the autonomic nerves of the user U, that is, information whose value changes regardless of the will of the user U. Specifically, the biosensor 22 includes a pulse wave sensor 22A and an electroencephalogram sensor 22B, and detects the pulse waves and electroencephalograms of the user U as bioinformation.

The pulse wave sensor 22A is a sensor that detects the pulse wave of the user U. The pulse wave sensor 22A may be, for example, a transmission type photoelectric sensor equipped with a light emitting unit and a light receiving unit. In this case, the pulse wave sensor 22A may be configured such that the light emitting unit and the light receiving unit face each other across the fingertip of the user U, and the light receiving unit receives light that has passed through the fingertip, and measures the pulse waveform by utilizing the fact that the greater the pressure of the pulse wave, the greater the blood flow. However, the pulse wave sensor 22A is not limited to this and may be of any type that can detect pulse waves.

The EEG sensor 22B is a sensor that detects the brain waves of the user U. The EEG sensor 22B may be of any configuration as long as it can detect the brain waves of the user U, but in principle, it is sufficient to grasp waves such as alpha waves and beta waves, as well as basic rhythmic (background brain wave) activity that appears throughout the brain, and detect improvements or decreases in activity of the brain as a whole, so several sensors may be provided. Unlike EEG measurements for medical purposes, this embodiment only requires measuring rough changes in the state of the user U, so it is possible to detect very simple surface EEG signals by attaching only two electrodes to the forehead and ears, for example.

The biosensor 22 is not limited to detecting pulse waves and brain waves as bioinformation, and may detect, for example, at least one of pulse waves and brain waves. The biosensor 22 may also detect bioinformation other than pulse waves and brain waves, and may detect, for example, the amount of sweat or the size of the pupils. The biosensor 22 is not an essential component, and may not be provided in the information processing device 10.

(Input section)
The input unit 24 is a device that accepts operations from a user, and may be, for example, a touch panel.

(Output section)
The output unit 26 is a device that outputs a stimulus for at least one of the five senses to the user U. Specifically, the output unit 26 has a display unit 26A, an audio output unit 26B, and a sensory stimulus output unit 26C. The display unit 26A is a display that outputs a visual stimulus to the user U by displaying an image, and can also be referred to as a visual stimulus output unit. In this embodiment, the display unit 26A is a so-called HMD (Head Mount Display). As described above, the display unit 26A displays the content image PS. The audio output unit 26B is a device (speaker) that outputs an auditory stimulus to the user U by outputting audio, and can also be referred to as an auditory stimulus output unit. The sensory stimulus output unit 26C is a device that outputs a sensory stimulus to the user U, which in this embodiment is a tactile stimulus. For example, the sensory stimulation output unit 26C is a vibration motor such as a vibrator, and outputs a tactile stimulation to the user by physically operating, such as by vibrating, but the type of tactile stimulation is not limited to vibration and may be any type.

In this way, the output unit 26 stimulates, of the five human senses, sight, hearing, and a sense different from sight and hearing (touch in this embodiment). However, the output unit 26 is not limited to outputting a visual stimulus, an auditory stimulus, and a sense different from sight and hearing. For example, the output unit 26 may output at least one of a visual stimulus, an auditory stimulus, and a sense different from sight and hearing, may output at least a visual stimulus (display an image), may output either an auditory stimulus or a touch in addition to a visual stimulus, or may output another sensory stimulus of the five senses (i.e., at least one of a taste stimulus and an olfactory stimulus) in addition to at least one of a visual stimulus, an auditory stimulus, and a touch.

(Communications Department)
The communication unit 28 is a module that communicates with an external device, and may include, for example, an antenna. In this embodiment, the communication method by the communication unit 28 is wireless communication, but the communication method may be any communication method. The communication unit 28 includes a content image receiving unit 28A. The content image receiving unit 28A is a receiver that receives content image data, which is image data of a content image. Note that the content displayed by the content image may include sound and sensory stimuli other than visual and auditory. In this case, the content image receiving unit 28A may receive sound data and sensory stimuli data as content image data together with the image data of the content image. Note that the content image data is received by the content image receiving unit 28A in this way, but may also be stored in advance in the storage unit 30, for example, and the content image receiving unit 28A may read the content image data from the storage unit 30.

(Memory unit)
The storage unit 30 is a memory that stores various information such as the contents of calculations and programs of the control unit 32, and includes at least one of a main storage device such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive).

The memory unit 30 stores a learning model 30A, map data 30B, and a specification setting database 30C. The learning model 30A is an AI model used to identify the environment in which the user U is located based on environmental information. The map data 30B is data including location information of real buildings and natural objects, and can be said to be data in which the earth coordinates are associated with real buildings and natural objects. The specification setting database 30C is a database including information for determining the display specifications of the content image PS, as described later. The processing using the learning model 30A, the map data 30B, the specification setting database 30C, and the like will be described later. The learning model 30A, the map data 30B, the specification setting database 30C, and the program for the control unit 32 stored in the memory unit 30 may be stored in a recording medium readable by the information providing device 10. In addition, the program for the control unit 32, the learning model 30A, the map data 30B, and the specification setting database 30C stored in the storage unit 30 are not limited to being stored in advance in the storage unit 30, and when using these data, the information providing device 10 may obtain them from an external device via communication.

(Control unit)
The control unit 32 is a calculation device, that is, a central processing unit (CPU). The control unit 32 includes an environmental information acquisition unit 40, a biometric information acquisition unit 42, an environment determination unit 44, a user state determination unit 46, an output selection unit 48, an output specification determination unit 50, a content image acquisition unit 52, and an output control unit 54. The control unit 32 reads out a program (software) from the storage unit 30 and executes it to realize the environmental information acquisition unit 40, the biometric information acquisition unit 42, the environment determination unit 44, the user state determination unit 46, the output selection unit 48, the output specification determination unit 50, the content image acquisition unit 52, and the output control unit 54, and executes the processes. The control unit 32 may execute these processes using one CPU, or may be provided with multiple CPUs and execute the processes using the multiple CPUs. In addition, at least a portion of the environmental information acquisition unit 40, the biometric information acquisition unit 42, the environmental determination unit 44, the user state determination unit 46, the output selection unit 48, the output specification determination unit 50, the content image acquisition unit 52, and the output control unit 54 may be realized in hardware.

The environmental information acquisition unit 40 controls the environmental sensor 20 to cause the environmental sensor 20 to detect environmental information. The environmental information acquisition unit 40 acquires the environmental information detected by the environmental sensor 20. The processing of the environmental information acquisition unit 40 will be described later. Note that when the environmental information acquisition unit 40 is hardware, it can also be called an environmental information detector.

The biometric information acquisition unit 42 controls the biometric sensor 22 to cause the biometric sensor 22 to detect biometric information. The biometric information acquisition unit 42 acquires environmental information detected by the biometric sensor 22. The processing of the biometric information acquisition unit 42 will be described later. Note that if the biometric information acquisition unit 42 is hardware, it can also be called a biometric information detector. Note that the biometric information acquisition unit 42 is not a required component.

The environment identification unit 44 identifies the environment in which the user U is located based on the environmental information acquired by the environmental information acquisition unit 40. The environment identification unit 44 calculates an environmental score, which is a score for identifying the environment, and identifies an environmental state pattern indicating the state of the environment based on the environmental score, thereby identifying the environment. The processing of the environment identification unit 44 will be described later.

The user status identification unit 46 identifies the status of the user U based on the biometric information acquired by the biometric information acquisition unit 42. The processing of the user status identification unit 46 will be described later. Note that the user status identification unit 46 is not a required component.

The output selection unit 48 selects a target device to be operated among the output unit 26 based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the bioinformation acquired by the bioinformation acquisition unit 42. The processing of the output selection unit 48 will be described later. When the output selection unit 48 is hardware, it may be called a sensory selector. In cases where the output specifications are determined based on environmental information or the like by the output specification determination unit 50 described later, the output selection unit 48 does not need to be provided. In this case, for example, the information providing device 10 may operate all of the output unit 26, i.e., the display unit 26A, the audio output unit 26B, and the sensory stimulation output unit 26C, without selecting a target device.

The output specification determination unit 50 determines the output specification of the stimulus (here, visual stimulus, auditory stimulus, tactile stimulus) to be output by the output unit 26 based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the bioinformation acquired by the bioinformation acquisition unit 42. For example, the output specification determination unit 50 can be said to determine the display specification (output specification) of the content image PS displayed by the display unit 26A based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the bioinformation acquired by the bioinformation acquisition unit 42. The output specification is an index indicating how the stimulus output by the output unit 26 is to be output, and will be described in detail later. The processing of the output specification determination unit 50 will be described later. Note that, for example, when the output selection unit 48 selects the target device based on the environmental information or the like, the output specification determination unit 50 may not be provided. In this case, for example, the information providing device 10 may output the stimulus to the selected target device with any output specification without determining the output specification from the environmental information or the like.

The content image acquisition unit 52 acquires content image data via the content image receiving unit 28A.

The output control unit 54 controls the output unit 26 to perform output. The output control unit 54 causes output to be performed to the target device selected by the output selection unit 48 in the output specifications determined by the output specification determination unit 50. For example, the output control unit 54 controls the display unit 26A to display the content image PS acquired by the content image acquisition unit 52 superimposed on the main image PM and in the display specifications determined by the output specification determination unit 50. Note that when the output control unit 54 is hardware, it may be called a multi-sensory sensory providing device.

The information providing device 10 has the configuration described above.

(Processing Contents)
Next, the processing contents of the information providing device 10, more specifically, the processing contents of the output unit 26 based on the environmental information and the biological information will be described. Fig. 4 is a flowchart for explaining the processing contents of the information providing device according to this embodiment.

(Getting environmental information)
As shown in Fig. 4, the information providing device 10 acquires the environmental information detected by the environmental sensor 20 by the environmental information acquiring unit 40 (step S10). In this embodiment, the environmental information acquiring unit 40 acquires image data of the surroundings of the information providing device 10 (user U) from the camera 20A, acquires audio data of the surroundings of the information providing device 10 (user U) from the microphone 20B, acquires position information of the information providing device 10 (user U) from the GNSS receiver 20C, acquires acceleration information of the information providing device 10 (user U) from the acceleration sensor 20D, acquires orientation information, i.e., attitude information, of the information providing device 10 (user U) from the gyro sensor 20E, acquires infrared and ultraviolet intensity information of the surroundings of the information providing device 10 (user U) from the light sensor 20F, acquires temperature information of the surroundings of the information providing device (user U) from the temperature sensor 20G, and acquires humidity information of the surroundings of the information providing device 10 (user U) from the humidity sensor 20H. The environmental information acquiring unit 40 acquires these pieces of environmental information one after another at a predetermined time interval. The environmental information acquiring unit 40 may acquire each piece of environmental information at the same timing, or may acquire each piece of environmental information at different timing. The predetermined time interval until the next piece of environmental information is acquired may be set arbitrarily, and the predetermined time interval may be the same for each piece of environmental information, or may be different for each piece of environmental information.

(Determination of Dangerous Conditions)
After acquiring the environmental information, the information providing device 10 determines, by the environmental identification unit 44, whether the environment around the user U is in a dangerous state based on the environmental information (step S12).

The environment identification unit 44 determines whether or not there is a danger state based on an image of the surroundings of the information providing device 10 captured by the camera 20A. Hereinafter, the image of the surroundings of the information providing device 10 captured by the camera 20A will be referred to as a surrounding image as appropriate. The environment identification unit 44, for example, identifies an object captured in the surrounding image and determines whether or not there is a danger state based on the type of the identified object. More specifically, the environment identification unit 44 may determine that there is a danger state when the object captured in the surrounding image is a specific object set in advance, and may determine that there is no danger state when the object is not a specific object. The specific object may be set arbitrarily, but may be, for example, an object that may pose a danger to the user U, such as a flame indicating a fire, a vehicle, or a sign indicating that construction is underway. The environment identification unit 44 may also determine whether or not there is a danger state based on multiple surrounding images captured consecutively in time series. For example, the environment determination unit 44 determines whether the objects are specific objects and the same object for each of the multiple peripheral images captured in succession in time series. If the same specific object is captured, the environment determination unit 44 determines whether the specific object captured in the peripheral image captured later in time series is relatively larger in the image, that is, whether the specific object is approaching the user U. If the specific object captured in the peripheral image captured later is larger, that is, if the specific object is approaching the user U, the environment determination unit 44 determines that the environment is in a dangerous state. On the other hand, if the specific object captured in the peripheral image captured later is not larger, that is, if the specific object is not approaching the user U, the environment determination unit 44 determines that the environment is not in a dangerous state. In this way, the environment determination unit 44 may determine whether the environment is in a dangerous state based on one peripheral image, or may determine whether the environment is in a dangerous state based on multiple peripheral images captured in succession in time series. For example, the environment determination unit 44 may switch the determination method depending on the type of object captured in the peripheral image. When a specific object whose danger can be determined from one peripheral image, such as a flame indicating a fire, is captured, the environment determination unit 44 may determine that a dangerous state exists from one peripheral image. Also, for example, when a specific object whose danger cannot be determined from one peripheral image, such as a vehicle, is captured, the environment determination unit 44 may determine that a dangerous state exists based on multiple peripheral images captured consecutively in time series.

The environment identification unit 44 may identify the object in the peripheral image by any method, but may identify the object using the learning model 30A, for example. In this case, for example, the learning model 30A is an AI model constructed by learning from multiple data sets as teacher data, with the image data and information indicating the type of object in the image being treated as one data set. The environment identification unit 44 inputs the image data of the peripheral image into the trained learning model 30A, obtains information that identifies the type of object in the peripheral image, and identifies the object.

In addition, the environment identification unit 44 may determine whether or not there is a danger state based on the location information acquired by the GNSS receiver 20C in addition to the surrounding image. In this case, the environment identification unit 44 acquires location information indicating the location of the user U based on the location information of the information providing device 10 (user U) acquired by the GNSS receiver 20C and the map data 30B. The location information is information indicating the type of location in which the user U (information providing device 10) is located. That is, for example, the location information is information that the user U is in a shopping center or information that the user U is on a road. The environment identification unit 44 reads the map data 30B, identifies the type of structure or natural object within a predetermined distance range from the current location of the user U, and identifies the location information from the structure or natural object. For example, if the current location of the user U overlaps with the coordinates of a shopping center, the location information is identified as the fact that the user U is in a shopping center. The environment determination unit 44 determines that a dangerous state exists when there is a specific relationship between the location information and the type of object determined from the surrounding image, and determines that there is no dangerous state when there is no specific relationship. The specific relationship may be set arbitrarily, but for example, a combination of an object and a location that may cause danger if that object is present in a certain location may be set as the specific relationship.

The environment identification unit 44 also determines whether or not there is a dangerous state based on the audio information acquired by the microphone 20B. Hereinafter, the audio information around the information providing device 10 acquired by the microphone 20B will be referred to as the surrounding audio as appropriate. The environment identification unit 44, for example, identifies the type of audio contained in the surrounding audio, and determines whether or not there is a dangerous state based on the identified type of audio. More specifically, the environment identification unit 44 may determine that there is a dangerous state if the type of audio contained in the surrounding audio is a specific audio set in advance, and may determine that there is no dangerous state if the type of audio is not the specific audio. The specific audio may be set arbitrarily, and may be audio that may pose a danger to the user U, such as audio indicating a fire, audio of a vehicle, or audio indicating that construction is underway.

The environment identification unit 44 may identify the type of sound contained in the ambient sound using any method, but may, for example, identify the target object using the learning model 30A. In this case, for example, the learning model 30A is an AI model constructed by learning from multiple data sets that are teacher data, with sound data (e.g., data indicating the frequency and intensity of the sound) and information indicating the type of sound as one data set. The environment identification unit 44 inputs the sound data of the ambient sound into the trained learning model 30A, obtains information that identifies the type of sound contained in the ambient sound, and identifies the type of sound.

The environment identification unit 44 may also determine whether a dangerous state exists based on location information acquired by the GNSS receiver 20C in addition to the surrounding sounds. In this case, the environment identification unit 44 acquires location information indicating the location of the user U based on the location information of the information providing device 10 (user U) acquired by the GNSS receiver 20C and the map data 30B. The environment identification unit 44 then determines that a dangerous state exists when there is a specific relationship between the location information and the type of sound identified from the surrounding sounds, and determines that there is no dangerous state when there is no specific relationship. The specific relationship may be set arbitrarily, but for example, a combination of sound and location that may lead to danger if the sound occurs in a certain location may be set as the specific relationship.

In this manner, in this embodiment, the environment determination unit 44 determines a dangerous state based on the surrounding image and the surrounding sound. However, the method of determining a dangerous state is not limited to the above and is arbitrary. For example, the environment determination unit 44 may determine a dangerous state based on either the surrounding image or the surrounding sound. Furthermore, the environment determination unit 44 may determine whether a dangerous state exists based on at least one of the image of the surroundings of the information providing device 10 captured by the camera 20A, the sound of the surroundings of the information providing device 10 detected by the microphone 20B, and the position information acquired by the GNSS receiver 20C. Furthermore, in this embodiment, the determination of a dangerous state is not essential and need not be performed.

(Setting the content of danger notifications)
When it is determined that the vehicle is in a dangerous state (step S10, Yes), the information providing device 10 sets the danger notification content, which is the notification content for notifying the user that the vehicle is in a dangerous state, by the output control unit 54 (step S12). The information providing device 10 sets the danger notification content based on the content of the dangerous state. The content of the dangerous state is information indicating what kind of danger is approaching, and is specified from the type of object shown in the surrounding image and the type of sound included in the surrounding sound. For example, if the object is a vehicle approaching, the content of the dangerous state is "a vehicle is approaching". The danger notification content is information indicating the content of the dangerous state. For example, if the content of the dangerous state is that a vehicle is approaching, the danger notification content is information indicating that a vehicle is approaching.

The danger notification content differs depending on the type of target device selected in step S26 described below. For example, when the display unit 26A is the target device, the danger notification content becomes the display content (content) of the content image PS. That is, the danger notification content is displayed as the content image PS. In this case, for example, the danger notification content is image data indicating the content "Watch out, a car is approaching!" On the other hand, when the audio output unit 26B is the target device, the danger notification content becomes the content of the voice output from the audio output unit 26B. In this case, for example, the danger notification content becomes voice data for emitting the voice "A car is approaching. Watch out." Also, when the sensory stimulation output unit 26C is the target device, the danger notification content becomes the content of the sensory stimulation output from the sensory stimulation output unit 26C. In this case, for example, the danger notification content becomes a tactile stimulation that attracts the attention of the user U.

The setting of the danger notification content in step S14 may be performed at any time after it has been determined in step S12 that a danger state exists and before the danger notification content is output in the subsequent step S38, for example, after the target device is selected in the subsequent step S32.

(Calculation of environmental score)
When it is determined that the user U is not in a dangerous state (step S12; No), the information providing device 10 calculates various environmental scores based on the environmental information by the environment identification unit 44, as shown in steps S16 to S22. The environmental score is a score for identifying the environment in which the user U (information providing device 10) is placed. Specifically, the environment identification unit 44 calculates the posture score (step S16), the location score (step S18), the movement score (step S20), and the safety score (step S22) as the environmental score. The order of steps S16 to S22 is not limited to this and is arbitrary. Note that even when the danger notification content is set in step S14, various environmental scores are calculated as shown in steps S16 to S22. The environmental score will be described in more detail below.

Figure 5 is a table illustrating an example of an environmental score. As shown in Figure 5, the environmental identification unit 44 calculates an environmental score for each environmental category. An environmental category indicates the type of environment for the user U, and in the example of Figure 5, includes the posture of the user U, the location of the user U, the movements of the user U, and the safety of the environment around the user U. In addition, the environmental identification unit 44 divides the environmental categories into more specific subcategories and calculates an environmental score for each subcategory.

(Posture score)
The environment determination unit 44 calculates a posture score as an environment score for the posture category of the user U. In other words, the posture score is information indicating the posture of the user U, and can be said to be information indicating, as a numerical value, what posture the user U is in. The environment determination unit 44 calculates the posture score based on environmental information related to the posture of the user U, among multiple types of environmental information. Examples of the environmental information related to the posture of the user U include a surrounding image acquired by the camera 20A and the orientation of the information providing device 10 detected by the gyro sensor 20E.

5, the category of the user U's posture includes a subcategory of standing and a subcategory of the face facing horizontally. The environment identification unit 44 calculates a posture score for the subcategory of standing based on the peripheral image acquired by the camera 20A. The posture score for the subcategory of standing can be a numerical value indicating the degree of agreement of the user U's posture with respect to the standing state. The posture score for the subcategory of standing may be calculated in any manner, and may be calculated using the learning model 30A, for example. In this case, for example, the learning model 30A is an AI model constructed by learning a plurality of datasets as teacher data, with image data of the scenery in the person's field of vision and information indicating whether the person is standing as one dataset. The environment identification unit 44 inputs image data of the peripheral image into the learned learning model 30A to obtain a numerical value indicating the degree of agreement with the standing state, and sets it as the posture score. Note that the degree of agreement with the standing state is used here, but is not limited to the standing state, and may be, for example, the degree of agreement with a sitting state or a lying state.

The environment determination unit 44 also calculates a posture score for the subcategory of the face orientation being horizontal, based on the orientation of the information providing device 10 detected by the gyro sensor 20E. The posture score for the subcategory of the face orientation being horizontal can be a numerical value indicating the degree of agreement of the posture (face orientation) of the user U with respect to the horizontal direction. Any method may be used to calculate the posture score for the subcategory of the face orientation being horizontal. Note that, although the degree of agreement with the face orientation being horizontal is used here, it is not limited to the horizontal direction, and may be the degree of agreement with any direction.

In this way, the environment determination unit 44 can be said to set information indicating the posture of the user U (here, posture score) based on the peripheral image and the orientation of the information providing device 10. However, the environment determination unit 44 is not limited to using the peripheral image and the orientation of the information providing device 10 to set the information indicating the posture of the user U, and may use any environmental information, for example, at least one of the peripheral image and the orientation of the information providing device 10.

(Location score)
The environment determination unit 44 calculates a location score as an environment score for the category of the location of the user U. In other words, the location score is information indicating the location of the user U, and can be said to be information indicating, as a numerical value, what type of location the user U is located in. The environment determination unit 44 calculates the location score based on environmental information related to the location of the user U among multiple types of environmental information. Examples of environmental information related to the location of the user U include a surrounding image acquired by the camera 20A, position information of the information providing device 10 acquired by the GNSS receiver 20C, and surrounding audio acquired by the microphone 20B.

5, the category of the user U's location includes a subcategory of being on a train, a subcategory of being on the tracks, and a subcategory of sounds on a train. The environment identification unit 44 calculates a location score for the subcategory of being on a train based on the surrounding image acquired by the camera 20A. The location score for the subcategory of being on a train can be said to be a numerical value indicating the degree of agreement of the user U's location with respect to the location of being on a train. The method of calculating the location score for the subcategory of being on a train may be arbitrary, but may be calculated using the learning model 30A, for example. In this case, for example, the learning model 30A is an AI model constructed by learning multiple data sets as teacher data, with image data of the scenery in a person's field of vision and information indicating whether the person is on a train as one data set. The environment identification unit 44 inputs image data of the surrounding image into the learned learning model 30A to obtain a numerical value indicating the degree of agreement with the location of being on a train, and sets it as the location score. Note that, although the degree of match was calculated for the location of being on a train, this is not limited thereto, and the degree of match may be calculated for being in any type of vehicle.

The environment determination unit 44 calculates a location score for the subcategory of being on the railway tracks based on the position information of the information providing device 10 acquired by the GNSS receiver 20C. The location score for the subcategory of being on the railway tracks can be a numerical value indicating the degree of match of the user U's location with respect to the location of being on the railway tracks. Any method for calculating the location score for the subcategory of being on the railway tracks may be used, for example, map data 30B may be used. For example, the environment determination unit 44 reads out the map data 30B, and when the current position of the user U overlaps with the coordinates of the railway tracks, calculates the location score so that the degree of match of the user U's location with respect to the location of being on the railway tracks is high. Note that, although the degree of match with the railway tracks is calculated here, the present invention is not limited thereto, and the degree of match with the position of any type of structure or natural object may be calculated.

The environment identification unit 44 calculates a location score for the subcategory of sounds on a train based on the ambient sounds acquired by the microphone 20B. The location score for the subcategory of sounds on a train can be a numerical value indicating the degree of match between the ambient sounds and the sounds on a train. The method of calculating the location score for the subcategory of sounds on a train can be arbitrary, but for example, it can be determined in a similar manner to the method of determining whether or not a dangerous state is present based on the ambient sounds as described above, that is, for example, by determining whether the ambient sounds are a specific type of sound. Note that, although the degree of match to sounds on a train is calculated here, the invention is not limited to this and the degree of match with sounds in any location can also be calculated.

In this way, the environment determination unit 44 can be said to set information indicating the location of the user U (here, the location score) based on the surrounding image, surrounding sound, and location information of the information providing device 10. However, the environment determination unit 44 is not limited to using the surrounding image, surrounding sound, and location information of the information providing device 10 to set the information indicating the location of the user U, and may use any environmental information, for example, at least one of the surrounding image, surrounding sound, and location information of the information providing device 10.

(Motion Score)
The environment determination unit 44 calculates a motion score as an environment score for a category of the motion of the user U. In other words, the motion score is information indicating the motion of the user U, and can be said to be information indicating, as a numerical value, how the user U is moving. The environment determination unit 44 calculates the motion score based on environmental information related to the motion of the user U, among multiple types of environmental information. Examples of environmental information related to the motion of the user U include acceleration information acquired by the acceleration sensor 20D.

5, the category of user U's movement includes a subcategory of "moving". The environment identification unit 44 calculates a location score for the subcategory of "moving" based on the acceleration information of the information providing device 10 acquired by the acceleration sensor 20D. The movement score for the subcategory of "moving" can be a numerical value indicating the degree of agreement between the current situation of the user U and the fact that the user U is moving. The calculation method of the movement score for the subcategory of "moving" may be arbitrary, but for example, the movement score may be calculated from a change in acceleration over a predetermined period. For example, when there is a change in acceleration over a predetermined period, the movement score is calculated so that the degree of agreement with the fact that the user U is moving is high. In addition, for example, the position information of the information providing device 10 may be acquired, and the movement score may be calculated based on the degree of change in position over a predetermined period. In this case, the speed can be predicted from the amount of change in position over a predetermined period, and the means of transportation, such as a vehicle or walking, can also be identified. Note that, although the degree of agreement with the fact that the user U is moving is calculated here, the invention is not limited thereto, and the degree of agreement with the fact that the user U is moving at a predetermined speed may be calculated.

In this way, the environment determination unit 44 can be said to set information indicating the movement of the user U (here, the movement score) based on the acceleration information of the information providing device 10 and the position information of the information providing device 10. However, the environment determination unit 44 is not limited to using acceleration information and position information to set the information indicating the movement of the user U, and may use any environmental information, for example, at least one of acceleration information and position information.

(Safety Score)
The environment determination unit 44 calculates a safety score as an environment score for the category of safety of the user U. That is, the safety score is information indicating the safety of the user U, and can be said to be information indicating, as a numerical value, whether the user U is in a safe environment. The environment determination unit 44 calculates the safety score based on environmental information related to the safety of the user U among multiple types of environmental information. The environmental information related to the safety of the user U includes a surrounding image acquired by the camera 20A, a surrounding sound acquired by the microphone 20B, light intensity information detected by the optical sensor 20F, surrounding temperature information detected by the temperature sensor 20G, and surrounding humidity information detected by the humidity sensor 20H.

5, the category of safety for the user U includes a subcategory of "bright," a subcategory of "appropriate amounts of infrared rays and ultraviolet rays," a subcategory of "appropriate temperature," a subcategory of "appropriate humidity," and a subcategory of "presence of dangerous objects." The environment identification unit 44 calculates a safety score for the "bright" subcategory based on the intensity of the surrounding visible light acquired by the light sensor 20F. The safety score for the "bright" subcategory can be a numerical value indicating the degree of agreement of the surrounding brightness with respect to sufficient brightness. The method of calculating the safety score for the "bright" subcategory may be any method, and may be calculated based on the intensity of visible light detected by the light sensor 20F, for example. Also, for example, the safety score for the "bright" subcategory may be calculated based on the luminance of an image captured by the camera 20A. Note that, although the degree of agreement with sufficient brightness is calculated here, the invention is not limited thereto, and the degree of agreement with any degree of brightness may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of appropriate amounts of infrared and ultraviolet light based on the intensities of infrared and ultraviolet light in the surroundings acquired by the optical sensor 20F. The safety score for the subcategory of appropriate amounts of infrared and ultraviolet light can be a numerical value indicating the degree of agreement between the intensities of infrared and ultraviolet light in the surroundings and the appropriate intensities of infrared and ultraviolet light. The method of calculating the safety score for the subcategory of appropriate amounts of infrared and ultraviolet light may be arbitrary, and may be calculated, for example, based on the intensities of infrared and ultraviolet light detected by the optical sensor 20F. Note that, although the degree of agreement with the appropriate intensities of infrared and ultraviolet light is calculated here, the invention is not limited thereto, and the degree of agreement with any intensity of infrared and ultraviolet light may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of suitable temperature based on the surrounding temperature acquired by the temperature sensor 20G. The safety score for the subcategory of suitable temperature can be a numerical value indicating the degree of agreement of the surrounding temperature with the suitable temperature. The method of calculating the safety score for the subcategory of suitable temperature can be any method, but for example, it can be calculated based on the surrounding temperature detected by the temperature sensor 20G. Note that, although the degree of agreement with the suitable temperature is calculated here, it is not limited thereto, and the degree of agreement with any temperature can be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of suitable humidity based on the surrounding humidity acquired by the humidity sensor 20H. The safety score for the subcategory of suitable humidity can be a numerical value indicating the degree of agreement of the surrounding humidity with the suitable humidity. The method of calculating the safety score for the subcategory of suitable humidity may be arbitrary, and may be calculated, for example, based on the surrounding humidity detected by the humidity sensor 20H. Note that, although the degree of agreement with the suitable humidity is calculated here, this is not limiting, and the degree of agreement with any humidity may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of dangerous object based on the surrounding image acquired by the camera 20A. The safety score for the subcategory of dangerous object is a numerical value indicating the degree of agreement with the presence of a dangerous object. The method of calculating the safety score for the subcategory of dangerous object may be any, but may be determined, for example, in a manner similar to the method of determining whether a dangerous state is present based on the surrounding image as described above, that is, by determining whether the object included in the surrounding image is a specific object. Furthermore, the environment identification unit 44 calculates a safety score for the subcategory of dangerous object based on the surrounding sound acquired by the microphone 20B. The method of calculating the safety score for the subcategory of dangerous object may be any, but may be determined, for example, in a manner similar to the method of determining whether a dangerous state is present based on the surrounding sound as described above, that is, by determining whether the surrounding sound is a specific type of sound.

(An example of an environmental score)
5 illustrates examples of environment scores calculated for environments D1 to D4. Environments D1 to D4 each indicate a case where user U is in a different environment, and an environment score is calculated for each category (subcategory) in each environment.

Note that the types of environmental categories and subcategories shown in FIG. 5 are examples, and the environmental score values for environments D1 to D4 are also examples. Furthermore, by expressing the information indicating user U's environment as a numerical value such as an environmental score, the information providing device 10 can take errors into account and more accurately estimate user U's environment. In other words, the information providing device 10 can accurately estimate user U's environment by classifying environmental information into one of three or more degrees (here, environmental scores). However, the information indicating user U's environment that the information providing device 10 sets based on environmental information is not limited to a value such as an environmental score, and may be data of any type, for example, information indicating a binary choice of either Yes or No.

(Determination of environmental patterns)
The information providing device 10 calculates various environmental scores by the above-described method in steps S16 to S22 shown in Fig. 4. As shown in Fig. 4, after calculating the environmental scores, the information providing device 10 determines an environmental pattern indicating the environment in which the user U is placed based on each environmental score by the environmental identification unit 44 (step S24). That is, the environmental identification unit 44 determines what kind of environment the user U is in based on the environmental scores. While the environmental information and the environmental score are information indicating some elements of the user U's environment detected by the environmental sensor 20, the environmental pattern can be said to be an index indicating the environment comprehensively, which is set based on the information indicating those some elements.

FIG. 6 is a table showing an example of an environmental pattern. In this embodiment, the environmental identification unit 44 selects an environmental pattern that matches the environment in which the user U is located from among the environmental patterns corresponding to various environments based on the environmental score. In this embodiment, for example, the specification setting database 30C records correspondence information (table) that associates the environmental score value with the environmental pattern. The environmental identification unit 44 determines the environmental pattern based on the environmental information and the correspondence information. Specifically, the environmental identification unit 44 selects an environmental pattern that corresponds to the calculated environmental score value from the correspondence information and selects it as the environmental pattern to be adopted. In the example of FIG. 6, the environmental pattern PT1 indicates that the user U is sitting in a train, the environmental pattern PT2 indicates that the user U is walking on the sidewalk, the environmental pattern PT3 indicates that the user U is walking on a dark sidewalk, and the environmental pattern PT4 indicates that the user U is shopping.

5 and 6, in environment D1, the environmental score for "standing" is 10, and the environmental score for "face orientation horizontal" is 100, so it can be predicted that user U is sitting with his/her face facing almost horizontally. Also, the environmental score for "inside the train" is 90, the environmental score for "on the tracks" is 100, and the environmental score for "train interior sounds" is 90, so it can be seen that user U is inside a train. Also, the environmental score for "moving" is 100, so it can be seen that user U is moving at a constant speed or with acceleration. Also, the environmental score for "bright" is 50, so it can be seen that it is darker than outside because it is inside a train. Also, the environmental scores for "appropriate amount of infrared rays and ultraviolet rays," "appropriate temperature," and "appropriate humidity" are 100, so it can be said to be safe. Also, the environmental score for "presence of dangerous objects" is 10 for visuals and 20 for sounds, so this is also considered to be safe. That is, in environment D1, it can be estimated from each environmental score that user U is moving on a train, sitting in a seat, and is in a safe and comfortable situation, and the environmental pattern of environment D1 is environment pattern PT1, which indicates that user U is sitting on a train.

5 and 6, in environment D2, the environmental score for "standing" is 10, and the environmental score for "face orientation horizontal" is 90, so it can be predicted that user U is sitting with his/her face facing almost horizontally. Also, the environmental score for "inside the train" is 0, the environmental score for "on the tracks" is 0, and the environmental score for "train interior sounds" is 10, so it can be seen that user U is not inside the train. Although not shown here, in environment D2, it can be confirmed that user U is on the road based on the environmental scores of the locations. Also, since the environmental score for "moving" is 100, it can be seen that user U is moving at a constant speed or with acceleration. Also, the environmental score for "bright" is 100, so it can be seen that it is bright outdoors. Also, "appropriate amount of infrared rays and ultraviolet rays" is 80, so it can be seen that there is a slight influence of ultraviolet rays and the like. Also, the environmental scores for "suitable temperature" and "suitable humidity" are 100, so it can be said to be safe. Also, the environmental score for "dangerous objects" is 10 for visuals and 20 for sounds, so this is also considered to be safe. That is, in environment D2, it can be inferred from each environmental score that user U is walking on the sidewalk, it is bright outdoors, and no dangerous objects are found, and the environmental pattern of environment D2 is set to environmental pattern PT2, which indicates walking on the sidewalk.

5 and 6, in environment D3, the environmental score for "standing" is 0, and the environmental score for "face orientation horizontal" is 90, so it can be predicted that user U is sitting with his/her face facing almost horizontally. Also, the environmental score for "inside the train" is 5, the environmental score for "on the tracks" is 0, and the environmental score for "train noise" is 5, so it can be seen that user U is not inside the train. Although not shown here, in environment D3, it can be confirmed that user U is on the road based on the environmental scores of the locations. Also, the environmental score for "moving" is 100, so it can be seen that user U is moving with a constant speed or acceleration. Also, the environmental score for "bright" is 10, so it can be seen that it is a dark environment. Also, the environmental score for "appropriate amount of infrared and ultraviolet rays" is 100, so it can be seen that it is safe. Also, the environmental score for "appropriate temperature" is 75, so it can be said that it is hotter or colder than normal. Furthermore, the environmental score for "dangerous object present" is 90 visually and 80 audioly, so it is clear that something is making a sound and approaching. Although not shown, the object can be determined from the sound and image, and in this case, it can be determined that a car is approaching from the front and the sound is the sound of a car engine. That is, in environment D3, it can be estimated from each environmental score that user U is walking on the sidewalk, it is dark outdoors, and a vehicle is approaching as a dangerous object, and the environmental pattern of environment D3 is environmental pattern PT3, which indicates walking on a dark sidewalk.

5 and 6, in environment D4, the environmental score for "standing" is 0, and the environmental score for "face orientation horizontal" is 90, so it can be predicted that user U is sitting with his face facing almost horizontally. Also, the environmental score for "inside the train" is 20, the environmental score for "on the tracks" is 0, and the environmental score for "train noise" is 5, so it can be seen that user U is not inside the train. Although not shown here, in environment D3, it can be confirmed that user U is in a shopping center based on the environmental scores of the locations. Also, since the environmental score for "moving" is 80, it can be seen that user U is moving slowly. Also, the environmental score for "bright" is 70, so it can be predicted that it is relatively bright, but the brightness is about the same as indoor lighting. Also, "appropriate amount of infrared and ultraviolet rays" is 100, so it can be seen that it is safe. Also, the environmental score for "appropriate temperature" is 100, which is comfortable, but since the environmental score for "appropriate humidity" is 90, it can be said that it is not necessarily comfortable. Furthermore, the environmental score for "dangerous objects present" is 10 for visuals and 20 for sound, so this is also considered safe. That is, in environment D4, it can be estimated from each environmental score that user U is walking through a shopping center, the surroundings are relatively bright, and there are no dangerous objects, and the environmental pattern of environment D4 is environmental pattern PT4, which indicates that the user is shopping.

(Setting the target device and standard output specifications)
After selecting the environmental pattern, the information providing device 10, as shown in FIG. 4, selects the target devices to be operated from the output unit 26 based on the environmental pattern using the output selection unit 48 and the output specification determination unit 50, and sets the standard output specifications (step S26).

(Target device settings)
As described above, the target device is a device to be operated among the output unit 26, and in this embodiment, the output selection unit 48 selects the target device from among the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C based on environmental information, more preferably based on an environmental pattern. Since the environmental pattern is information indicating the current environment of the user U, by selecting the target device based on the environmental pattern, it is possible to select an appropriate sensory stimulus according to the current environment of the user U.

For example, the output selection unit 48 may determine whether there is a high need for the user U to visually check the surrounding environment based on the environmental information, and may determine whether to select the display unit 26A as the target device based on the result of the determination. In this case, for example, the output specification determination unit 50 may select the display unit 26A as the target device when the need to visually check the surrounding environment is lower than a predetermined standard, and may not select the display unit 26A as the target device when the need is equal to or greater than the predetermined standard. The determination of whether there is a high need for the user U to visually check the surrounding environment may be made arbitrarily, but may be determined to be equal to or greater than the predetermined standard when, for example, the user U is on the move or there is a dangerous object.

For example, the output selection unit 48 may determine whether the user U has a high need to hear surrounding sounds based on the environmental information, and may determine whether to select the audio output unit 26B as the target device based on the result of the determination. In this case, for example, the output specification determination unit 50 selects the audio output unit 26B as the target device when the need to hear surrounding sounds is lower than a predetermined standard, and does not select the audio output unit 26B as the target device when the need is equal to or greater than the predetermined standard. The determination of whether the user U has a high need to hear surrounding sounds may be made arbitrarily, but may be determined to be equal to or greater than the predetermined standard when, for example, the user U is on the move or there is a dangerous object.

For example, the output selection unit 48 may determine whether the user U may receive tactile stimulation based on the environmental information, and may determine whether the sensory stimulation output unit 26C is to be the target device based on the result of the determination. In this case, for example, if the output specification determination unit 50 determines that the user U may receive tactile stimulation, it selects the sensory stimulation output unit 26C as the target device, and if it determines that the user U may not receive tactile stimulation, it does not select the sensory stimulation output unit 26C as the target device. The determination of whether the user U may receive tactile stimulation may be made arbitrarily, but it may be determined that the user U may not receive tactile stimulation when, for example, the user U is moving or there is a dangerous object.

The above describes an example of a method for selecting a target device by the output selection unit 48. More specifically, it is preferable for the output selection unit 48 to select a target device based on a table showing the relationship between the environmental pattern and the target device, as shown in FIG. 8 described below.

(Setting the standard output specifications)
Further, the output specification determination unit 50 determines the standard output specification, which is the standard output specification, based on the environmental information, more preferably based on the environmental pattern. The output specification is an index indicating how the stimulus output by the output unit 26 is output. For example, the output specification of the display unit 26A indicates how the output content image PS is displayed, and can be rephrased as the display specification. Examples of the output specification of the display unit 26A include the size (area) of the content image PS, the transparency of the content image PS, and the display contents (contents) of the content image PS. The size of the content image PS refers to the area occupied by the content image PS within the screen of the display unit 26A. The transparency of the content image PS refers to the degree of transparency of the content image PS. The higher the transparency of the content image PS, the more the light entering the user U's eyes as the background image PA passes through the content image PS, and the background image PA superimposed on the content image PS is more clearly visible. In this way, it can be said that the output specification determination unit 50 determines the size, transparency, and display content of the content image PS as the output specification of the display unit 26A based on the environmental pattern. However, the output specification of the display unit 26A is not limited to all of the size, transparency, and display content of the content image PS. For example, the output specification of the display unit 26A may be at least one of the size, transparency, and display content of the content image PS, or may be something else.

For example, the output specification determination unit 50 may determine whether the user U has a high need to visually recognize the surrounding environment based on the environmental information, and may determine the output specification (standard output specification) of the display unit 26A based on the result of the determination. In this case, the output specification determination unit 50 determines the output specification (standard output specification) of the display unit 26A so that the visibility of the environmental image PM increases as the need to visually recognize the surrounding environment increases. The visibility here refers to the ease of visual recognition of the environmental image PM. For example, the output specification determination unit 50 may reduce the size of the content image PS, increase the transparency of the content image PS, or increase the restrictions on the display content of the content image PS, or may combine these, as the need to visually recognize the surrounding environment increases. In addition, when increasing the restrictions on the display content of the content image PS, for example, the delivery image may be excluded from the display content, and the display content may be at least one of a navigation image and a notification image. Furthermore, the user U may decide at his/her own discretion whether there is a high need for visually checking the surrounding environment, but examples of such decisions include when the user U is on the move or when there is a dangerous object.

Figure 7 is a schematic diagram illustrating an example of the output specification levels of a content image. In this embodiment, the output specification determination unit 50 may classify the output specifications of the content image PS into levels and select the output specification level based on the environmental information. In this case, the output specifications of the content image PS are set so that the visibility of the environmental image PM differs for each level. In this embodiment, each level of the output specification is set so that the higher the level, the stronger the output stimulus and the lower the visibility of the environmental image PM. Therefore, the output specification determination unit 50 sets the output specification level higher the lower the need to view the surrounding environment. In the example of Figure 7, at level 0, the content image PS is not displayed and only the environmental image PM is visible, so the visibility of the environmental image PM is the highest.

As shown in FIG. 7, at level 1, a content image PS is displayed, but the display contents of the content image PS are restricted. Here, the delivery image is excluded from the display contents, and the display contents are at least one of a navigation image and a notification image. Also, at level 1, the size of the content image PS is set small. At level 1, the content image PS is displayed superimposed on the environmental image PM only when it becomes necessary to display a navigation image and a notification image. Therefore, the visibility of the environmental image PM at level 1 is lower than at level 0 because the content image PS is displayed, but is higher because the display contents of the content image PS are restricted.

As shown in FIG. 7, at level 2, the display contents of the content image PS are not restricted, but the size of the content image PS is restricted and set small. The visibility of the environmental image PM at level 2 is lower than at level 1 because the display contents are not restricted.

As shown in FIG. 7, at level 3, the display content and size of the content image PS are not restricted, and the content image PS is displayed, for example, on the entire screen of the display unit 26A. However, at level 3, the transparency of the content image PS is restricted and set to a high level. Therefore, at level 3, the semi-transparent content image PS and the environmental image PM superimposed on the content image PS are visible. The visibility of the environmental image PM at level 3 is lower than at level 2 because the size of the content image PS is not restricted.

As shown in FIG. 7, at level 4, the display content, size, and transparency of the content image PS are not limited, and the content image PS is displayed, for example, so that the transparency is zero across the entire screen of the display unit 26A. At level 4, the transparency of the content image PS is zero (opaque), so the environmental image PM is not visible, and only the content image PS is visible. Therefore, the visibility of the environmental image PM at level 4 is at its lowest. Note that at level 4, for example, an image that falls within the field of view of the user U may be displayed as the environmental image PM in a partial area of the screen of the display unit 26A.

The output specifications of the display unit 26A have been described above, but the output specification determination unit 50 also determines the output specifications of the audio output unit 26B and the sensory stimulation output unit 26C. Examples of the output specifications (audio specifications) of the audio output unit 26B include volume, the presence or absence of sound, and the degree of sound. Sound refers to special effects such as surround and stereo sound fields. The louder the volume or the louder the sound, the stronger the degree of auditory stimulation for the user U. For example, the output specification determination unit 50 may determine whether the user U needs to hear surrounding sounds based on the environmental information, and determine the output specifications (reference output specifications) of the audio output unit 26B based on the determination result. In this case, the output specification determination unit 50 determines the output specifications (reference output specifications) of the audio output unit 26B so that the lower the need to hear surrounding sounds, the louder the volume or the louder the sound. The user U may decide whether or not there is a high need for listening to surrounding sounds at will, but examples of such decisions include when the user U is moving or when there is a dangerous object. Note that the output specification determination unit 50 may set a level for the output specification of the audio output unit 26B in the same way as for the output specification of the display unit 26A.

Examples of the output specifications of the sensory stimulation output unit 26C include the strength of the tactile stimulation and the frequency of outputting the tactile stimulation. The stronger the strength or frequency of the tactile stimulation, the stronger the degree of the tactile stimulation to the user U. For example, the output specification determination unit 50 may determine whether the user U is in a state suitable for receiving the tactile stimulation based on the environmental information, and determine the output specifications (standard output specifications) of the sensory stimulation output unit 26C based on the determination result. In this case, the output specification determination unit 50 determines the output specifications (standard output specifications) of the sensory stimulation output unit 26C so that the strength of the tactile stimulation is higher or the frequency of the tactile stimulation is higher as the user U is more suitable for receiving the tactile stimulation. The determination of whether the user U is in a state suitable for receiving the tactile stimulation may be made arbitrarily, but examples include a case where the user U is moving or there is a dangerous object. The output specification determination unit 50 may set a level for the output specifications of the sensory stimulation output unit 26C as well as the output specifications of the display unit 26A.

(Specific examples of target devices and standard output specification settings)
It is more preferable that the output selection unit 48 and the output specification determination unit 50 determine the target device and the reference output specification based on the relationship between the environment pattern, the target device, and the reference output specification. FIG. 8 is a table showing the relationship between the environment pattern, the target device, and the reference output specification. The output selection unit 48 and the output specification determination unit 50 determine the target device and the reference output specification based on the relationship information showing the relationship between the environment pattern, the target device, and the reference output specification. The relationship information is information (table) in which the environment pattern, the target device, and the reference output specification are associated and stored, and is stored in, for example, the specification setting database 30C. In the relationship information, the reference output specification is set for each type of the output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the sensory stimulation output unit 26C in this case. The output selection unit 48 and the output specification determination unit 50 determine the target device and the reference output specification based on the relationship information and the environment pattern set by the environment specification unit 44. Specifically, the output selection unit 48 and the output specification determination unit 50 read out the relationship information, select from the relationship information the target equipment and the standard output specification that correspond to the environmental pattern set by the environment determination unit 44, and determine the target equipment and the standard output specification.

In the example of FIG. 8, for the environmental pattern PT1 in which the subject is sitting in a train, the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C are all set as target devices, and the standard output specification level of these units is assigned to 4. Note that the higher the level, the higher the output stimulus. For the environmental pattern PT2 in which the subject is walking on the sidewalk, the situation is almost safe and comfortable, but since the subject is walking, attention to the front is considered to be necessary, so the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C are all set as target devices, and the standard output specification level of these units is assigned to 3. For the environmental pattern PT3 in which the subject is walking on a dark sidewalk, the situation is not safe, and the subject must be gazing forward and be able to hear external sounds well, so the audio output unit 26B and the sensory stimulus output unit 26C are set as target devices, and the standard output specification levels of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C are assigned to 0, 2, and 2, respectively. Furthermore, for environmental pattern PT4, which is assumed to be shopping, it is generally safe, but because it is a shopping center, it is assumed that the provision of distracting information is unnecessary, so the display unit 26A, the audio output unit 26B, and the sensory stimulation output unit 26C are all set as target devices, and their standard output specification level is assigned to 2. However, the assignment of target devices and standard output specifications for each environmental pattern in FIG. 8 is merely an example, and may be set as appropriate.

In this manner, in this embodiment, the information providing device 10 sets the target device and the reference output specifications based on a preset relationship between the environmental pattern and the target device and the reference output specifications. However, the method of setting the target device and the reference output specifications is not limited to this, and the information providing device 10 may set the target device and the reference output specifications in any manner based on the environmental information detected by the environmental sensor 20. Furthermore, the information providing device 10 is not limited to selecting both the target device and the reference output specifications based on the environmental information, and may select at least one of the target device and the reference output specifications.

(Acquisition of Biometric Information)
As shown in FIG. 4, the information providing device 10 acquires the bioinformation of the user U detected by the biosensor 22 by the bioinformation acquiring unit 42 (step S28). The bioinformation acquiring unit 42 acquires the pulse wave information of the user U from the pulse wave sensor 22A, and acquires the brain wave information of the user U from the brain wave sensor 22B. FIG. 9 is a graph showing an example of a pulse wave. As shown in FIG. 9, the pulse wave has a waveform in which a peak called an R wave WR appears at a predetermined time. The heart is controlled by the autonomic nervous system, and the pulse rate is controlled by generating an electrical signal at the cellular level that triggers the heart to move. Usually, the pulse rate increases when adrenaline is secreted due to the stimulation of the sympathetic nerves, and decreases when acetylcholine is secreted due to the stimulation of the parasympathetic nerves. According to Nobuyuki Ueda, "Evaluation of Diabetic Autonomic Neuropathy Using Power Spectral Analysis of Electrocardiogram R-R Intervals" (Diabetes 35 (1): 17-23, 1992), the function of the autonomic nerves can be understood by examining the variation of the R-R interval in the time waveform of the pulse wave as shown in the example of FIG. 9. The R-R sensation is the interval between consecutive R waves WR in a time series. At the cellular level, an electrocardiogram is a repetition of depolarization/action potential and repolarization/resting potential, and an electrocardiogram can be detected by detecting this electrical activity from the surface of the body. The pulse wave is very fast and is transmitted throughout the body almost simultaneously with the heartbeat, so it can be said that the heartbeat is synchronized with the pulse wave. Since the pulse wave of the heart and the R wave of the electrocardiogram are synchronized, the R-R interval of the pulse wave can be considered to be equivalent to the R-R interval of the electrocardiogram. Fluctuations in the R-R interval of the pulse wave can be considered as time derivatives, so by calculating the derivatives and detecting the magnitude of the fluctuations, it is possible to predict to a certain extent the degree of activation or calming of the autonomic nervous system of the living body, i.e., irritation due to mental disturbance, discomfort on a crowded train, stress that occurs over a relatively short period of time, etc., almost regardless of the wearer's will.

On the other hand, by detecting the basic rhythmic (background brain wave) activity that appears throughout the brain, such as alpha and beta waves, and detecting their amplitude, it is possible to predict to some extent whether the activity of the brain as a whole is increasing or decreasing. For example, the level of activity in the prefrontal cortex of the brain can reveal the level of attention, such as the level of interest shown in an object that is visually stimulated.

(Identifying user status and calculating output specification correction degree)
4, after acquiring the biometric information, the information providing device 10 causes the user state identifying unit 46 to identify the user state indicating the mental state of the user U based on the biometric information of the user U, and calculates an output specification correction degree based on the user state (step S30). The output specification correction degree is a value for correcting the standard output specification set by the output specification determining unit 50, and the final output specification is determined based on the standard output specification and the output specification correction degree.

Figure 10 is a table showing an example of the relationship between the user state and the output specification correction degree. In this embodiment, the user state identification unit 46 identifies the brain activity of the user U as the user state based on the brain wave information of the user U. The user state identification unit 46 may identify the brain activity of the user U by any method based on the brain wave information of the user U, for example, the brain activity may be identified from a specific frequency range for the waveforms of alpha waves and beta waves. In this case, for example, the user state identification unit 46 performs a fast Fourier transform on the time waveform of the brain wave to calculate the power spectrum amount of the high frequency part of the alpha wave (for example, 10 Hz to 11.75 Hz). When the power spectrum amount of the high frequency part of the alpha wave is large, it can be expected that the user is very concentrated while being relaxed, so the user state identification unit 46 determines that the brain activity is higher as the power spectrum amount of the high frequency part of the alpha wave is large. The user state identification unit 46 defines the brain activity level when the power spectrum amount of the high frequency part of α waves is within a predetermined numerical range as VA3, defines the brain activity level when the power spectrum amount of the high frequency part of α waves is within a predetermined numerical range lower than the numerical range when the brain activity level is VA3 as VA2, and defines the brain activity level when the power spectrum amount of the high frequency part of α waves is within a predetermined numerical range lower than the numerical range when the brain activity level is VA2 as VA1. Here, the brain activity levels are considered to be increasing in the order of VA1, VA2, and VA3. Note that the greater the power spectrum amount of the high frequency component of β waves (e.g., 18 Hz to 29.75 Hz), the more likely it is that the user is psychologically "alert" or "agitated," so the power spectrum amount of the high frequency component of β waves may also be used to identify the brain activity level.

The user state identification unit 46 determines the output specification correction degree based on the brain activity of the user U. In this embodiment, the output specification correction degree is determined based on the output specification correction degree relationship information indicating the relationship between the user state (brain activity in this example) and the output specification correction degree. The output specification correction degree relationship information is information (table) in which the user state and the output specification correction degree are associated and stored, and is stored, for example, in the specification setting database 30C. In the output specification correction degree relationship information, the output specification correction degree is set for each type of the output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the sensory stimulation output unit 26C in this case. The user state identification unit 46 determines the output specification correction degree based on this output specification correction degree relationship information and the identified user state. Specifically, the user state identification unit 46 reads out the output specification correction degree relationship information, selects the output specification correction degree associated with the set brain activity of the user U from the output specification correction degree relationship information, and determines the output specification correction degree. In the example of FIG. 10, the output specification correction degree of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C is set to -1 for the brain activity level VA3, the output specification correction degree of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C is set to 0 for the brain activity level VA2, and the output specification correction degree of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C is set to 1 for the brain activity level VA1. The output specification correction degree here is set to a value that increases the output specification as the value increases. In other words, the user state identification unit 46 sets the output specification correction degree so that the output specification is increased as the brain activity level decreases. Note that increasing the output specification here refers to strengthening the sensory stimulus, and the same applies hereinafter. The value of the output specification correction degree in FIG. 10 is an example and may be set as appropriate.

Furthermore, the user state identification unit 46 identifies the mental stability of the user U as the user state based on the pulse wave information of the user U. In this embodiment, the user state identification unit 46 calculates the fluctuation value of the interval length between successive R waves WH in a time series, i.e., the differential value of the R-R interval, from the brain wave information of the user U, and identifies the brain activity of the user U based on the differential value of the R-R interval. The user state identification unit 46 identifies the mental stability of the user U as being higher when the differential value of the R-R interval is smaller, i.e., the interval length between the R waves WH does not fluctuate. In the example of FIG. 10, the user state identification unit 46 classifies the mental stability of the user U into one of three types, VB3, VB2, and VB1, based on the pulse wave information of the user U. The user state identification unit 46 defines the mental stability when the differential value of the R-R interval is within a predetermined numerical range as VB3, defines the mental stability when the differential value of the R-R interval is within a predetermined numerical range higher than the numerical range when the mental stability is VB3 as VB2, and defines the mental stability when the differential value of the R-R interval is within a predetermined numerical range lower than the numerical range when the mental stability is VB2 as VB1. Note that the mental stability is considered to be increasing in the order of VB1, VB2, and VB3.

The user state identification unit 46 determines the output specification correction degree based on the output specification correction degree related information and the identified mental stability. Specifically, the user state identification unit 46 reads the output specification correction degree related information, selects the output specification correction degree associated with the set mental stability of the user U from the output specification correction degree related information, and determines the output specification correction degree. In the example of FIG. 10, for mental stability VB3, the output specification correction degrees of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C are each set to 1, for mental stability VB2, the output specification correction degrees of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C are each set to 0, and for mental stability VB1, the output specification correction degrees of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C are each set to -1. That is, the user state identification unit 46 sets the output specification correction degree so that the higher the mental stability, the higher the output specification (sensory stimulus). Note that the value of the output specification correction degree in FIG. 10 is an example and may be set as appropriate.

In this way, the user state identification unit 46 sets the output specification correction degree based on the relationship between the user state and the output specification correction degree that is set in advance. However, the method of setting the output specification correction degree is not limited to this, and the information providing device 10 may set the output specification correction degree by any method based on the bioinformation detected by the biosensor 22. In addition, the information providing device 10 calculates the output specification correction degree using both the brain activity level identified from the brain wave and the mental stability identified from the pulse wave, but is not limited to this. For example, the information providing device 10 may calculate the output specification correction degree using one of the brain activity level identified from the brain wave and the mental stability identified from the pulse wave. In addition, the information providing device 10 treats the bioinformation as a numerical value, and by estimating the user state based on the bioinformation, it is possible to take into account errors in the bioinformation, etc., and it is possible to more accurately estimate the psychological state of the user U. In other words, it can be said that the information providing device 10 can accurately estimate the psychological state of the user U by classifying the bioinformation and the user state based on the bioinformation into one of three or more degrees. However, the information providing device 10 is not limited to classifying the biometric information or the user state based on the biometric information into three or more degrees, and may treat it as information indicating, for example, a binary option of either Yes or No.

(Generation of output restriction necessity information)
As shown in FIG. 4, the information providing device 10 generates output restriction necessity information based on the biometric information of the user U by the user state identification unit 46 (step S32). FIG. 11 is a table showing an example of output restriction necessity information. The output restriction necessity information is information indicating whether or not output restriction of the output unit 26 is necessary, and can be said to be information indicating whether or not the operation of the output unit 26 is permitted. The output restriction necessity information is generated for each output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the sensory stimulation output unit 26C. In other words, the user state identification unit 46 generates output restriction necessity information indicating whether or not the operation of each of the display unit 26A, the audio output unit 26B, and the sensory stimulation output unit 26C is permitted based on the biometric information. More specifically, the user state identification unit 46 generates output restriction necessity information based on both the biometric information and the environmental information. The user state identification unit 46 generates output restriction necessity information based on the user state set based on the biometric information and the environmental score calculated based on the environmental information. In the example of FIG. 11, the user state identification unit 46 generates output restriction necessity information based on the brain activity level as the user state and the location score for the subcategory of being on the railroad as the environment score. In the example of FIG. 11, the user state identification unit 46 generates output restriction necessity information that prohibits the use of the display unit 26A when the location score for the subcategory of being on the railroad is 100 and the brain activity level is VA3 or VA2, and satisfies a first condition. Note that the first condition is not limited to the case where the location score for the subcategory of being on the railroad is 100 and the brain activity level is VA3 or VA2, and may be, for example, the case where the position of the information providing device 10 is within a predetermined area and the brain activity level is equal to or lower than a predetermined brain activity level threshold. The predetermined area here may be, for example, a railroad or a roadway.

In the example of FIG. 11, the user state identification unit 46 generates output restriction necessity information based on the brain activity level as the user state and the motion score for the subcategory of moving as the environment score. In the example of FIG. 11, the user state identification unit 46 generates output restriction necessity information that does not permit use of the display unit 26A when the motion score for the subcategory of moving is 0 and the brain activity level is VA3 or VA2, and the first condition is satisfied. Note that the second condition is not limited to the case where the motion score for the subcategory of moving is 0 and the brain activity level is VA3 or VA2, and may be, for example, the case where the amount of change per unit time of the position of the information providing device 10 is equal to or less than a predetermined change threshold and the brain activity level is equal to or less than a predetermined brain activity threshold.

In this way, when the biometric information and the environmental information satisfy a specific relationship, in this case when the user state and the environmental score satisfy at least one of the first condition and the second condition, the user state identification unit 46 generates output restriction necessity information that disallows use of the display unit 26A. On the other hand, when the user state and the environmental score do not satisfy the first condition and do not satisfy the second condition, the user state identification unit 46 does not generate output restriction necessity information that disallows use of the display unit 26A, but generates output restriction necessity information that permits use of the display unit 26A. However, generating output restriction necessity information is not a required process.

(Getting content images)
4, the information providing device 10 acquires image data of the content image PS by the content image acquisition unit 52 (step S34). The image data of the content image PS is image data for displaying the content (display contents) of the content image. The content image acquisition unit 52 acquires the image data of the content image from an external device via the content image receiving unit 28A.

The content image acquisition unit 52 may acquire image data of a content image of a content (display content) corresponding to the position (earth coordinates) of the information providing device 10 (user U). The position of the information providing device 10 is specified by the GNSS receiver 20C. For example, when the user U is located within a predetermined range of a certain position, the content image acquisition unit 52 receives content related to that position. In principle, the display of the content image PS can be controlled by the user U's will, but if the display is set to be possible, it is not known when, where, and at what timing it will be displayed, which is convenient but may also be a nuisance. Therefore, the specification setting database 30C may record information indicating whether the content image PS set by the user U can be displayed and the display specifications. The content image acquisition unit 52 reads out this information from the specification setting database 30C and controls the acquisition of the content image PS based on this information. The position information and the specification setting database 30C may also be written with the same information on a site on the Internet, and the content image acquisition unit 52 may control the acquisition of the content image PS while checking the contents. Note that step S34 of acquiring image data of the content image PS does not necessarily have to be performed before step S36 described below, but may be performed at any time before step S38 described below.

The content image acquisition unit 52 may also acquire audio data and tactile stimulation data related to the content image PS along with the image data of the content image PS. The audio output unit 26B outputs the audio data related to the content image PS as audio content (audio content), and the sensory stimulation output unit 26C outputs the tactile stimulation data related to the content image PS as tactile stimulation content (tactile stimulation content).

(Output specification settings)
4, the information providing device 10 determines output specifications based on the standard output specifications and the output specification correction degree by the output specification determination unit 50 (step S36). The output specification determination unit 50 corrects the standard output specifications set based on the environmental information by the output specification correction degree set based on the bioinformation, thereby determining the output specifications as the final output specifications for the output unit 26. The formula for correcting the standard output specifications by the output specification correction degree may be arbitrary.

As described above, the information providing device 10 determines the final output specifications by correcting the standard output specifications set based on the environmental information with the output specification correction degree set based on the biometric information. However, the information providing device 10 is not limited to determining the output specifications by correcting the standard output specifications with the output specification correction degree, and may determine the output specifications by any method using at least one of the environmental information and the biometric information. That is, the information providing device 10 may determine the output specifications by any method based on the environmental information and the biometric information, or may determine the output specifications by any method based on either the environmental information or the biometric information. For example, the information providing device 10 may determine the output specifications using the method in which the above-mentioned standard output specifications were determined based on the environmental information out of the environmental information and the biometric information. Also, for example, the information providing device 10 may determine the output specifications using the method in which the above-mentioned output specification correction degree was determined based on the biometric information out of the environmental information and the biometric information.

When output restriction necessity information that prohibits the use of the output unit 26 is generated in step S32, the output selection unit 48 selects the target device based on not only the environmental score but also the output restriction necessity information. That is, even if the output unit 26 was selected as the target device based on the environmental score in step S26, if the specification is prohibited in the output restriction necessity information, it is excluded from the target devices. In other words, the output selection unit 48 selects the target device based on the output restriction necessity information and the environmental information. Furthermore, since the output restriction necessity information is set based on the biometric information, it can be said that the target device is set based on the biometric information and the environmental information. However, the output selection unit 48 is not limited to setting the target device based on both the biometric information and the environmental information, and may select the target device based on at least one of the biometric information and the environmental information.

(Output Control)
After setting the target device and the output specifications and acquiring the image data of the content image PS, the information providing device 10 causes the target device to output based on the output specifications (step S38) via the output control unit 54, as shown in Fig. 4. The output control unit 54 does not operate the output units 26 that are not set as target devices.

For example, when the display unit 26A is set as the target device, the output control unit 54 causes the display unit 26A to display a content image PS based on the content image data acquired by the content image acquisition unit 52 in accordance with the output specifications of the display unit 26A. Since the output specifications are set based on environmental information and biometric information as described above, by displaying the content image PS in accordance with the output specifications, it is possible to display the content image PS in an appropriate manner according to the environment in which the user U is placed and the psychological state of the user U.

Furthermore, when the audio output unit 26B is the target device, the output control unit 54 causes the audio output unit 26B to output audio based on the audio data acquired by the content image acquisition unit 52 in accordance with the output specifications of the audio output unit 26B. Even in this case, for example, the higher the brain activity level of the user U or the lower the mental stability of the user U, the weaker the auditory stimulation can be made, thereby reducing the risk of the user U being annoyed by the audio when he or she is concentrating on something else or is not mentally stable. On the other hand, the lower the brain activity level of the user U or the higher the mental stability of the user U, the stronger the auditory stimulation can be made, allowing the user U to obtain information appropriately through the audio.

Furthermore, when the sensory stimulus output unit 26C is determined to be the target device, the output control unit 54 causes the sensory stimulus output unit 26C to output tactile stimuli based on the tactile stimulus data acquired by the content image acquisition unit 52 in accordance with the output specifications of the sensory stimulus output unit 26C. Even in this case, for example, the higher the brain activity level of the user U or the lower the mental stability of the user U, the weaker the tactile stimulus can be made, thereby reducing the risk of the user U being annoyed by the tactile stimulus when he or she is concentrating on something else or is not mentally stable. On the other hand, the lower the brain activity level of the user U or the higher the mental stability of the user U, the stronger the tactile stimulus can be made, allowing the user U to obtain appropriate information through the tactile stimulus.

In addition, if a dangerous state is determined in step S12 and the content of the danger notification is set, the output control unit 54 notifies the target device of the danger notification content in accordance with the set output specifications.

In this way, the information providing device 10 according to this embodiment can output sensory stimuli at an appropriate level according to the environment in which the user U is placed and the psychological state of the user U by setting the output specifications based on the environmental information and the biometric information. Furthermore, the information providing device 10 can select appropriate sensory stimuli according to the environment in which the user U is placed and the psychological state of the user U by selecting the target device to be operated based on the environmental information and the biometric information. However, the information providing device 10 is not limited to using both the environmental information and the biometric information, and may use only one of them, for example. Therefore, the information providing device 10 may select the target device and set the output specifications based on the environmental information, or may select the target device and set the output specifications based on the biometric information, for example.

(effect)
As described above, the information providing device 10 according to one aspect of the present embodiment is a device that provides information to the user U, and includes the output unit 26, the environment sensor 20, the output specification determination unit 50, and the output control unit 54. The output unit 26 includes the display unit 26A that outputs a visual stimulus, the audio output unit 26B that outputs an audio stimulus, and the sensory stimulus output unit 26C that outputs a sensory stimulus different from the visual and audio. The environment sensor 20 detects environmental information around the information providing device 10. The output specification determination unit 50 determines the output specifications of the visual stimulus, the audio stimulus, and the sensory stimulus, that is, the output specifications of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C, based on the environmental information. The output control unit 54 causes the output unit 26 to output the visual stimulus, the audio stimulus, and the sensory stimulus based on the output specification. The information providing device 10 sets output specifications for visual stimuli, auditory stimuli, and sensory stimuli based on environmental information, and thereby can output visual stimuli, auditory stimuli, and sensory stimuli in an appropriate balance according to the environment of the user U. Therefore, the information providing device 10 can provide information appropriately to the user U.

The information providing device 10 according to one aspect of this embodiment includes a plurality of environmental sensors that detect different types of environmental information, and an environmental identification unit 44. The environmental identification unit 44 identifies an environmental pattern that comprehensively indicates the current environment of the user U, based on the different types of environmental information. The output specification determination unit 50 determines the output specifications based on the environmental pattern. The information providing device 10 can provide information more appropriately according to the environment in which the user U is placed by setting the output specifications for visual stimuli, auditory stimuli, and sensory stimuli based on the environmental pattern identified from the plurality of types of environmental information.

In addition, in the information providing device 10 according to one aspect of this embodiment, the output specification determination unit 50 determines at least one of the size of the image displayed by the display unit 26A, the transparency of the image displayed by the display unit 26A, and the content (display content) of the image displayed by the display unit 26A as the output specification of the visual stimulus. By determining these as the output specification of the visual stimulus, the information providing device 10 can provide visual information more appropriately.

In addition, in the information providing device 10 according to one aspect of this embodiment, the output specification determination unit 50 determines at least one of the volume and sound of the sound output by the audio output unit 26B as the output specification of the auditory stimulus. By determining these as the output specification of the auditory stimulus, the information providing device 10 can provide auditory information more appropriately.

In addition, in the information providing device 10 according to one aspect of this embodiment, the sensory stimulus output unit 26C outputs a tactile stimulus as the sensory stimulus, and the output specification determination unit 50 determines at least one of the intensity of the tactile stimulus output by the sensory stimulus output unit 26C and the frequency of outputting the tactile stimulus as the output specification of the auditory stimulus. By determining these as the output specification of the tactile stimulus, the information providing device 10 can provide tactile information more appropriately.

The information providing device 10 according to one aspect of the present embodiment is a device for providing information to a user U, and includes an output unit 26, a biosensor 22, an output specification determination unit 50, and an output control unit 54. The output unit 26 includes a display unit 26A that outputs visual stimuli, a voice output unit 26B that outputs auditory stimuli, and a sensory stimulus output unit 26C that outputs sensory stimuli different from visual and auditory stimuli. The biosensor 22 detects bioinformation of the user U. The output specification determination unit 50 determines the output specifications of the visual, auditory, and sensory stimuli, that is, the output specifications of the display unit 26A, the voice output unit 26B, and the sensory stimulus output unit 26C, based on the bioinformation. The output control unit 54 causes the output unit 26 to output the visual, auditory, and sensory stimuli based on the output specifications. The information providing device 10 can output the visual, auditory, and sensory stimuli in an appropriate balance according to the psychological state of the user U by setting the output specifications of the visual, auditory, and sensory stimuli based on environmental information. Therefore, the information providing device 10 can provide appropriate information to the user U.

In one aspect of this embodiment, the biometric information includes information regarding the autonomic nerves of the user U, and the output specification determination unit 50 determines the output specifications based on the information regarding the autonomic nerves of the user U. The information providing device 10 can provide information more appropriately according to the psychological state of the user U by setting the output specifications for visual stimuli, auditory stimuli, and sensory stimuli based on the information regarding the autonomic nerves of the user U.

The information providing device 10 according to one aspect of the present embodiment is a device that provides information to a user U, and includes an output unit 26, an environmental sensor 20, an output selection unit 48, and an output control unit 54. The output unit 26 includes a display unit 26A that outputs visual stimuli, an audio output unit 26B that outputs auditory stimuli, and a sensory stimulus output unit 26C that outputs sensory stimuli different from visual and auditory stimuli. The environmental sensor 20 detects environmental information around the information providing device 10. The output selection unit 48 selects a target device to be used from among the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C based on the environmental information. The output control unit 54 controls the target device. The information providing device 10 can appropriately select which of the visual, auditory, and sensory stimuli to output according to the environment in which the user U is placed by selecting the target device based on the environmental information. Therefore, according to the information providing device 10, information can be appropriately provided to the user U according to the environment in which the user U is placed.

The information providing device 10 according to one aspect of this embodiment further includes a biosensor 22 that detects bioinformation of the user, and the output selection unit 48 selects a target device based on the environmental information and the bioinformation of the user U. The information providing device 10 can select an appropriate sensory stimulus according to the environment in which the user U is placed and the psychological state of the user U by selecting the target device to be activated based on the environmental information and the bioinformation.

In one aspect of this embodiment, the environmental sensor 20 detects the position information of the information providing device 10 as the environmental information, and the biosensor 22 detects the brain activity of the user U as the bioinformation. The output selection unit 48 selects the display unit 26A as the target device when at least one of the following conditions is satisfied: the position of the information providing device 10 is within a predetermined area and the brain activity is equal to or less than the brain activity threshold; and the change in the position of the information providing device 10 per unit time is equal to or less than the predetermined change threshold and the brain activity is equal to or less than the brain activity threshold. On the other hand, if the first condition and the second condition are not satisfied, the output selection unit 48 does not select the display unit 26A as the target device. Since the information providing device 10 determines whether to operate the display unit 26A in this way, it can appropriately output visual stimulation to the user U, for example, when the user U is not moving and is relaxed, or when the user is in a vehicle and is relaxed.

Although the embodiments of the present invention have been described above, the contents of these embodiments do not limit the embodiments. The above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be combined as appropriate, and the configurations of each embodiment can also be combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-mentioned embodiments.

REFERENCE SIGNS LIST 10 Information providing device 20 Environmental sensor 22 Biometric sensor 26 Output section 26A Display section 26B Audio output section 26C Sensory stimulation output section 40 Environmental information acquisition section 42 Biometric information acquisition section 44 Environmental determination section 46 User state determination section 48 Output selection section 50 Output specification determination section 52 Content image acquisition section 54 Output control section PM Environmental image PS Content image

Claims (7)

An information providing device for providing information to a user,
An output unit including a display unit that outputs visual stimuli, an audio output unit that outputs auditory stimuli, and a sensory stimulus output unit that outputs sensory stimuli different from visual and auditory stimuli;
an environmental sensor for detecting environmental information around the information providing device;
an output specification determination unit that determines output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the environmental information;
an output control unit that causes the output unit to output the visual stimulus, the auditory stimulus, and the sensory stimulus based on the output specification;
an environment identification unit that identifies an environment pattern indicating an environment state of the user based on the environment information;
a biometric information acquiring unit for acquiring biometric information of the user;
a user condition identification unit that identifies a condition of the user based on the biological information;
Including,
the output specification determination unit determines a reference output specification indicating a reference output based on the environmental pattern, determines an output specification correction degree, which is a value for correcting the reference output specification, based on the state of the user identified by the user state identification unit, and determines the output specification by correcting the reference output specification with the output specification correction degree.
Information providing device. The system further includes a plurality of the environmental sensors for detecting different types of environmental information,
The information providing device according to claim 1 , wherein the environment identifying unit identifies the environment pattern that comprehensively indicates the current environment of the user based on different types of the environmental information. The information providing device according to claim 1 or 2, wherein the output specification determination unit determines at least one of the following as the output specification of the visual stimulus: the size of the image displayed by the display unit, the transparency of the image displayed by the display unit, and the content of the image displayed by the display unit. The information providing device according to any one of claims 1 to 3, wherein the output specification determination unit determines at least one of the volume and sound of the sound output by the audio output unit as the output specification of the auditory stimulation. The sensory stimulus output unit outputs a tactile stimulus as the sensory stimulus,
The information providing device according to any one of claims 1 to 4, wherein the output specification determination unit determines at least one of the intensity of the tactile stimulus output by the sensory stimulus output unit and the frequency at which the tactile stimulus is output as the output specification of the tactile stimulus. An information providing method for providing information to a user, comprising:
Detecting surrounding environmental information;
determining output specifications for visual stimuli, auditory stimuli, and sensory stimuli other than visual and auditory stimuli based on the environmental information;
outputting the visual stimulus, the auditory stimulus, and the sensory stimulus to an output unit based on the output specification;
identifying an environmental pattern indicative of an environmental state of the user based on the environmental information;
acquiring biometric information of the user;
Identifying a state of the user based on the biometric information;
Including,
In the step of determining the output specifications, a reference output specification indicating a reference output is determined based on the environmental pattern, an output specification correction degree, which is a value for correcting the reference output specification, is determined based on the identified state of the user, and the output specifications are determined by correcting the reference output specification with the output specification correction degree.
How to provide information. Detecting surrounding environmental information;
determining output specifications for visual stimuli, auditory stimuli, and sensory stimuli other than visual and auditory stimuli based on the environmental information;
outputting the visual stimulus, the auditory stimulus, and the sensory stimulus to an output unit based on the output specification;
Identifying an environmental pattern indicative of a state of the user's environment based on the environmental information;
acquiring biometric information of the user;
identifying a state of the user based on the biological information;
The method includes causing a computer to execute an information providing method for providing information to a user,
In the step of determining the output specifications, a reference output specification indicating a reference output is determined based on the environmental pattern, an output specification correction degree, which is a value for correcting the reference output specification, is determined based on the identified state of the user, and the output specifications are determined by correcting the reference output specification with the output specification correction degree.
program.

Images