JP7552127B2 - Display device, display method, and program - Google Patents

Description

The present invention relates to a display device, a display method, and a program.

Recently, information devices have evolved significantly with the advancement of high-speed CPUs, high-definition screen display technology, advances in small and lightweight battery technology, the spread of wireless network environments, and the expansion of broadband. As display devices that provide images to users, not only smartphones, which are a representative example, but also so-called wearable devices that are worn by users have become widespread. For example, Patent Document 1 describes a device that gives the user the sensation that a virtual object is actually present by presenting multiple sensory information to the user. Patent Document 2 also describes a device that determines preferences from the user's biometric information and determines advertising information based on the results of the determination.

JP 2011-96171 A JP 2014-52518 A

Here, display devices that provide images to users are required to provide images appropriately to users.

In view of the above problems, the present invention aims to provide a display device, a display method, and a program that can provide appropriate images to a user.

A display device according to one aspect of the present invention includes a display unit that displays an image, a sub-image acquisition unit that acquires image data of a sub-image including advertising information to be displayed on the display unit and advertising fee information paid to display the advertising information, an output specification determination unit that determines a display mode indicating how the sub-image is to be displayed as a display specification of the sub-image based on the advertising fee information, and an output control unit that causes the display unit to display the sub-image by superimposing it on a main image viewed through the display unit and based on the display specification.

A display method according to one aspect of the present invention includes the steps of acquiring image data of a sub-image including advertising information to be displayed on a display unit and advertising fee information paid to display the advertising information, determining a display mode indicating how the sub-image is to be displayed as a display specification of the sub-image based on the advertising fee information, and displaying the sub-image on the display unit by superimposing the sub-image on a main image viewed through the display unit and based on the display specification.

A program according to one aspect of the present invention causes a computer to execute a display method including the steps of acquiring image data of a sub-image including advertising information to be displayed on a display unit and advertising fee information paid to display the advertising information, determining a display mode indicating how the sub-image is to be displayed as a display specification of the sub-image based on the advertising fee information, and displaying the sub-image on the display unit by superimposing the sub-image on a main image viewed through the display unit and based on the display specification.

The present invention allows appropriate images to be provided to the user.

FIG. 1 is a schematic diagram of a display device according to a first embodiment. FIG. 2 is a diagram showing an example of an image displayed by the display device. FIG. 3 is a schematic block diagram of the display device according to the present embodiment. FIG. 4 is a flowchart illustrating the processing contents of the display device according to the first embodiment. FIG. 5 is a table illustrating an example of the environment score. FIG. 6 is a table showing an example of an environment pattern. FIG. 7 is a diagram showing an example in which the display mode is changed. FIG. 8 is a diagram showing an example in which the display mode is changed. FIG. 9 is a diagram showing an example in which the display mode is changed. FIG. 10 is a table showing the relationship between the environment patterns, the target devices, and the reference output specifications. FIG. 11 is a graph showing an example of a pulse wave. FIG. 12 is a table showing an example of the relationship between the user state and the output specification correction degree. FIG. 13 is a table showing an example of the output restriction necessity information. FIG. 14 is a flowchart illustrating the processing contents of the display device according to the second embodiment. FIG. 15 is a schematic block diagram of a display device according to the third embodiment. FIG. 16 is a flowchart illustrating the processing contents of the display device according to the third embodiment. FIG. 17 is a diagram showing an example of a display image according to the third embodiment. FIG. 18 is a diagram showing an example of a sub-image in which the shape of an object is different from the actual shape. FIG. 19 is a schematic block diagram of a display device according to the fourth embodiment. FIG. 20 is a flowchart illustrating the processing contents of the display device according to the fourth embodiment. FIG. 21 is a schematic block diagram of a display system according to the fourth embodiment. FIG. 22 is a schematic block diagram of a display device according to the fifth embodiment. FIG. 23 is a flowchart illustrating the processing contents of the display device according to the fifth embodiment. FIG. 24 is a table illustrating an example of the age restriction requirement information. FIG. 25 is a table illustrating an example of physical restriction necessity information. FIG. 26 is a table showing an example of content ratings. FIG. 27 is a schematic block diagram of a display device according to the sixth embodiment. FIG. 28 is a flowchart illustrating the processing contents of the display device according to the sixth embodiment. FIG. 29 is a table showing an example of the final ratings. FIG. 30 is a table illustrating an example of determining the output contents based on the final rating.

The following describes in detail an embodiment of the present invention with reference to the drawings. Note that the present invention is not limited to the embodiment described below.

First Embodiment
FIG. 1 is a schematic diagram of a display device according to the first embodiment. The display device 10 according to the first embodiment is a display device that displays an image. As shown in FIG. 1, the display device 10 is a so-called wearable device that is worn on the body of a user U. In the example of this embodiment, the display device 10 includes a device 10A worn on the eye of the user U, a device 10B worn on the ear of the user U, and a device 10C worn on the arm of the user. The device 10A worn on the eye of the user U includes a display unit 26A (described later) that outputs a visual stimulus to the user U (displays an image), the device 10B worn on the ear of the user U includes a sound output unit 26B (described later) that outputs an auditory stimulus (sound) to the user U, and the device 10C worn on the arm of the user U includes a tactile stimulus output unit 26C (described later) that outputs a tactile stimulus to the user U. However, the configuration of FIG. 1 is an example, and the number of devices and the position of wearing on the user U may be arbitrary. For example, the display device 10 is not limited to a wearable device, but may be a device carried by the user U, such as a so-called smartphone or tablet terminal.

(Main statue)
FIG. 2 is a diagram showing an example of an image displayed by the display device. As shown in FIG. 2, the display device 10 provides the user U with a main image PM through the display unit 26A. As a result, the user U wearing the display device 10 can view the main image PM. In this embodiment, the main image PM is an image of a landscape that the user U would view if the user U were not wearing the display device 10, and can also be said to be an image of an actual object that falls within the range of the user U's field of vision. In this embodiment, the display device 10 provides the user U with the main image PM by, for example, transmitting external light (ambient visible light) from the display unit 26A. That is, in this embodiment, it can be said that the user U directly views an image of an actual landscape through the display unit 26A. However, the display device 10 is not limited to allowing the user U to directly view an image of an actual landscape, and may provide the user U with the main image PM through the display unit 26A by displaying an image of the main image PM on the display unit 26A. In this case, the user U visually recognizes the image of the scenery displayed on the display unit 26A as the main image PM. In this case, the display device 10 displays an image captured by a camera 20A (described later) that is within the visual field of the user U as the main image PM on the display unit 26A. Note that, although a road and a building are included as the main image PM in FIG. 2, this is merely an example.

(Sub image)
As shown in Fig. 2, the display device 10 displays the sub-image PS on the display unit 26A so as to be superimposed on the main image PM provided through the display unit 26A. This allows the user U to view an image in which the sub-image PS is superimposed on the main image PM. The sub-image PS is an image superimposed on the main image PM, and can be said to be an image other than an actual scene that falls within the visual field of the user U. In other words, the display device 10 can be said to provide the user U with AR (Augmented Reality) by superimposing the sub-image PS on the main image PM, which is an actual scene.

The sub-image PS may be any content, but in this embodiment, it is an advertisement. Here, advertisement refers to information that informs of products or services. However, the sub-image is not limited to being an advertisement, and may be any image that includes information to inform the user U. For example, the sub-image may be a navigation image that shows the user U directions. Note that in FIG. 2, the sub-image PS is the letters AAAA, but this is merely an example.

In this way, the display device 10 provides the main image PM and the sub-image PS, but may also display a content image on the display unit 26A that has a different content from the main image PM and the sub-image PS. The content image may be an image of any content, such as a movie or a television program.

(Configuration of the display device)
3 is a schematic block diagram of the display device according to the present embodiment. As shown in FIG. 3, the display device 10 includes an environment sensor 20, a biosensor 22, an input unit 24, an output unit 26, a communication unit 28, a storage unit 30, and a control unit 32.

(Environmental Sensor)
The environmental sensor 20 is a sensor that detects environmental information around the display device 10. The environmental information around the display device 10 can be said to be information indicating the environment in which the display device 10 is placed. In addition, since the display device 10 is worn by the user U, it can be said that the environmental sensor 20 detects environmental information around the user U.

The environmental sensor 20 includes a camera 20A, a microphone 20B, a GNSS receiver 20C, an acceleration sensor 20D, a gyro sensor 20E, a light sensor 20F, a temperature sensor 20G, and a humidity sensor 20H. However, the environmental sensor 20 may include any sensor that detects environmental information, and may include, for example, at least one of the camera 20A, the microphone 20B, the GNSS receiver 20C, the acceleration sensor 20D, the gyro sensor 20E, the light sensor 20F, the temperature sensor 20G, and the humidity sensor 20H, or may include other sensors.

The camera 20A is an imaging device, and captures the periphery of the display device 10 by detecting visible light around the display device 10 (user U) as environmental information. The camera 20A may be a video camera that captures images at a predetermined frame rate. The position and orientation of the camera 20A in the display device 10 are arbitrary, but for example, the camera 20A may be provided in the device 10A shown in FIG. 1, and the imaging direction may be the direction in which the face of the user U is facing. This allows the camera 20A to capture an object in the line of sight of the user U, that is, an object within the range of the user U's field of vision. The number of cameras 20A is arbitrary, and may be one or more. Note that when there are multiple cameras 20A, information on the direction in which the camera 20A is facing is also acquired.

The microphone 20B is a microphone that detects sounds (sound wave information) around the display device 10 (user U) as environmental information. The position, orientation, and number of microphones 20B installed on the display device 10 are arbitrary. Note that if there are multiple microphones 20B, information on the direction in which the microphones 20B are facing is also obtained.

The GNSS receiver 20C is a device that detects the position information of the display device 10 (user U) as environmental information. The position information here is Earth coordinates. In this embodiment, the GNSS receiver 20C is a so-called GNSS (Global Navigation Satellite System) module that receives radio waves from satellites and detects the position information of the display device 10 (user U).

The acceleration sensor 20D is a sensor that detects the acceleration of the display device 10 (user U) as environmental information, such as gravity, vibration, and impact.

The gyro sensor 20E is a sensor that detects the rotation and orientation of the display device 10 (user U) as environmental information, and detects this using the principles of Coriolis force, Euler force, centrifugal force, etc.

The optical sensor 20F is a sensor that detects the intensity of light around the display device 10 (user U) as environmental information. The optical sensor 20F can detect the intensity of visible light, infrared light, and ultraviolet light.

The temperature sensor 20G is a sensor that detects the temperature around the display device 10 (user U) as environmental information.

The humidity sensor 20H is a sensor that detects the humidity around the display device 10 (user U) as environmental information.

(Biosensor)
The biosensor 22 is a sensor that detects bioinformation of the user U. The biosensor 22 may be provided at any position as long as it can detect the bioinformation of the user U. The bioinformation here is preferably not unchanging, such as a fingerprint, but is information whose value changes depending on the state of the user U. Furthermore, the bioinformation here is preferably information related to the autonomic nerves of the user U, that is, information whose value changes regardless of the will of the user U. Specifically, the biosensor 22 includes a pulse wave sensor 22A and an electroencephalogram sensor 22B, and detects the pulse waves and electroencephalograms of the user U as bioinformation.

The pulse wave sensor 22A is a sensor that detects the pulse wave of the user U. The pulse wave sensor 22A may be, for example, a transmission type photoelectric sensor equipped with a light emitting unit and a light receiving unit. In this case, the pulse wave sensor 22A may be configured such that the light emitting unit and the light receiving unit face each other across the fingertip of the user U, and the light receiving unit receives light that has passed through the fingertip, and measures the pulse waveform by utilizing the fact that the greater the pressure of the pulse wave, the greater the blood flow. However, the pulse wave sensor 22A is not limited to this and may be of any type that can detect pulse waves.

The EEG sensor 22B is a sensor that detects the brain waves of the user U. The EEG sensor 22B may be of any configuration as long as it can detect the brain waves of the user U, but in principle, it is sufficient to grasp waves such as alpha waves and beta waves, as well as basic rhythmic (background brain wave) activity that appears throughout the brain, and to detect improvements or decreases in activity of the brain as a whole, so several sensors may be provided. Unlike EEG measurements for medical purposes, this embodiment is sufficient to measure rough changes in the state of the user U, so it is possible to detect very simple surface EEG signals by attaching only two electrodes to the forehead and ears, for example.

The biosensor 22 is not limited to detecting pulse waves and brain waves as bioinformation, and may detect, for example, at least one of pulse waves and brain waves. The biosensor 22 may also detect bioinformation other than pulse waves and brain waves, for example, the amount of sweating or the size of the pupils.

(Input section)
The input unit 24 is a device that accepts operations from a user, and may be, for example, a touch panel.

(Output section)
The output unit 26 is a device that outputs a stimulus for at least one of the five senses to the user U. Specifically, the output unit 26 has a display unit 26A, an audio output unit 26B, and a tactile stimulus output unit 26C. The display unit 26A is a display that outputs a visual stimulus to the user U by displaying an image, and can also be referred to as a visual stimulus output unit. In this embodiment, the display unit 26A is a so-called HMD (Head Mount Display). As described above, the display unit 26A displays the sub-image PS so as to be superimposed on the main image PM. The audio output unit 26B is a device (speaker) that outputs an auditory stimulus to the user U by outputting audio, and can also be referred to as an auditory stimulus output unit. The tactile stimulus output unit 26C is a device that outputs a tactile stimulus to the user U. For example, the tactile stimulus output unit 26C outputs a tactile stimulus to the user U by physically operating, such as by vibration, but the type of tactile stimulus is not limited to vibration and may be any type.

In this way, the output unit 26 stimulates the vision, hearing, and touch of the five human senses. However, the output unit 26 is not limited to outputting visual, auditory, and tactile stimuli. For example, the output unit 26 may output at least one of the visual, auditory, and tactile stimuli, may output at least a visual stimulus (display an image), may output either an auditory or tactile stimulus in addition to a visual stimulus, or may output another sensory stimulus of the five senses (i.e., at least one of a taste stimulus and an olfactory stimulus) in addition to at least one of the visual, auditory, and tactile stimuli.

(Communications Department)
The communication unit 28 is a module that communicates with an external device, and may include, for example, an antenna. In this embodiment, the communication method by the communication unit 28 is wireless communication, but the communication method may be any communication method. The communication unit 28 includes a sub-image receiving unit 28A. The sub-image receiving unit 28A is a receiver that receives sub-image data, which is image data of the sub-image. Note that the content displayed by the sub-image may include sound and tactile stimulation. In this case, the sub-image receiving unit 28A may receive sound data and tactile stimulation data as sub-image data together with the image data of the sub-image. In addition, when the display unit 26A displays a content image other than the above-mentioned sub-image, the communication unit 28 also receives image data of the content image.

(Memory unit)
The storage unit 30 is a memory that stores various information such as the contents of calculations and programs of the control unit 32, and includes at least one of a main storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), and an external storage device such as a HDD (Hard Disk Drive).

The memory unit 30 stores a learning model 30A, map data 30B, and a specification setting database 30C. The learning model 30A is an AI model used to identify the environment in which the user U is located based on environmental information. The map data 30B is data including location information of real buildings and natural objects, and can be said to be data in which the earth coordinates are associated with real buildings and natural objects. The specification setting database 30C is a database including information for determining the display specifications of the sub-image PS as described later. The processing using the learning model 30A, the map data 30B, the specification setting database 30C, and the like will be described later. The learning model 30A, the map data 30B, the specification setting database 30C, and the program for the control unit 32 stored in the memory unit 30 may be stored in a recording medium readable by the display device 10. In addition, the program for the control unit 32, the learning model 30A, the map data 30B, and the specification setting database 30C stored in the memory unit 30 are not limited to being stored in advance in the memory unit 30, and when using these data, the display device 10 may obtain them from an external device via communication.

(Control unit)
The control unit 32 is a calculation device, that is, a central processing unit (CPU). The control unit 32 includes an environmental information acquisition unit 40, a biometric information acquisition unit 42, an environment determination unit 44, a user state determination unit 46, an output selection unit 48, an output specification determination unit 50, a sub-image acquisition unit 52, and an output control unit 54. The control unit 32 reads out a program (software) from the storage unit 30 and executes it to realize the environmental information acquisition unit 40, the biometric information acquisition unit 42, the environment determination unit 44, the user state determination unit 46, the output selection unit 48, the output specification determination unit 50, the sub-image acquisition unit 52, and the output control unit 54, and executes the processes. The control unit 32 may execute these processes using one CPU, or may be provided with multiple CPUs and execute the processes using the multiple CPUs. In addition, at least a portion of the environmental information acquisition unit 40, the biometric information acquisition unit 42, the environmental determination unit 44, the user state determination unit 46, the output selection unit 48, the output specification determination unit 50, the sub-image acquisition unit 52, and the output control unit 54 may be realized in hardware.

The environmental information acquisition unit 40 controls the environmental sensor 20 to cause the environmental sensor 20 to detect environmental information. The environmental information acquisition unit 40 acquires the environmental information detected by the environmental sensor 20. The processing of the environmental information acquisition unit 40 will be described later. Note that when the environmental information acquisition unit 40 is hardware, it can also be called an environmental information detector.

The biometric information acquisition unit 42 controls the biometric sensor 22 to cause the biometric sensor 22 to detect biometric information. The biometric information acquisition unit 42 acquires environmental information detected by the biometric sensor 22. The processing of the biometric information acquisition unit 42 will be described later. Note that when the biometric information acquisition unit 42 is hardware, it can also be called a biometric information detector.

The environment identification unit 44 identifies the environment in which the user U is located based on the environmental information acquired by the environmental information acquisition unit 40. The environment identification unit 44 calculates an environmental score, which is a score for identifying the environment, and identifies an environmental state pattern indicating the state of the environment based on the environmental score, thereby identifying the environment. The processing of the environment identification unit 44 will be described later.

The user status identification unit 46 identifies the status of the user U based on the biometric information acquired by the biometric information acquisition unit 42. The processing of the user status identification unit 46 will be described later.

The output selection unit 48 selects a target device to be operated in the output unit 26 based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the bioinformation acquired by the bioinformation acquisition unit 42. The processing of the output selection unit 48 will be described later. Note that when the output selection unit 48 is hardware, it may be called a sensory selector.

The output specification determination unit 50 determines the output specifications of the stimuli (here, visual stimuli, auditory stimuli, and tactile stimuli) to be output by the output unit 26 based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the bioinformation acquired by the bioinformation acquisition unit 42. For example, the output specification determination unit 50 can be said to determine the display specifications (output specifications) of the sub-image PS displayed by the display unit 26A based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the bioinformation acquired by the bioinformation acquisition unit 42. The output specifications are an index indicating how the stimuli to be output by the output unit 26 are to be output, and will be described in detail later. The processing of the output specification determination unit 50 will be described later.

The sub-image acquisition unit 52 acquires the sub-image data via the sub-image receiving unit 28A.

The output control unit 54 controls the output unit 26 to perform output. The output control unit 54 causes output to be performed to the target device selected by the output selection unit 48 in the output specifications determined by the output specification determination unit 50. For example, the output control unit 54 controls the display unit 26A to display the sub-image PS acquired by the sub-image acquisition unit 52 superimposed on the main image PM and in the display specifications determined by the output specification determination unit 50. Note that when the output control unit 54 is hardware, it may be called a multi-sensory sensory providing device.

The display device 10 is configured as described above.

(Processing Contents)
Next, the processing contents of the display device 10, more specifically, the processing contents to be output by the output unit 26 based on the environmental information and the biological information will be described. Fig. 4 is a flowchart for explaining the processing contents of the display device according to the first embodiment.

(Getting environmental information)
As shown in FIG. 4, the display device 10 acquires the environmental information detected by the environmental sensor 20 by the environmental information acquisition unit 40 (step S10). In this embodiment, the environmental information acquisition unit 40 acquires image data of the periphery of the display device 10 (user U) from the camera 20A, acquires audio data of the periphery of the display device 10 (user U) from the microphone 20B, acquires position information of the display device 10 (user U) from the GNSS receiver 20C, acquires acceleration information of the display device 10 (user U) from the acceleration sensor 20D, acquires orientation information, i.e., attitude information, of the display device 10 (user U) from the gyro sensor 20E, acquires infrared and ultraviolet intensity information of the periphery of the display device 10 (user U) from the light sensor 20F, acquires temperature information of the periphery of the display device (user U) from the temperature sensor 20G, and acquires humidity information of the periphery of the display device 10 (user U) from the humidity sensor 20H. The environmental information acquisition unit 40 acquires these environmental information sequentially at predetermined intervals. The environmental information acquiring unit 40 may acquire each piece of environmental information at the same timing, or may acquire each piece of environmental information at different timings. In addition, the predetermined period until the next piece of environmental information is acquired may be set arbitrarily, and the predetermined period may be the same for each piece of environmental information, or may be different for each piece of environmental information.

(Determination of Dangerous Conditions)
After acquiring the environmental information, the display device 10 determines, by the environmental identification unit 44, whether the environment around the user U is in a dangerous state based on the environmental information (step S12).

The environment identification unit 44 judges whether or not the display device 10 is in a dangerous state based on an image of the surroundings of the display device 10 captured by the camera 20A. Hereinafter, the image of the surroundings of the display device 10 captured by the camera 20A will be referred to as a surrounding image as appropriate. The environment identification unit 44, for example, identifies an object captured in the surrounding image and judges whether or not the display device 10 is in a dangerous state based on the type of the identified object. More specifically, the environment identification unit 44 may judge that the display device 10 is in a dangerous state if the object captured in the surrounding image is a specific object set in advance, and may judge that the display device 10 is not in a dangerous state if the object is not a specific object. The specific object may be set arbitrarily, but may be an object that may pose a danger to the user U, such as a flame indicating a fire, a vehicle, or a sign indicating that construction is underway. The environment identification unit 44 may also judge whether or not the display device 10 is in a dangerous state based on multiple surrounding images captured in succession in time series. For example, the environment identification unit 44 may identify an object for each of multiple surrounding images captured in succession in time series, and judge whether the objects are specific objects and the same object. Then, when the same specific object is captured, the environment determination unit 44 determines whether the specific object captured in the peripheral image captured later in the time series is relatively larger in the image, that is, whether the specific object is approaching the user U. Then, the environment determination unit 44 determines that the specific object is in a dangerous state when the specific object captured in the peripheral image captured later is larger, that is, when the specific object is approaching the user U. On the other hand, the environment determination unit 44 determines that the specific object is not in a dangerous state when the specific object captured in the peripheral image captured later is not larger, that is, when the specific object is not approaching the user U. In this way, the environment determination unit 44 may determine whether the specific object is in a dangerous state based on one peripheral image, or may determine whether the specific object is in a dangerous state based on multiple peripheral images captured consecutively in a time series. For example, the environment determination unit 44 may switch the determination method depending on the type of object captured in the peripheral image. The environment determination unit 44 may determine a dangerous state from a single surrounding image when a specific object whose danger can be determined from a single surrounding image, such as a flame indicating a fire, is captured. Also, for example, when a specific object whose danger cannot be determined from a single surrounding image, such as a vehicle, the environment determination unit 44 may determine a dangerous state based on multiple surrounding images captured consecutively in time series.

The environment identification unit 44 may identify the object in the peripheral image by any method, but may identify the object using the learning model 30A, for example. In this case, for example, the learning model 30A is an AI model constructed by learning from multiple data sets as teacher data, with the image data and information indicating the type of object in the image being treated as one data set. The environment identification unit 44 inputs the image data of the peripheral image into the trained learning model 30A, obtains information that identifies the type of object in the peripheral image, and identifies the object.

In addition, the environment identification unit 44 may determine whether or not there is a danger state based on the location information acquired by the GNSS receiver 20C in addition to the surrounding image. In this case, the environment identification unit 44 acquires location information indicating the location of the user U based on the location information of the display device 10 (user U) acquired by the GNSS receiver 20C and the map data 30B. The location information is information indicating the type of location in which the user U (display device 10) is located. That is, for example, the location information is information indicating that the user U is in a shopping center or on a road. The environment identification unit 44 reads the map data 30B, identifies the type of structure or natural object within a predetermined distance range from the current location of the user U, and identifies the location information from the structure or natural object. For example, if the current location of the user U overlaps with the coordinates of a shopping center, the location information is identified as indicating that the user U is in a shopping center. The environment determination unit 44 determines that a dangerous state exists when there is a specific relationship between the location information and the type of object determined from the surrounding image, and determines that a dangerous state exists when there is no specific relationship. The specific relationship may be set arbitrarily, but for example, a combination of an object and a location that may cause danger if that object is present in a certain location may be set as the specific relationship.

The environment identification unit 44 also determines whether or not there is a dangerous state based on the audio information acquired by the microphone 20B. Hereinafter, the audio information around the display device 10 acquired by the microphone 20B will be referred to as the surrounding audio, as appropriate. The environment identification unit 44, for example, identifies the type of audio contained in the surrounding audio, and determines whether or not there is a dangerous state based on the identified type of audio. More specifically, the environment identification unit 44 may determine that there is a dangerous state if the type of audio contained in the surrounding audio is a specific audio set in advance, and may determine that there is no dangerous state if the type of audio is not the specific audio. The specific audio may be set arbitrarily, and may be audio that may pose a danger to the user U, such as audio indicating a fire, audio of a vehicle, or audio indicating that construction is underway.

The environment identification unit 44 may identify the type of sound contained in the ambient sound using any method, but may, for example, identify the target object using the learning model 30A. In this case, for example, the learning model 30A is an AI model constructed by learning from multiple data sets that are teacher data, with sound data (e.g., data indicating the frequency and intensity of the sound) and information indicating the type of sound as one data set. The environment identification unit 44 inputs the sound data of the ambient sound into the trained learning model 30A, obtains information that identifies the type of sound contained in the ambient sound, and identifies the type of sound.

The environment identification unit 44 may also determine whether or not a dangerous state exists based on the location information acquired by the GNSS receiver 20C in addition to the surrounding sounds. In this case, the environment identification unit 44 acquires location information indicating the location of the user U based on the location information of the display device 10 (user U) acquired by the GNSS receiver 20C and the map data 30B. The environment identification unit 44 then determines that a dangerous state exists when there is a specific relationship between the location information and the type of sound identified from the surrounding sounds, and determines that there is no dangerous state when there is no specific relationship. The specific relationship may be set arbitrarily, but for example, a combination of sound and location that may lead to danger if the sound occurs in a certain location may be set as the specific relationship.

In this manner, in this embodiment, the environment determination unit 44 determines a dangerous state based on the surrounding image and the surrounding sound. However, the method of determining a dangerous state is not limited to the above and is arbitrary. For example, the environment determination unit 44 may determine a dangerous state based on either the surrounding image or the surrounding sound. Furthermore, the environment determination unit 44 may determine whether a dangerous state exists based on at least one of the image of the surroundings of the display device 10 captured by the camera 20A, the sound of the surroundings of the display device 10 detected by the microphone 20B, and the position information acquired by the GNSS receiver 20C. Furthermore, in this embodiment, the determination of a dangerous state is not essential and need not be performed.

(Setting the content of danger notifications)
When it is determined that the state is dangerous (step S10, Yes), the display device 10 sets the danger notification content, which is the notification content for notifying that the state is dangerous, by the output control unit 54 (step S12). The display device 10 sets the danger notification content based on the content of the dangerous state. The content of the dangerous state is information indicating what kind of danger is approaching, and is specified from the type of object shown in the peripheral image and the type of sound included in the peripheral sound. For example, if the object is a vehicle approaching, the content of the dangerous state is "a vehicle is approaching". The danger notification content is information indicating the content of the dangerous state. For example, if the content of the dangerous state is that a vehicle is approaching, the danger notification content is information indicating that a vehicle is approaching.

The danger notification content differs depending on the type of target device selected in step S26 described below. For example, when the display unit 26A is the target device, the danger notification content becomes the display content (content) of the sub-image PS. That is, the danger notification content is displayed as the sub-image PS superimposed on the main image PM. In this case, for example, the danger notification content is image data indicating the content "Watch out, a car is approaching!" On the other hand, when the audio output unit 26B is the target device, the danger notification content becomes the content of the sound output from the audio output unit 26B. In this case, for example, the danger notification content becomes audio data for emitting the sound "A car is approaching. Watch out." Also, when the tactile stimulation output unit 26C is the target device, the danger notification content becomes the content of the tactile stimulation output from the tactile stimulation output unit 26C. In this case, for example, the danger notification content becomes a tactile stimulation that attracts the attention of the user U.

The setting of the danger notification content in step S14 may be performed at any time after it has been determined in step S12 that a danger state exists and before the danger notification content is output in the subsequent step S38, for example, after the target device is selected in the subsequent step S32.

(Calculation of environmental score)
When it is determined that the display device 10 is not in a dangerous state (step S12; No), the display device 10 calculates various environmental scores based on the environmental information by the environment identification unit 44, as shown in steps S16 to S22. The environmental score is a score for identifying the environment in which the user U (display device 10) is placed. Specifically, the environment identification unit 44 calculates the posture score (step S16), calculates the location score (step S18), calculates the movement score (step S20), and calculates the safety score (step S22) as the environmental score. The order of steps S16 to S22 is not limited to this and is arbitrary. Note that even when the danger notification content is set in step S14, various environmental scores are calculated as shown in steps S16 to S22. The environmental score will be described in more detail below.

Figure 5 is a table illustrating an example of an environmental score. As shown in Figure 5, the environmental identification unit 44 calculates an environmental score for each environmental category. An environmental category indicates the type of environment for the user U, and in the example of Figure 5, includes the posture of the user U, the location of the user U, the movements of the user U, and the safety of the environment around the user U. In addition, the environmental identification unit 44 divides the environmental categories into more specific subcategories and calculates an environmental score for each subcategory.

(Posture score)
The environment determination unit 44 calculates a posture score as an environment score for the posture category of the user U. In other words, the posture score is information indicating the posture of the user U, and can be said to be information indicating, as a numerical value, what posture the user U is in. The environment determination unit 44 calculates the posture score based on environmental information related to the posture of the user U, among multiple types of environmental information. Examples of the environmental information related to the posture of the user U include a surrounding image acquired by the camera 20A and the orientation of the display device 10 detected by the gyro sensor 20E.

5, the category of the user U's posture includes a subcategory of standing and a subcategory of the face facing horizontally. The environment identification unit 44 calculates a posture score for the subcategory of standing based on the peripheral image acquired by the camera 20A. The posture score for the subcategory of standing can be a numerical value indicating the degree of agreement of the user U's posture with respect to the standing state. The method of calculating the posture score for the subcategory of standing may be arbitrary, and may be calculated using the learning model 30A, for example. In this case, for example, the learning model 30A is an AI model constructed by learning a plurality of datasets as teacher data, with image data of the scenery in the person's field of vision and information indicating whether the person is standing as one dataset. The environment identification unit 44 inputs image data of the peripheral image into the learned learning model 30A to obtain a numerical value indicating the degree of agreement with the standing state, and sets it as the posture score. Note that the degree of agreement with the standing state is used here, but is not limited to the standing state, and may be, for example, the degree of agreement with a sitting state or a lying state.

The environment determination unit 44 also calculates a posture score for the subcategory of the face orientation being horizontal, based on the orientation of the display device 10 detected by the gyro sensor 20E. The posture score for the subcategory of the face orientation being horizontal can be a numerical value indicating the degree of agreement of the user U's posture (face orientation) with respect to the horizontal direction. Any method may be used to calculate the posture score for the subcategory of the face orientation being horizontal. Note that, although the degree of agreement with the face orientation being horizontal is used here, it is not limited to the horizontal direction, and the degree of agreement with any direction may be used.

In this way, the environment determination unit 44 can be said to set information indicating the posture of the user U (here, posture score) based on the peripheral image and the orientation of the display device 10. However, the environment determination unit 44 is not limited to using the peripheral image and the orientation of the display device 10 to set the information indicating the posture of the user U, and may use any environmental information, for example, at least one of the peripheral image and the orientation of the display device 10.

(Location score)
The environment determination unit 44 calculates a location score as an environment score for the category of the location of the user U. In other words, the location score is information indicating the location of the user U, and can be said to be information indicating, as a numerical value, what type of location the user U is located in. The environment determination unit 44 calculates the location score based on environmental information related to the location of the user U among multiple types of environmental information. Examples of environmental information related to the location of the user U include a surrounding image acquired by the camera 20A, position information of the display device 10 acquired by the GNSS receiver 20C, and surrounding audio acquired by the microphone 20B.

5, the category of the user U's location includes a subcategory of being on a train, a subcategory of being on the tracks, and a subcategory of sounds on a train. The environment identification unit 44 calculates a location score for the subcategory of being on a train based on the surrounding image acquired by the camera 20A. The location score for the subcategory of being on a train can be said to be a numerical value indicating the degree of agreement of the user U's location with respect to the location of being on a train. The method of calculating the location score for the subcategory of being on a train may be arbitrary, but may be calculated using the learning model 30A, for example. In this case, for example, the learning model 30A is an AI model constructed by learning multiple data sets as teacher data, with image data of the scenery in a person's field of vision and information indicating whether the person is on a train as one data set. The environment identification unit 44 inputs image data of the surrounding image into the learned learning model 30A to obtain a numerical value indicating the degree of agreement with the location of being on a train, and sets it as the location score. Note that, although the degree of match was calculated for the location of being on a train, this is not limited thereto, and the degree of match may be calculated for being in any type of vehicle.

The environment determination unit 44 calculates a location score for the subcategory of being on the tracks based on the position information of the display device 10 acquired by the GNSS receiver 20C. The location score for the subcategory of being on the tracks can be a numerical value indicating the degree of match of the user U's location with respect to the location of being on the tracks. Any method for calculating the location score for the subcategory of being on the tracks may be used, for example, map data 30B may be used. For example, the environment determination unit 44 reads out the map data 30B, and when the current position of the user U overlaps with the coordinates of the tracks, calculates the location score so that the degree of match of the user U's location with respect to the location of being on the tracks is high. Note that, although the degree of match with the tracks is calculated here, the present invention is not limited thereto, and the degree of match with the position of any type of structure or natural object may be calculated.

The environment identification unit 44 calculates a location score for the subcategory of sounds on a train based on the ambient sounds acquired by the microphone 20B. The location score for the subcategory of sounds on a train can be a numerical value indicating the degree of match between the ambient sounds and the sounds on a train. The method of calculating the location score for the subcategory of sounds on a train can be any method, but for example, it can be determined in a similar manner to the method of determining whether a dangerous state is present based on the ambient sounds as described above, that is, for example, by determining whether the ambient sounds are a specific type of sound. Note that, although the degree of match to the sounds on a train is calculated here, the method is not limited thereto, and the degree of match with sounds in any location can also be calculated.

In this way, the environment determination unit 44 can be said to set information indicating the location of the user U (here, the location score) based on the surrounding image, surrounding sound, and position information of the display device 10. However, the environment determination unit 44 is not limited to using the surrounding image, surrounding sound, and position information of the display device 10 to set the information indicating the location of the user U, and may use any environmental information, for example, at least one of the surrounding image, surrounding sound, and position information of the display device 10.

(Motion Score)
The environment determination unit 44 calculates a motion score as an environment score for a category of the motion of the user U. In other words, the motion score is information indicating the motion of the user U, and can be said to be information indicating, as a numerical value, how the user U is moving. The environment determination unit 44 calculates the motion score based on environmental information related to the motion of the user U, among multiple types of environmental information. Examples of environmental information related to the motion of the user U include acceleration information acquired by the acceleration sensor 20D.

5, the category of the user U's movement includes a subcategory of "moving". The environment identification unit 44 calculates a location score for the subcategory of "moving" based on the acceleration information of the display device 10 acquired by the acceleration sensor 20D. The movement score for the subcategory of "moving" can be a numerical value indicating the degree of agreement between the current situation of the user U and the user U's movement. The calculation method of the movement score for the subcategory of "moving" may be arbitrary, but the movement score may be calculated from a change in acceleration over a predetermined period of time, for example. For example, when there is a change in acceleration over a predetermined period of time, the movement score is calculated so that the degree of agreement with the user U's movement is high. For example, the position information of the display device 10 may be acquired, and the movement score may be calculated based on the degree of change in position over a predetermined period of time. In this case, the speed can be predicted from the amount of change in position over a predetermined period of time, and the means of transportation, such as a vehicle or walking, can also be identified. Note that the degree of agreement with the movement is calculated here, but is not limited thereto, and the degree of agreement with the movement at a predetermined speed may be calculated.

In this way, the environment determination unit 44 can be said to set information indicating the movement of the user U (here, the movement score) based on the acceleration information of the display device 10 and the position information of the display device 10. However, the environment determination unit 44 is not limited to using acceleration information and position information to set the information indicating the movement of the user U, and may use any environmental information, for example, at least one of acceleration information and position information.

(Safety Score)
The environment determination unit 44 calculates a safety score as an environment score for the category of safety of the user U. That is, the safety score is information indicating the safety of the user U, and can be said to be information indicating, as a numerical value, whether the user U is in a safe environment. The environment determination unit 44 calculates the safety score based on environmental information related to the safety of the user U among a plurality of types of environmental information. The environmental information related to the safety of the user U includes a surrounding image acquired by the camera 20A, a surrounding sound acquired by the microphone 20B, light intensity information detected by the optical sensor 20F, surrounding temperature information detected by the temperature sensor 20G, and surrounding humidity information detected by the humidity sensor 20H.

5, the category of safety for the user U includes a subcategory of "bright," a subcategory of "appropriate amounts of infrared rays and ultraviolet rays," a subcategory of "appropriate temperature," a subcategory of "appropriate humidity," and a subcategory of "presence of dangerous objects." The environment identification unit 44 calculates a safety score for the "bright" subcategory based on the intensity of the surrounding visible light acquired by the light sensor 20F. The safety score for the "bright" subcategory can be a numerical value indicating the degree of agreement of the surrounding brightness with respect to sufficient brightness. The method of calculating the safety score for the "bright" subcategory may be arbitrary, and may be calculated based on the intensity of visible light detected by the light sensor 20F, for example. Also, for example, the safety score for the "bright" subcategory may be calculated based on the luminance of an image captured by the camera 20A. Note that, although the degree of agreement with sufficient brightness is calculated here, the invention is not limited thereto, and the degree of agreement with respect to any degree of brightness may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of appropriate amounts of infrared and ultraviolet light based on the intensities of infrared and ultraviolet light in the surroundings acquired by the optical sensor 20F. The safety score for the subcategory of appropriate amounts of infrared and ultraviolet light can be a numerical value indicating the degree of agreement between the intensities of infrared and ultraviolet light in the surroundings and the appropriate intensities of infrared and ultraviolet light. The method of calculating the safety score for the subcategory of appropriate amounts of infrared and ultraviolet light may be arbitrary, and may be calculated, for example, based on the intensities of infrared and ultraviolet light detected by the optical sensor 20F. Note that, although the degree of agreement with the appropriate intensities of infrared and ultraviolet light is calculated here, the invention is not limited thereto, and the degree of agreement with any intensity of infrared and ultraviolet light may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of suitable temperature based on the surrounding temperature acquired by the temperature sensor 20G. The safety score for the subcategory of suitable temperature can be a numerical value indicating the degree of agreement of the surrounding temperature with the suitable temperature. The method of calculating the safety score for the subcategory of suitable temperature may be arbitrary, and may be calculated, for example, based on the surrounding temperature detected by the temperature sensor 20G. Note that, although the degree of agreement with the suitable temperature is calculated here, this is not limiting, and the degree of agreement with any temperature may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of suitable humidity based on the surrounding humidity acquired by the humidity sensor 20H. The safety score for the subcategory of suitable humidity can be a numerical value indicating the degree of agreement of the surrounding humidity with the suitable humidity. The method of calculating the safety score for the subcategory of suitable humidity may be arbitrary, but for example, it may be calculated based on the surrounding humidity detected by the temperature sensor 20H. Note that, although the degree of agreement with the suitable humidity is calculated here, it is not limited thereto, and the degree of agreement with any humidity may be calculated.

The environment identification unit 44 calculates a safety score for the subcategory of dangerous object based on the surrounding image acquired by the camera 20A. The safety score for the subcategory of dangerous object is a numerical value indicating the degree of agreement with the presence of a dangerous object. The method of calculating the safety score for the subcategory of dangerous object may be any, but may be determined, for example, in a manner similar to the method of determining whether a dangerous state is present based on the surrounding image as described above, that is, by determining whether the object included in the surrounding image is a specific object. Furthermore, the environment identification unit 44 calculates a safety score for the subcategory of dangerous object based on the surrounding sound acquired by the microphone 20B. The method of calculating the safety score for the subcategory of dangerous object may be any, but may be determined, for example, in a manner similar to the method of determining whether a dangerous state is present based on the surrounding sound as described above, that is, by determining whether the surrounding sound is a specific type of sound.

(An example of an environmental score)
5 illustrates examples of environment scores calculated for environments D1 to D4. Environments D1 to D4 each indicate a case where user U is in a different environment, and an environment score is calculated for each category (subcategory) in each environment.

Note that the types of environmental categories and subcategories shown in FIG. 5 are examples, and the environmental score values for environments D1 to D4 are also examples. Furthermore, by expressing the information indicating user U's environment as a numerical value such as the environmental score, the display device 10 can take errors into account and more accurately estimate user U's environment. In other words, the display device 10 can accurately estimate user U's environment by classifying environmental information into one of three or more degrees (here, environmental scores). However, the information indicating user U's environment that the display device 10 sets based on the environmental information is not limited to a value such as the environmental score, and may be data of any type, for example, information indicating a binary choice of either Yes or No.

(Determination of environmental patterns)
The display device 10 calculates various environmental scores by the above-described method in steps S16 to S22 shown in Fig. 4. As shown in Fig. 4, after calculating the environmental scores, the display device 10 determines an environmental pattern indicating the environment in which the user U is placed based on each environmental score by the environmental identification unit 44 (step S24). That is, the environmental identification unit 44 determines what kind of environment the user U is in based on the environmental scores. While the environmental information and the environmental score are information indicating some elements of the user U's environment detected by the environmental sensor 20, the environmental pattern can be said to be an index that indicates the environment comprehensively, set based on the information indicating those some elements.

FIG. 6 is a table showing an example of an environmental pattern. In this embodiment, the environmental identification unit 44 selects an environmental pattern that matches the environment in which the user U is located from among environmental patterns corresponding to various environments based on the environmental score. In this embodiment, for example, the specification setting database 30C records correspondence information (table) that associates the environmental score value with the environmental pattern. The environmental identification unit 44 determines the environmental pattern based on the environmental information and the correspondence information. Specifically, the environmental identification unit 44 selects an environmental pattern that corresponds to the calculated environmental score value from the correspondence information and selects it as the environmental pattern to be adopted. In the example of FIG. 6, the environmental pattern PT1 indicates that the user U is sitting in a train, the environmental pattern PT2 indicates that the user U is walking on the sidewalk, the environmental pattern PT3 indicates that the user U is walking on a dark sidewalk, and the environmental pattern PT4 indicates that the user U is shopping.

5 and 6, in environment D1, the environmental score for "standing" is 10, and the environmental score for "face orientation horizontal" is 100, so it can be predicted that user U is sitting with his/her face facing almost horizontally. Also, the environmental score for "inside the train" is 90, the environmental score for "on the tracks" is 100, and the environmental score for "train interior sounds" is 90, so it can be seen that user U is inside a train. Also, the environmental score for "moving" is 100, so it can be seen that user U is moving at a constant speed or with acceleration. Also, the environmental score for "bright" is 50, so it can be seen that it is darker than outside because it is inside a train. Also, the environmental scores for "appropriate amount of infrared rays and ultraviolet rays," "appropriate temperature," and "appropriate humidity" are 100, so it can be said to be safe. Also, the environmental score for "presence of dangerous objects" is 10 for visuals and 20 for sounds, so this is also considered to be safe. That is, in environment D1, it can be estimated from each environmental score that user U is moving on a train, sitting in a seat, and is in a safe and comfortable situation, and the environmental pattern of environment D1 is environment pattern PT1, which indicates that user U is sitting on a train.

5 and 6, in environment D2, the environmental score for "standing" is 10, and the environmental score for "face orientation horizontal" is 90, so it can be predicted that user U is sitting with his/her face facing almost horizontally. Also, the environmental score for "inside the train" is 0, the environmental score for "on the tracks" is 0, and the environmental score for "train interior sounds" is 10, so it can be seen that user U is not inside the train. Although not shown here, in environment D2, it can be confirmed that user U is on the road based on the environmental scores of the locations. Also, since the environmental score for "moving" is 100, it can be seen that user U is moving at a constant speed or with acceleration. Also, the environmental score for "bright" is 100, so it can be seen that it is bright outdoors. Also, "appropriate amount of infrared rays and ultraviolet rays" is 80, so it can be seen that there is a slight influence of ultraviolet rays and the like. Also, the environmental scores for "suitable temperature" and "suitable humidity" are 100, so it can be said to be safe. Also, the environmental score for "there is a dangerous object" is 10 for visuals and 20 for sounds, so it can also be said to be safe. That is, in environment D2, it can be inferred from each environmental score that user U is walking on the sidewalk, it is bright outdoors, and no dangerous objects are found, and the environmental pattern of environment D2 is environment pattern PT2, which indicates walking on the sidewalk.

5 and 6, in environment D3, the environmental score for "standing" is 0, and the environmental score for "face orientation horizontal" is 90, so it can be predicted that user U is sitting with his/her face facing almost horizontally. Also, the environmental score for "inside the train" is 5, the environmental score for "on the tracks" is 0, and the environmental score for "train noise" is 5, so it can be seen that user U is not inside the train. Although not shown here, in environment D3, it can be confirmed that user U is on the road based on the environmental scores of the locations. Also, the environmental score for "moving" is 100, so it can be seen that user U is moving at a constant speed or with acceleration. Also, the environmental score for "bright" is 10, so it can be seen that it is a dark environment. Also, the environmental score for "appropriate amount of infrared and ultraviolet rays" is 100, so it can be seen that it is safe. Also, the environmental score for "appropriate temperature" is 75, so it can be said that it is hotter or colder than normal. Furthermore, the environmental score for "dangerous object present" is 90 visually and 80 audioly, so it is clear that something is making a sound and approaching. Although not shown, the object can be determined from the sound and image, and in this case, it can be determined that a car is approaching from the front and the sound is the sound of a car engine. That is, in environment D3, it can be estimated from each environmental score that user U is walking on the sidewalk, it is dark outdoors, and a vehicle is approaching as a dangerous object, and the environmental pattern of environment D3 is environmental pattern PT3, which indicates walking on a dark sidewalk.

5 and 6, in environment D4, the environmental score for "standing" is 0, and the environmental score for "face orientation horizontal" is 90, so it can be predicted that user U is sitting with his face facing almost horizontally. In addition, the environmental score for "inside the train" is 20, the environmental score for "on the tracks" is 0, and the environmental score for "train noise" is 5, so it can be seen that user U is not inside the train. Although not shown here, in environment D3, it can be confirmed that user U is in a shopping center based on the environmental scores of the locations. In addition, the environmental score for "moving" is 80, so it can be seen that user U is moving slowly. In addition, the environmental score for "bright" is 70, so it can be predicted that it is relatively bright, but the brightness is about the same as indoor lighting. In addition, "appropriate amount of infrared and ultraviolet rays" is 100, so it can be seen that it is safe. In addition, the environmental score for "appropriate temperature" is 100, which is comfortable, but the environmental score for "appropriate humidity" is 90, so it cannot be said to be comfortable. Furthermore, the environmental score for "dangerous objects present" is 10 for visuals and 20 for sound, so this is also considered safe. That is, in environment D4, it can be estimated from each environmental score that user U is walking through a shopping center, the surroundings are relatively bright, and there are no dangerous objects, and the environmental pattern of environment D4 is environmental pattern PT4, which indicates that the user is shopping.

(Setting the target device and standard output specifications)
Once the environmental pattern has been selected, as shown in FIG. 4, the display device 10 uses the output selection unit 48 and the output specification determination unit 50 to select the target devices to be operated from the output unit 26 based on the environmental pattern, and sets the standard output specifications (step S26).

As described above, the target device is a device among the output unit 26 that is to be activated, and in this embodiment, the output selection unit 48 selects the target device from among the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C based on the environmental pattern. Since the environmental pattern is information that indicates the current environment of the user U, by selecting the target device based on the environmental pattern, it is possible to select an appropriate sensory stimulation according to the current environment of the user U.

The output specification determination unit 50 determines the standard output specification, which is the standard output specification, based on the environmental pattern. The output specification is an index indicating how the stimulus output by the output unit 26 is output. For example, the output specification of the display unit 26A indicates how the sub-image PS to be output is displayed, and can be rephrased as the display specification. In this embodiment, the output specification of the display unit 26A includes the display time of the sub-image PS per unit time. The output specification determination unit 50 determines the display time of the sub-image PS per unit time based on the environmental pattern. Note that the output specification determination unit 50 may specify the display time of the sub-image PS per unit time by changing the time for displaying the sub-image PS per one time, or may specify the display time of the sub-image PS per unit time by changing the frequency of displaying the sub-image PS, or may combine both of these. In this way, the visual stimulus given to the user U can be changed by changing the display time of the sub-image PS per unit time. For example, it can be said that the longer the display time, the stronger the visual stimulus given to the user U.

In addition, the output specifications of the display unit 26A include a display mode that shows how the sub-image PS is displayed when the sub-image PS is assumed to be viewed as a still image. The display mode will be described in more detail. Figures 7 to 9 are diagrams showing examples of when the display mode is changed. For example, the display mode may be the display position of the sub-image PS, that is, the position where the sub-image PS is displayed in the display screen of the display unit 26A. Figure 7 shows an example in which the display position of the sub-image PS is changed. As shown in Figure 7, the sub-image PS is displayed superimposed on the main image PM, so that the display position of the sub-image PS can be said to be the relative position of the sub-image PS with respect to the main image PM. Therefore, when the display position of the sub-image PS is changed, the distance between the reference position C of the main image PM and the sub-image PS changes. Here, the reference position C is the center position of the main image PM (display unit 26A). In this way, by changing the display position of the sub-image PS, the degree of visual stimulation to the user U by the sub-image PS can be changed. For example, the closer the sub-image PS is to the reference position C at the center, the stronger the degree of visual stimulation to the user U can be.

Another display mode is a decoration display, which is an image that decorates the content (display contents) included in the sub-image PS. In this embodiment, the decoration display indicates the degree to which the sub-image PS, which is an advertisement, is emphasized. FIG. 8 shows an example of a decoration display in which the size of the sub-image PS is changed. FIG. 9 shows an example of a decoration display in which the presence or absence and content of a decoration image added to the content of the sub-image PS are changed. The example of FIG. 9 shows an example in which the presence or absence and number of decoration images "!" are changed for the content (display contents) "AAAA". Note that the content of the decoration image may be arbitrary. In this way, by changing the decoration display, the visual stimulation given to the user U can be changed. For example, the larger the sub-image PS is and the more decoration images there are, the stronger the degree of visual stimulation to the user U can be.

In this embodiment, as described above, the display position of the sub-image PS and the modified display are exemplified as the display mode, but the display mode is not limited to these and may be any display mode. However, it is preferable that the display mode is not the content of the sub-image PS, i.e., in this case, it is not the content of the advertisement. In other words, it is preferable that the content of the sub-image PS itself is not changed as the display mode. Note that, if multiple types of display modes are expected, only one of them may be changed, or multiple types of display modes may be changed.

In this way, the output specification determination unit 50 determines, as the output specification of the display unit 26A, at least one of the display time of the sub-image PS per unit time and the display mode of the sub-image PS, based on the environmental pattern. In other words, the output specification determination unit 50 may determine, as the output specification of the display unit 26A, both the display time of the sub-image PS per unit time and the display mode of the sub-image PS, or may determine only one of them.

The output specifications of the display unit 26A have been described above, but the output specification determination unit 50 also determines the output specifications of the audio output unit 26B and the tactile stimulation output unit 26C. Examples of the output specifications (audio specifications) of the audio output unit 26B include the volume, and the presence or absence and degree of sound. Sound refers to special effects such as surround and three-dimensional sound fields. The higher the volume or the greater the degree of sound, the stronger the degree of auditory stimulation to the user U. In addition, examples of the output specifications of the tactile stimulation output unit 26C include the strength of the tactile stimulation and the frequency with which the tactile stimulation is output. The higher the strength or frequency of the tactile stimulation, the stronger the degree of tactile stimulation to the user U.

10 is a table showing the relationship between the environmental pattern and the target device and the standard output specification. The output selection unit 48 and the output specification determination unit 50 determine the target device and the standard output specification based on the relationship information showing the relationship between the environmental pattern and the target device and the standard output specification. The relationship information is information (table) in which the environmental pattern, the target device, and the standard output specification are stored in association with each other, and is stored, for example, in the specification setting database 30C. In the relationship information, the standard output specification is set for each type of output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C in this case. The output selection unit 48 and the output specification determination unit 50 determine the target device and the standard output specification based on this relationship information and the environmental pattern set by the environmental identification unit 44. Specifically, the output selection unit 48 and the output specification determination unit 50 read out the relationship information, select the target device and the standard output specification associated with the environmental pattern set by the environmental identification unit 44 from the relationship information, and determine the target device and the standard output specification.

In the example of FIG. 10, for environmental pattern PT1, which is assumed to be sitting on a train, the display unit 26A, audio output unit 26B, and tactile stimulation output unit 26C are all set as target devices, and their standard output specification level is assigned to 4. Note that a higher level indicates a stronger output stimulation. For environmental pattern PT2, which is assumed to be walking on the sidewalk, the situation is generally safe and comfortable, but since the person is walking, attention to the front is considered to be necessary, so the display unit 26A, audio output unit 26B, and tactile stimulation output unit 26C are all set as target devices, and their standard output specification level is assigned to 3. For environmental pattern PT3, in which the subject is walking on a dark sidewalk, the situation is not considered to be safe, and the subject must be gazing ahead and be able to hear external sounds clearly. Therefore, the audio output unit 26B and the tactile stimulation output unit 26C are set as target devices, and the standard output specification levels of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C are assigned to 0, 2, and 2, respectively. For environmental pattern PT4, in which the subject is shopping, the subject is almost safe, but since it is a shopping center, it is assumed that the provision of information that is distracting is unnecessary. Therefore, the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C are all set as target devices, and the standard output specification level of these units is assigned to 2. However, the assignment of target devices and standard output specifications for each environmental pattern in FIG. 10 is merely an example, and may be set as appropriate.

In this manner, in this embodiment, the display device 10 sets the target device and the reference output specifications based on a preset relationship between the environmental pattern and the target device and the reference output specifications. However, the method of setting the target device and the reference output specifications is not limited to this, and the display device 10 may set the target device and the reference output specifications in any manner based on the environmental information detected by the environmental sensor 20. Furthermore, the display device 10 is not limited to selecting both the target device and the reference output specifications based on the environmental information, and may select at least one of the target device and the reference output specifications.

(Acquisition of Biometric Information)
As shown in FIG. 4, the display device 10 acquires the bioinformation of the user U detected by the biosensor 22 by the bioinformation acquisition unit 42 (step S28). The bioinformation acquisition unit 42 acquires the pulse wave information of the user U from the pulse wave sensor 22A, and acquires the brain wave information of the user U from the brain wave sensor 22B. FIG. 11 is a graph showing an example of a pulse wave. As shown in FIG. 11, the pulse wave has a waveform in which a peak called an R wave WR appears at a predetermined time. The heart is controlled by the autonomic nervous system, and the pulse rate is controlled by generating an electrical signal at the cellular level that triggers the heart to move. Usually, the pulse rate increases when adrenaline is secreted due to the excitement of the sympathetic nerves, and decreases when acetylcholine is secreted due to the excitement of the parasympathetic nerves. According to Nobuyuki Ueda, "Evaluation of Diabetic Autonomic Neuropathy Using Power Spectral Analysis of Electrocardiogram R-R Intervals" (Diabetes 35 (1): 17-23, 1992), the function of the autonomic nerves can be understood by examining the variation of the R-R interval in the time waveform of the pulse wave as shown in the example of FIG. 11. The R-R sensation is the interval between consecutive R waves WR in a time series. At the cellular level, an electrocardiogram is a repetition of depolarization/action potential and repolarization/resting potential, and an electrocardiogram can be detected by detecting this electrical activity from the body surface. The pulse wave is very fast and is transmitted throughout the body almost simultaneously with the heartbeat, so it can be said that the heartbeat is synchronized with the pulse wave. Since the pulse wave of the heart and the R wave of the electrocardiogram are synchronized, the R-R interval of the pulse wave can be considered to be equivalent to the R-R interval of the electrocardiogram. Fluctuations in the R-R interval of the pulse wave can be considered as time derivatives, so by calculating the derivatives and detecting the magnitude of the fluctuations, it is possible to predict to a certain extent the degree of activation or calming of the autonomic nervous system of the living body, i.e., irritation due to mental disturbance, discomfort due to being on a crowded train, stress that occurs over a relatively short period of time, etc., almost regardless of the wearer's wishes.

On the other hand, by detecting the basic rhythmic (background brain wave) activity that appears throughout the brain, such as alpha and beta waves, and detecting their amplitude, it is possible to predict to some extent whether the activity of the brain as a whole is increasing or decreasing. For example, the degree of activity of the prefrontal cortex of the brain can reveal the level of attention, such as the level of interest shown in an object that is visually stimulated.

(Identifying user status and calculating output specification correction degree)
4, after acquiring the biometric information, the display device 10 causes the user state identifying unit 46 to identify the user state indicating the mental state of the user U based on the biometric information of the user U, and calculates an output specification correction degree based on the user state (step S30). The output specification correction degree is a value for correcting the standard output specification set by the output specification determining unit 50, and the final output specification is determined based on the standard output specification and the output specification correction degree.

Figure 12 is a table showing an example of the relationship between the user state and the output specification correction degree. In this embodiment, the user state identification unit 46 identifies the brain activity of the user U as the user state based on the brain wave information of the user U. The user state identification unit 46 may identify the brain activity of the user U by any method based on the brain wave information of the user U, for example, the brain activity may be identified from a specific frequency range for the waveforms of alpha waves and beta waves. In this case, for example, the user state identification unit 46 performs a fast Fourier transform on the time waveform of the brain wave to calculate the power spectrum amount of the high frequency part of the alpha wave (for example, 10 Hz to 11.75 Hz). When the power spectrum amount of the high frequency part of the alpha wave is large, it can be expected that the user is very concentrated while being relaxed, so the user state identification unit 46 determines that the brain activity is higher as the power spectrum amount of the high frequency part of the alpha wave is large. The user state identification unit 46 defines the brain activity level when the power spectrum amount of the high frequency part of α waves is within a predetermined numerical range as VA3, defines the brain activity level when the power spectrum amount of the high frequency part of α waves is within a predetermined numerical range lower than the numerical range when the brain activity level is VA3 as VA2, and defines the brain activity level when the power spectrum amount of the high frequency part of α waves is within a predetermined numerical range lower than the numerical range when the brain activity level is VA2 as VA1. Here, the brain activity levels are considered to be increasing in the order of VA1, VA2, and VA3. Note that the greater the power spectrum amount of the high frequency component of β waves (e.g., 18 Hz to 29.75 Hz), the more likely it is that the user is psychologically "alert" or "agitated," so the power spectrum amount of the high frequency component of β waves may also be used to identify the brain activity level.

The user state identification unit 46 determines the output specification correction degree based on the brain activity of the user U. In this embodiment, the output specification correction degree is determined based on the output specification correction degree relationship information indicating the relationship between the user state (brain activity in this example) and the output specification correction degree. The output specification correction degree relationship information is information (table) in which the user state and the output specification correction degree are associated and stored, and is stored, for example, in the specification setting database 30C. In the output specification correction degree relationship information, the output specification correction degree is set for each type of the output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C in this case. The user state identification unit 46 determines the output specification correction degree based on this output specification correction degree relationship information and the identified user state. Specifically, the user state identification unit 46 reads out the output specification correction degree relationship information, selects the output specification correction degree associated with the set brain activity of the user U from the output specification correction degree relationship information, and determines the output specification correction degree. In the example of FIG. 12, the output specification correction degree of the display unit 26A, the audio output unit 26B, and the tactile stimulus output unit 26C is set to -1 for the brain activity level VA3, the output specification correction degree of the display unit 26A, the audio output unit 26B, and the tactile stimulus output unit 26C is set to 0 for the brain activity level VA2, and the output specification correction degree of the display unit 26A, the audio output unit 26B, and the tactile stimulus output unit 26C is set to 1 for the brain activity level VA1. The output specification correction degree here is set to a value that increases the output specification as the value increases. In other words, the user state identification unit 46 sets the output specification correction degree so that the output specification is increased as the brain activity level decreases. Note that increasing the output specification here refers to strengthening the sensory stimulus, and the same applies hereinafter. The value of the output specification correction degree in FIG. 12 is an example and may be set appropriately.

Furthermore, the user state identification unit 46 identifies the mental stability of the user U as the user state based on the pulse wave information of the user U. In this embodiment, the user state identification unit 46 calculates the fluctuation value of the interval length between successive R waves WH in a time series, i.e., the differential value of the R-R interval, from the brain wave information of the user U, and identifies the brain activity of the user U based on the differential value of the R-R interval. The user state identification unit 46 identifies the mental stability of the user U as being higher when the differential value of the R-R interval is smaller, i.e., the interval length between the R waves WH does not fluctuate. In the example of FIG. 12, the user state identification unit 46 classifies the mental stability of the user U into one of three types, VB3, VB2, and VB1, based on the pulse wave information of the user U. The user state identification unit 46 defines mental stability as VB3 when the differential value of the R-R interval is within a predetermined numerical range, as VB2 when the differential value of the R-R interval is within a predetermined numerical range higher than the numerical range when the mental stability is VB3, and as VB1 when the differential value of the R-R interval is within a predetermined numerical range lower than the numerical range when the mental stability is VB2. Note that the order of mental stability is VB1, VB2, and VB3.

The user state identification unit 46 determines the output specification correction degree based on the output specification correction degree related information and the identified mental stability. Specifically, the user state identification unit 46 reads out the output specification correction degree related information, selects the output specification correction degree associated with the set mental stability of the user U from the output specification correction degree related information, and determines the output specification correction degree. In the example of FIG. 12, for mental stability VB3, the output specification correction degrees of the display unit 26A, the audio output unit 26B, and the tactile stimulus output unit 26C are each set to 1, for mental stability VB2, the output specification correction degrees of the display unit 26A, the audio output unit 26B, and the tactile stimulus output unit 26C are each set to 0, and for mental stability VB1, the output specification correction degrees of the display unit 26A, the audio output unit 26B, and the tactile stimulus output unit 26C are each set to -1. That is, the user state identification unit 46 sets the output specification correction degree so that the higher the mental stability, the higher the output specification (sensory stimulation). Note that the value of the output specification correction degree in FIG. 12 is an example and may be set as appropriate.

In this way, the user state identification unit 46 sets the output specification correction degree based on the relationship between the user state and the output specification correction degree that is set in advance. However, the method of setting the output specification correction degree is not limited to this, and the display device 10 may set the output specification correction degree by any method based on the bioinformation detected by the biosensor 22. In addition, the display device 10 calculates the output specification correction degree using both the brain activity level identified from the brain wave and the mental stability identified from the pulse wave, but is not limited to this. For example, the display device 10 may calculate the output specification correction degree using one of the brain activity level identified from the brain wave and the mental stability identified from the pulse wave. In addition, the display device 10 treats the bioinformation as a numerical value, and by estimating the user state based on the bioinformation, it is possible to take into account errors in the bioinformation, etc., and it is possible to more accurately estimate the psychological state of the user U. In other words, it can be said that the display device 10 can accurately estimate the psychological state of the user U by classifying the bioinformation and the user state based on the bioinformation into one of three or more degrees. However, the display device 10 is not limited to classifying the biometric information or the user state based on the biometric information into three or more degrees, and may treat it as information indicating, for example, a binary option of either Yes or No.

(Generation of output restriction necessity information)
As shown in FIG. 4, the display device 10 generates output restriction necessity information based on the biometric information of the user U by the user state identification unit 46 (step S32). FIG. 13 is a table showing an example of output restriction necessity information. The output restriction necessity information is information indicating whether or not output restriction of the output unit 26 is necessary, and can be said to be information indicating whether or not the operation of the output unit 26 is permitted. The output restriction necessity information is generated for each output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C. In other words, the user state identification unit 46 generates output restriction necessity information indicating whether or not the operation of each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C is permitted based on the biometric information. More specifically, the user state identification unit 46 generates output restriction necessity information based on both the biometric information and the environmental information. The user state identification unit 46 generates output restriction necessity information based on the user state set based on the biometric information and the environmental score calculated based on the environmental information. In the example of Fig. 13, the user state identification unit 46 generates output restriction necessity information based on the brain activity level as the user state and the location score for the subcategory of being on the tracks as the environment score. In the example of Fig. 13, when the location score for the subcategory of being on the tracks is 100 and the brain activity level is VA3 or VA2, the user state identification unit 46 generates output restriction necessity information that prohibits the use of the display unit 26A. Also, in the example of Fig. 13, the user state identification unit 46 generates output restriction necessity information based on the brain activity level as the user state and the movement score for the subcategory of moving as the environment score. In the example of Fig. 13, the user state identification unit 46 generates output restriction necessity information that prohibits the use of the display unit 26A when the movement score for the subcategory of moving is 0 and the brain activity level is VA3 or VA2. In this way, when the biometric information and the environmental information satisfy a specific relationship, in this case when the user state and the environmental score satisfy a specific relationship, the user state identification unit 46 generates output restriction necessity information that prohibits the use of the display unit 26A. On the other hand, when the user state and the environmental score do not satisfy the specific relationship, the user state identification unit 46 does not generate output restriction necessity information that prohibits the use of the display unit 26A, but generates output restriction necessity information that permits the use of the display unit 26A. However, generating the output restriction necessity information is not a required process.

(Acquisition of sub-image)
4, the display device 10 acquires image data of the sub-image PS by the sub-image acquisition unit 52 (step S34). The image data of the sub-image PS is image data for displaying the content (display contents) of the sub-image. The sub-image acquisition unit 52 acquires the image data of the sub-image from an external device via the sub-image receiving unit 28A.

The sub-image acquisition unit 52 may acquire image data of a sub-image of a content (display content) corresponding to the position (earth coordinates) of the display device 10 (user U). The position of the display device 10 is specified by the GNSS receiver 20C. For example, when the user U is located within a predetermined range of a certain position, the sub-image acquisition unit 52 receives content related to that position. In principle, the display of the sub-image PS can be controlled by the user U's will, but if the display is set to be possible, it is not known when, where, and at what timing it will be displayed, which is convenient but may also be a nuisance. Therefore, the specification setting database 30C may record information indicating whether the sub-image PS set by the user U can be displayed and the display specifications. The sub-image acquisition unit 52 reads out this information from the specification setting database 30C and controls the acquisition of the sub-image PS based on this information. The position information and the specification setting database 30C may also be written with the same information on a site on the Internet, and the sub-image acquisition unit 52 may control the acquisition of the sub-image PS while checking the contents. Note that step S34 of acquiring image data of the sub-image PS does not necessarily have to be performed before step S36 described below, but may be performed at any time before step S38 described below.

The sub-image acquisition unit 52 may also acquire audio data and tactile stimulation data related to the sub-image PS along with the image data of the sub-image PS. The audio output unit 26B outputs the audio data related to the sub-image PS as audio content (audio content), and the tactile stimulation output unit 26C outputs the tactile stimulation data related to the sub-image PS as tactile stimulation content (tactile stimulation content).

(Output specification settings)
4, the display device 10 determines output specifications based on the standard output specifications and the output specification correction degree by the output specification determination unit 50 (step S36). The output specification determination unit 50 corrects the standard output specifications set based on the environmental information by the output specification correction degree set based on the biometric information, thereby determining the output specifications as the final output specifications for the output unit 26. The formula for correcting the standard output specifications by the output specification correction degree may be arbitrary.

As described above, the display device 10 determines the final output specifications by correcting the standard output specifications set based on the environmental information with the output specification correction degree set based on the biometric information. However, the display device 10 is not limited to determining the output specifications by correcting the standard output specifications with the output specification correction degree, and may determine the output specifications in any manner using at least one of the environmental information and the biometric information. In other words, the display device 10 may determine the output specifications in any manner based on the environmental information and the biometric information, or may determine the output specifications in any manner based on either the environmental information or the biometric information.

When output restriction necessity information that disallows use of the output unit 26 is generated in step S32, the output selection unit 48 selects the target device based not only on the environmental score but also on the output restriction necessity information. That is, even if the output unit 26 was selected as the target device based on the environmental score in step S26, if the specifications are disallowed in the output restriction necessity information, it is excluded from the target devices. In other words, the output selection unit 48 selects the target device based on the output restriction necessity information and the environmental information. Furthermore, since the output restriction necessity information is set based on the biometric information, it can be said that the target device is set based on the biometric information and the environmental information.

(Output Control)
After setting the target device and the output specifications and acquiring the image data of the sub-image PS, the display device 10 causes the target device to output based on the output specifications (step S38) by the output control unit 54, as shown in Fig. 4. The output control unit 54 does not operate the output unit 26 that is not set as the target device.

For example, when the display unit 26A is set as the target device, the output control unit 54 causes the display unit 26A to display the sub-image PS based on the sub-image data acquired by the sub-image acquisition unit 52 in accordance with the output specifications of the display unit 26A. More specifically, the output control unit 54 causes the display unit 26A to display the sub-image PS so as to be superimposed on the main image PM provided through the display unit 26A and in accordance with the output specifications of the display unit 26A. Since the output specifications are set based on environmental information and biological information as described above, the sub-image PS can be displayed in an appropriate manner according to the environment in which the user U is placed and the psychological state of the user U by displaying the sub-image PS in accordance with the output specifications. For example, when the display time per unit time of the sub-image PS is set as the output specifications, the display time of the sub-image PS becomes appropriate according to the environment in which the user U is placed and the psychological state of the user U, so that the sub-image can be appropriately provided to the user U. More specifically, for example, the higher the brain activity of the user U or the lower the mental stability of the user U, the shorter the display time of the sub-image PS and the less visual stimulation, so that the user U can be less bothered by the sub-image PS when he is concentrating on something else or has no time for it. On the other hand, when the user U is bored or has time for it, the longer the display time and the stronger the visual stimulation, so that the user U can appropriately obtain information from the sub-image PS. Also, for example, when the display mode of the sub-image PS (display position of the sub-image, size of the sub-image, decoration image, etc.) is set as the output specification, the display mode of the sub-image PS becomes appropriate according to the environment in which the user U is placed and the psychological state of the user U, so that the sub-image can be appropriately provided to the user U. More specifically, for example, the higher the brain activity of the user U or the lower the mental stability of the user U, the smaller the sub-image or the fewer decoration images, so that the visual stimulation can be reduced and the user U can be less bothered by the sub-image PS. On the other hand, for example, if the user U has a lower brain activity or a higher mental stability, the visual stimulation can be strengthened by positioning the sub-image closer to the center, making the sub-image larger, or increasing the number of modifying images, allowing the user U to obtain appropriate information from the sub-image PS.

Furthermore, when the audio output unit 26B is the target device, the output control unit 54 causes the audio output unit 26B to output audio based on the audio data acquired by the sub-image acquisition unit 52 in accordance with the output specifications of the audio output unit 26B. Even in this case, for example, the higher the brain activity level of the user U or the lower the mental stability of the user U, the weaker the auditory stimulation can be made, thereby reducing the risk of the user U being annoyed by the audio when he or she is concentrating on something else or is not mentally stable. On the other hand, the lower the brain activity level of the user U or the higher the mental stability of the user U, the stronger the auditory stimulation can be made, allowing the user U to obtain information appropriately through the audio.

Furthermore, when the tactile stimulus output unit 26C is determined to be the target device, the output control unit 54 causes the tactile stimulus output unit 26C to output tactile stimuli based on the tactile stimulus data acquired by the sub-image acquisition unit 52 in accordance with the output specifications of the tactile stimulus output unit 26C. Even in this case, for example, the higher the brain activity level of the user U or the lower the mental stability of the user U, the weaker the tactile stimulus can be made, thereby reducing the risk of the user U being annoyed by the tactile stimulus when he or she is concentrating on something else or is not mentally stable. On the other hand, the lower the brain activity level of the user U or the higher the mental stability of the user U, the stronger the tactile stimulus can be made, allowing the user U to obtain appropriate information through the tactile stimulus.

In addition, if a dangerous state is determined in step S12 and the content of the danger notification is set, the output control unit 54 notifies the target device of the danger notification content in accordance with the set output specifications.

In this way, the display device 10 according to this embodiment can output sensory stimuli at an appropriate level according to the environment in which the user U is placed and the psychological state of the user U by setting the output specifications based on the environmental information and the biometric information. Furthermore, the display device 10 can select appropriate sensory stimuli according to the environment in which the user U is placed and the psychological state of the user U by selecting the target device to be operated based on the environmental information and the biometric information. However, the display device 10 is not limited to using both the environmental information and the biometric information, and may use only one of them, for example. Therefore, the display device 10 can be said to select the target device and set the output specifications based on the environmental information, and to select the target device and set the output specifications based on the biometric information.

(effect)
As described above, the display device 10 according to the present embodiment includes the display unit 26A for displaying an image, the biosensor 22 for detecting biometric information of the user U, the output specification determination unit 50 for determining the display specification (output specification) of the sub-image PS to be displayed on the display unit 26A based on the biometric information of the user U, and the output control unit 54 for displaying the sub-image PS on the display unit 26A so as to be superimposed on the main image PM visible to the user U and conform to the display specification. The display device 10 according to the present embodiment can provide an appropriate image to the user U by superimposing the sub-image PS on the main image PM. Furthermore, the display specification of the sub-image PS to be superimposed on the main image PM is set based on the biometric information, so that the sub-image PS can be appropriately provided according to the state of the user U.

The biometric information also includes information related to the autonomic nerves of the user U, and the output specification determination unit 50 determines the display specifications of the sub-image PS based on the information related to the autonomic nerves of the user. The display device 10 according to this embodiment determines the display specifications from the biometric information related to the autonomic nerves of the user U, and can provide an appropriate sub-image PS according to the psychological state of the user U.

The display device 10 further includes an environmental sensor 20 that detects environmental information around the display device 10. The output specification determination unit 50 determines the display specifications of the sub-image PS based on the environmental information and biometric information of the user U. The display device 10 according to this embodiment determines the display specifications based on the environmental information in addition to the biometric information of the user U, and can therefore provide an appropriate sub-image PS according to the environment in which the user U is placed and the state of the user U.

The environmental information also includes location information of user U. The output specification determination unit 50 determines the display specifications of the sub-image PS based on the location information of user U and the biometric information of user U. The display device 10 according to this embodiment determines the display specifications based on the location of user U in addition to the biometric information of user U, and can therefore provide an appropriate sub-image PS according to the location of user U and the state of user U.

The output specification determination unit 50 also classifies the biometric information of the user U into three or more degrees and determines the display specifications of the sub-image PS according to the degree of classification. By classifying the biometric information of the user U into three or more degrees, the display device 10 according to this embodiment can grasp the condition of the user U in detail and determine the display specifications of the sub-image PS based on that, thereby providing the sub-image PS more appropriately according to the condition of the user U.

The output specification determination unit 50 also determines the display time of the sub-image PS per unit time as the display specification of the sub-image PS. The display device 10 according to this embodiment can provide an appropriate sub-image PS according to the state of the user U by adjusting the display time of the sub-image PS based on the biometric information.

The output specification determination unit 50 also determines the display mode, which indicates how the sub-image PS is to be displayed when viewed as a still image, as the display specification for the sub-image PS. The display device 10 according to this embodiment can provide an appropriate sub-image PS according to the state of the user U by adjusting the display mode of the sub-image PS based on the biometric information.

Second Embodiment
Next, a second embodiment will be described. The display device 10 according to the second embodiment differs from the first embodiment in that it also acquires advertising fee information for the sub-image PS and determines the output specifications of the sub-image PS based on the advertising fee information. That is, in the second embodiment, the sub-image PS includes advertising information. Descriptions of parts of the second embodiment that are common to the first embodiment will be omitted.

Figure 14 is a flow chart for explaining the processing contents of the display device according to the second embodiment. As shown in Figure 14, the display device 10 according to the second embodiment performs the same processing from step S10 to step S32 as in the first embodiment, so the explanation will be omitted. On the other hand, the sub-image acquisition unit 52 of the display device 10 according to the second embodiment acquires advertising fee information of the sub-image PS in addition to image data of the sub-image PS (step S34a). The advertising fee information is information on the advertising fee (cost) paid by the advertiser when the sub-image PS, which is an advertisement, is displayed on the display device 10, and can also be said to be information on the advertising fee paid to display the advertising information included in the sub-image PS. In addition, the advertising fee information can be said to be information indicating the degree of the advertising fee, that is, the degree of the high advertising fee. The advertising fee of the sub-image PS is determined, for example, between the advertiser and the communication company. The advertising fee information is set for each sub-image PS, that is, for each advertisement, and is associated with the image data of the sub-image PS. That is, the sub-image acquisition unit 52 of the second embodiment acquires the image data of the sub-image PS and the advertising fee information associated with the sub-image PS.

Then, the display device 10 according to the second embodiment determines the output specifications by the output specification determination unit 50 based on the advertising fee information in addition to the standard output specifications and the output specification correction degree (step S36a). That is, in the second embodiment, the output specifications are determined based on the standard output specifications set from the environmental information, the output specification correction degree set from the biometric information, and the advertising fee information.

Specifically, the output specification determination unit 50 sets an advertising fee correction degree for correcting the standard output specification based on the advertising fee information. The output specification determination unit 50 sets the advertising fee correction degree so that the higher the advertising fee in the advertising fee information, the higher the output specification (sensory stimulation). In the example of this embodiment, the output specification determination unit 50 determines the advertising fee correction degree based on advertising fee correction degree relationship information indicating the relationship between the advertising fee information and the advertising fee correction degree. The advertising fee correction degree relationship information is information (table) in which the advertising fee information and the advertising fee correction degree are associated and stored, and is stored, for example, in the specification setting database 30C. In the advertising fee correction degree relationship information, an advertising fee correction degree is set for each type of output unit 26, that is, here, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C. The output specification determination unit 50 determines the advertising fee correction degree based on this advertising fee correction degree relationship information and the acquired advertising fee information. Specifically, the user state identification unit 46 reads out the advertising fee correction rate related information, selects the advertising fee correction rate associated with the acquired advertising fee information from the advertising fee correction rate related information, and determines the advertising fee correction rate.

In this way, the output specification determination unit 50 sets the advertising fee correction rate based on pre-set advertising fee correction rate relationship information that associates the advertising fee information with the advertising fee correction rate. However, the method of setting the advertising fee correction rate is not limited to this, and the display device 10 may set the advertising fee correction rate in any manner based on the advertising fee information.

The output specification determination unit 50 corrects the standard output specifications with the output specification correction degree set based on the biometric information and the advertising fee correction degree set based on the advertising fee information to obtain the output specifications. The formula for correcting the standard output specifications with the output specification correction degree and the advertising fee correction degree may be arbitrary. Once the output specifications have been determined in this manner, the output control unit 54 of the second embodiment causes the target device to output based on the output specifications in the same manner as in the first embodiment (step S38).

As described above, the advertising fee correction rate is set so that the higher the advertising fee, the stronger the sensory stimulation. Therefore, for example, the higher the advertising fee, the longer the display time per unit time of the sub-image PS. Also, for example, the higher the advertising fee, the more the sub-image PS is displayed closer to the center as shown in FIG. 7, or larger as shown in FIG. 8, or the more decorative images are displayed as shown in FIG. 9.

As described above, the display device 10 according to the second embodiment determines the final output specifications by correcting the standard output specifications set based on the environmental information with the output specification correction degree set based on the biometric information and the advertising fee correction degree set based on the advertising fee information. However, the display device 10 according to the second embodiment is not limited to determining the output specifications by correcting the standard output specifications with the output specification correction degree and the advertising fee correction degree, and may determine the output specifications in any manner using at least the advertising fee information. That is, for example, the display device 10 according to the second embodiment may determine the output specifications in any manner using all of the advertising fee information, the environmental information, and the biometric information, may determine the output specifications in any manner using either the environmental information or the biometric information in addition to the advertising fee information, or may determine the output specifications in any manner using only the advertising fee information from among the advertising fee information, the environmental information, and the biometric information.

As described above, the display device 10 according to the second embodiment includes the display unit 26A for displaying an image, the sub-image acquisition unit 52, the output specification determination unit 50, and the output control unit 54. The sub-image acquisition unit 52 of the second embodiment acquires image data of the sub-image PS including the advertising information to be displayed on the display unit 26A, and advertising fee information paid to display the advertising information. The output specification determination unit 50 of the second embodiment determines a display mode indicating how the sub-image is displayed when viewed as a still image as the output specification (display specification) of the sub-image PS based on the advertising fee information. The output control unit 54 of the second embodiment causes the display unit 26A to display the sub-image PS so as to be superimposed on the main image PM visible to the user U provided through the display unit 26A and to conform to the output specification (display specification). The display device 10 according to the second embodiment determines the display mode of the sub-image PS, which is an advertisement, based on the advertising fee, and therefore can appropriately provide the sub-image PS while appropriately reflecting the intention of the advertiser.

The display device 10 according to the second embodiment includes a display unit 26A for displaying an image, a sub-image acquisition unit 52, an output specification determination unit 50, and an output control unit 54. The sub-image acquisition unit 52 of the second embodiment acquires image data of a sub-image PS including advertising information to be displayed on the display unit 26A, and advertising fee information paid to display the advertising information. The output specification determination unit 50 of the second embodiment determines the display time of the sub-image PS per unit time based on the advertising fee information. The output control unit 54 of the second embodiment causes the display unit 26A to display the sub-image PS so as to be superimposed on the main image PM visible to the user U provided through the display unit 26A and to comply with the output specification (display specification). The display device 10 according to the second embodiment determines the display time of the sub-image PS, which is an advertisement, based on the advertising fee, and therefore can appropriately provide the sub-image PS while appropriately reflecting the intention of the advertiser.

Third Embodiment
Next, a third embodiment will be described. The display device 10b according to the third embodiment is different from the first embodiment in that the position at which the sub-image PS is displayed is determined based on permission information indicating whether the sub-image PS may be displayed superimposed on an actual object in the main image PM. In the third embodiment, the description of the parts that are common to the first embodiment may be omitted. The third embodiment is also applicable to the second embodiment.

Figure 15 is a schematic block diagram of a display device according to the third embodiment. As shown in Figure 15, the control unit 32b of the display device 10b according to the third embodiment includes an object identification unit 60 and a permission information acquisition unit 62. The object identification unit 60 identifies an object that is an actual object captured in the main image PM based on the environmental information detected by the environmental sensor 20. The permission information acquisition unit 62 acquires permission information indicating whether the sub-image PS may be superimposed on the image of the object identified by the object identification unit 60. The output specification determination unit 50 according to the third embodiment determines the display position of the sub-image PS as the output specification based on this permission information. Below, the processing of the display device 10b according to the third embodiment will be described in more detail.

Figure 16 is a flowchart explaining the processing contents of the display device according to the third embodiment. As shown in Figure 16, the display device 10b according to the third embodiment performs the same processing from step S10 to step S34 as in the first embodiment, and therefore the explanation is omitted. On the other hand, the display device 10b according to the third embodiment uses the object identification unit 60 to identify the object in the main image PM based on the environmental information (step S50).

Specifically, in step S50, the object identification unit 60 acquires object information, which is information that identifies an object in the main image PM, based on the environmental information. The object information may be any information that can identify the object from other objects, and may be, for example, the name of the object, the address of the object, or location information. The object identification unit 60 acquires the object information based on the position information of the display device 10b (user U) acquired by the GNSS receiver 20C and the attitude information of the display device 10b (user U) acquired by the gyro sensor 20E. More specifically, the object identification unit 60 calculates the position information of the visual recognition area, which is the place where the user U is visualizing, from the position information of the display device 10b (user U) and the attitude information of the display device 10b (user U). In this case, the object identification unit 60 acquires the position information of the visual recognition area, which is the place where the user U is visualizing, based on, for example, the width of the user U's visual field, assuming that the range visually recognized by the user U is a visual recognition area having a predetermined range. The width of the user U's visual field may be set in advance or calculated by any method. The object identification unit 60 then identifies real objects, such as structures and natural objects, in the visual recognition area as objects based on the map data 30B, and acquires object information for the objects. That is, since the visual recognition area indicates the visual field range of the user U and the range of the main image PM, real objects contained in the visual recognition area are regarded as objects captured as the main image PM. If there are multiple objects in the visual recognition area, the object identification unit 60 acquires object information for each of these objects.

The method by which the object identification unit 60 identifies the object, i.e., the method by which the object information is acquired, is not limited to the above and may be any method.

When the display device 10b according to the third embodiment identifies an object, the permission information acquisition unit 62 acquires permission information about the identified object (step S52). That is, the permission information acquisition unit 62 acquires permission information about the object identified as being within the main image PM, indicating whether or not it is permitted to display the sub-image PS superimposed on the object.

Whether or not the sub-image PS may be superimposed on the object is determined in advance by the owner of the object, and is recorded as permission information. The permission information acquisition unit 62 acquires the permission information by transmitting the object information to an external device (server) in which the permission information is recorded, for example, via the communication unit 28. The external device acquires the permission information assigned to the object specified in the object information and transmits it to the display device 10b. The permission information acquisition unit 62 acquires the permission information assigned to the object from the external device. The permission information acquisition unit 62 acquires this permission information for each object. Note that the method of acquiring the permission information is not limited to this. For example, information in which the object information and the permission information are associated is stored in the storage unit 30 of the display device 10b, and the permission information acquisition unit 62 may read this information and acquire the permission information associated with the acquired object information.

Then, the display device 10b determines the output specifications based on the permission information in addition to the standard output specifications and the output specification correction degree by the output specification determination unit 50 (step S36a). That is, in the third embodiment, the output specification determination unit 50 determines the output specifications based on the standard output specifications set from the environmental information, the output specification correction degree set from the biometric information, and the permission information.

More specifically, the output specification determination unit 50 determines the display position of the sub-image PS as the output specification based on the permission information. Based on the permission information, the output specification determination unit 50 determines whether or not the sub-image PS may be displayed at a position superimposed on the object. For example, when the permission information indicates that the sub-image PS should not be displayed superimposed on the object, the output specification determination unit 50 determines that the sub-image PS should not be superimposed on the object, and determines a position other than the position overlapping with the object as the display position of the sub-image PS. In other words, when the permission information indicates that the sub-image PS should not be displayed superimposed on the object, the output specification determination unit 50 excludes the position overlapping with the object from the displayable positions where the sub-image PS can be displayed, and determines a position that does not overlap with the object as the displayable position of the sub-image PS.

On the other hand, when the permission information indicates that the sub-image PS may be displayed superimposed on an object, the output specification determination unit 50 determines that the sub-image PS may be superimposed on the object, and determines the display position of the sub-image PS from among positions that overlap with the object and positions that do not overlap with the object. In other words, when the permission information indicates that the sub-image PS may be displayed superimposed on an object, the output specification determination unit 50 determines both positions that overlap with the object and positions that do not overlap with the object as displayable positions of the sub-image PS.

Then, the output specification determination unit 50 sets the output specification based on the displayable position set based on the permission information, the standard output specification set based on the environmental information, and the output specification correction degree set based on the biometric information (step S36b). The output specification determination unit 50 sets the display position of the sub-image PS in the output specification based on the displayable position while setting the output specification from the standard output specification and the output specification correction degree in the same manner as in the first embodiment. That is, the output specification determination unit 50 sets the display position of the sub-image PS so that the sub-image PS is displayed within the displayable position. For example, when the permission information is information that the sub-image PS should not be displayed superimposed on the object, the output specification determination unit 50 sets the display position of the sub-image PS to a position that does not overlap with the object. On the other hand, when the permission information is information that the sub-image PS may be displayed superimposed on the object, the output specification determination unit 50 sets the display position of the sub-image PS to a position that overlaps with the object or a position that does not overlap with the object. If the permission information is information indicating that the sub-image PS may be displayed superimposed on the object, whether or not the display position of the sub-image PS is to be a position that overlaps with the object may be set based on, for example, a displayable position set based on the permission information or standard output specifications set based on environmental information.

Once the output specifications are determined, the output control unit 54 of the third embodiment causes the target device to output based on the output specifications in the same manner as in the first embodiment (step S38). The output control unit 54 causes the sub-image PS to be displayed at a display position based on the contents of the decision by the output specification determination unit 50 as to whether or not the sub-image PS may be displayed at a position superimposed on the target object. That is, when the permission information is information indicating that the sub-image PS should not be displayed superimposed on the target object, the output control unit 54 displays the sub-image PS at a position that does not overlap with the target object. On the other hand, when the permission information is information indicating that the sub-image PS may be displayed superimposed on the target object, the output control unit 54 displays the sub-image PS at a position that overlaps with the target object or at a position that does not overlap with the target object. FIG. 17 is a diagram showing an example of a display image according to the third embodiment. FIG. 17 shows an example of a case where the permission information is information indicating that the sub-image PS may be displayed superimposed on the target object PMA. As shown in FIG. 17, if the permission information indicates that the sub-image PS may be displayed superimposed on the object PMA, the output control unit 54 may display the sub-image PS at a position superimposed on the object PMA.

As described above, the display device 10b according to the third embodiment determines the display position of the sub-image PS based on the reference output specification set based on the environmental information, the output specification correction degree set based on the biometric information, and the permission information. However, the display device 10b is not limited to determining the display position of the sub-image PS using the reference output specification, the output specification correction degree, and the permission information. For example, the display device 10b may determine the display position of the sub-image PS in an arbitrary manner using all of the permission information, the environmental information, and the biometric information, or may determine the display position of the sub-image PS in an arbitrary manner using either the environmental information or the biometric information in addition to the permission information, or may determine the display position of the sub-image PS in an arbitrary manner using only the permission information from among the permission information, the environmental information, and the biometric information. In this way, in the third embodiment, it is not essential to use the environmental information or the biometric information as long as the display position of the sub-image PS is determined in an arbitrary manner using at least the permission information.

As described above, the display device 10b according to the third embodiment includes a display unit 26A that displays an image, an object identification unit 60, a permission information acquisition unit 62, an output specification determination unit 50, and an output control unit 54. The object identification unit 60 identifies an actual object in the main image PM that is provided through the display unit 26A and is visible to the user U. The permission information acquisition unit 62 acquires permission information indicating whether or not the sub-image PS may be displayed at a position superimposed on the object of the main image PM. The output specification determination unit 50 determines whether or not to display the sub-image PS at a position superimposed on the object of the main image PM based on the permission information. The output control unit 54 displays the sub-image PS so as to be superimposed on the main image PM based on the decision by the output specification determination unit 50 of whether or not to display the sub-image PS at a position superimposed on the object of the main image PM.

Here, the sub-image PS according to this embodiment is displayed superimposed on the main image PM in which an actual object is shown. However, it is assumed that the actual object has an owner, and the owner does not want the sub-image PS to be superimposed on the object. In response to this, the display device 10b according to this embodiment determines the display position of the sub-image PS based on permission information indicating whether the sub-image PS may be superimposed on the object. Therefore, for example, if the superimposition of the sub-image PS on the object is not permitted, the sub-image PS is not superimposed on the object, and if the superimposition of the sub-image PS on the object is permitted, the sub-image PS can be superimposed on the object. In this way, according to the display device 10b according to the third embodiment, by using permission information, the sub-image PS can be appropriately displayed, taking into account the intentions of, for example, the owner of the object.

In addition, in the third embodiment, the object identification unit 60 identifies the object from the position information of the user U and the posture information of the user. According to the display device 10b according to the third embodiment, it is possible to identify the object in the main image PS with high accuracy by using the position information of the user U and the posture information of the user.

(Another example of the sub-image)
In the example of FIG. 17, the sub-image PS is superimposed on the object PMA in the main image PM, but the image of the object PMA captured as the main image PM has the same shape as the actual shape of the object PMA. However, the sub-image PS may be displayed so that the image of the object PMA captured as the main image PM has a shape different from the actual shape of the object PMA. FIG. 18 is a diagram showing an example of a sub-image in which the shape of the object is different from the actual shape. As shown in the example of FIG. 18, the sub-image PS is an image in which a part of the object PMA, which is a building, is shifted, and is displayed at the position of the object PMA in the main image PM, so that the object PMA is visually recognized as a shape different from the actual shape, here as a shape with a part shifted. In other words, the sub-image PS imitates the shape of the object, but is an image different from the shape of the object PM, so that the object can be visually recognized as a shape different from the actual shape.

18, the display device 10b includes a display unit 26A that displays an image, an object identification unit 60, a permission information acquisition unit 62, an output specification determination unit 50, and an output control unit 54. The object identification unit 60 identifies an actual object in the main image PM that is provided through the display unit 26A and is visible to the user U. The permission information acquisition unit 62 acquires permission information indicating whether or not a sub-image PS that causes the object to be viewed as a shape different from its actual shape may be displayed at a position superimposed on the object of the main image PM. The output specification determination unit 50 determines whether or not to display the sub-image PS at a position superimposed on the object of the main image PM based on the permission information. The output control unit 54 displays the sub-image PS so as to be superimposed on the main image PM based on the decision by the output specification determination unit 50 of whether or not to display the sub-image PS at a position superimposed on the object of the main image PM. The owner of the object may not want such a sub-image PS that causes the object to be viewed as a shape different from the actual object to be displayed. For such sub-images PS, the display position can be controlled based on the permission information, so that the sub-images PS can be displayed appropriately, taking into account the intentions of the owner of the object, for example. Note that the sub-images PS that are visually recognized as having a shape different from the actual object, as exemplified in FIG. 18, can also be applied as the sub-images PS of other embodiments.

Fourth Embodiment
Next, a fourth embodiment will be described. The display device 10c according to the fourth embodiment is different from the first embodiment in that it counts the number of times the sub-image PS is superimposed on the object. In the fourth embodiment, the description of the parts of the configuration common to the first embodiment will be omitted. Note that the fourth embodiment is also applicable to the second and third embodiments.

Figure 19 is a schematic block diagram of a display device according to the fourth embodiment. As shown in Figure 19, the control unit 32c of the display device 10c according to the fourth embodiment includes an object identification unit 60 and a count information acquisition unit 64. The object identification unit 60 identifies an object, which is an actual object captured in the main image PM, based on environmental information detected by the environmental sensor 20. The count information acquisition unit 64 acquires count information, which is information on the number of times the sub-image PS has been superimposed on the object, and stores it in the memory unit 30. The count information acquisition unit 64 records the count information for each object.

Figure 20 is a flowchart explaining the processing contents of the display device according to the fourth embodiment. As shown in Figure 20, the display device 10c according to the fourth embodiment causes the target device to output based on the output specifications, as shown in step S38. That is, the description up to step S38 is omitted in Figure 20, and the processing from step S10 to step S38 of the first embodiment (see Figure 4) is also performed in the fourth embodiment to display the sub-image PS so as to be superimposed on the main image PM.

Next, the display device 10c uses the object identification unit 60 to identify the object on which the sub-image PS is superimposed (step S102). The object identification unit 60 extracts the object appearing in the main image PM using a method similar to that of the third embodiment. Then, the object identification unit 60 identifies, from among the objects appearing in the main image PM, the object on which the sub-image PS is superimposed.

Next, when the display device 10c identifies the object on which the sub-image PS is superimposed, the number of times the sub-image PS is superimposed is updated for each object by the number of times information acquisition unit 64 (step S104), and the number of times the sub-image PS is superimposed is recorded in the storage unit 30 for each object (step S106). The number of times information acquisition unit 64 counts the number of times the sub-image PS is superimposed for each object, and stores the count number in the storage unit 30 as number of times information. That is, the number of times information acquisition unit 64 increments the number of times the sub-image PS is superimposed by one each time the sub-image PS is superimposed, and stores it in the storage unit 30 as number of times information. The number of times information acquisition unit 64 associates the object information with the number of times information, i.e., associates the number of times the sub-image PS is superimposed with the object, and stores it in the storage unit 30.

As described above, the display device 10c according to the fourth embodiment includes a display unit 26A that displays an image, an output control unit 54, an object identification unit 60, and a number information acquisition unit 64. The output control unit 54 causes the display unit 26A to display the sub-image PS so as to be superimposed on an actual object included in the main image PM that is visible to the user U and provided through the display unit 26A. The object identification unit 60 identifies the object to be superimposed on the sub-image PS. The number information acquisition unit 64 acquires number information, which is information on the number of times the sub-image PS has been superimposed and displayed on the identified object, and stores it in the storage unit 30. The display device 10c according to this embodiment calculates and records the number of times the sub-image PS has been superimposed on the object. For example, if the sub-image PS is an advertisement, an advertising fee may be set according to the number of times it is displayed, or an advertising fee may be paid to the owner of the object on which the sub-image PS is superimposed. For example, in such a case, the display device 10c according to this embodiment can appropriately manage advertising fees, etc., by counting the number of times the sub-image PS is superimposed for each object. In this way, according to the display device 10c according to this embodiment, it can be said that the sub-image PS can be appropriately displayed by recording the number of times the sub-image PS is superimposed.

The display device 10 can communicate with a management device 12 that manages the number of times information, and may output the number of times information to the management device 12. FIG. 21 is a schematic block diagram of a display system according to a fourth embodiment. As shown in FIG. 21, a display management system 100 according to the fourth embodiment has a plurality of display devices 10 and a management device 12. The management device 12 is configured to be able to communicate with a plurality of display devices 10, and obtains number of times information, which is information on the number of times a sub-image PS is superimposed on an object, from each display device 10, and records the total number of times for each object.

In this embodiment, the management device 12 is a computer (server) and includes an input unit 12A, an output unit 12B, a memory unit 12C, a communication unit 12D, and a control unit 12E. The input unit 12A is a device that accepts operations by a user of the management device 12, and may be, for example, a touch panel, a keyboard, or a mouse. The output unit 12B is a device that outputs information, and may be, for example, a display that displays images. The memory unit 12C is a memory that stores various information such as the calculation contents and programs of the control unit 12E, and may include, for example, at least one of a RAM, a main memory device such as a ROM, and an external memory device such as a HDD. The communication unit 12D is a module that communicates with an external device, and may include, for example, an antenna. In this embodiment, the communication method by the communication unit 28 is wireless communication, but the communication method may be arbitrary.

The control unit 12E is a calculation device, i.e., a CPU. The control unit 12E performs the processes described below by reading and executing a program (software) from the storage unit 30, and these processes may be performed by a single CPU, or by multiple CPUs that are provided and perform the processes. In addition, at least a portion of the processes described below performed by the control unit 12E may be realized by hardware.

The control unit 12E acquires number information, which is information on the number of times the sub-image PS has been superimposed on the object, from each display device 10 via the communication unit 12D. The control unit 12E calculates a total number of superimpositions, which is the total number of times the sub-image PS has been superimposed on the same object, based on the number of superimpositions information acquired from each display device 10. That is, the control unit 12E adds up the number of times the sub-image PS has been superimposed on the same object for each display device 10 and calculates it as the total number of superimpositions. The control unit 12E calculates the total number of superimpositions for each object and stores it in the memory unit 12C as total number of superimpositions information.

The control unit 12E may output the calculated total number of overlaps to an external device. For example, the control unit 12E may transmit the total number of overlaps to a computer managed by the owner of the object, or to a computer managed by the advertiser of the sub-image PS. By transmitting the total number of overlaps in this manner, advertising fees can be appropriately managed.

As described above, the display management system 100 according to the fourth embodiment includes a display device 10 and a management device 12. The management device 12 acquires count information from the multiple display devices 10, adds up the number of times the sub-image PS is superimposed on the same object and displayed by the multiple display devices 10, and records this as the total count information for that object. According to the display management system 100 according to the fourth embodiment, the management device 12 centrally manages the count information of the multiple display devices 10c, thereby making it possible to appropriately manage the display of the sub-image PS.

Fifth Embodiment
Next, a fifth embodiment will be described. The display device 10d according to the fifth embodiment is different from the first embodiment in that it selects a target device and determines the output contents (contents) of the sub-image PS based on age information indicating the age of the user U. In the fifth embodiment, the description of the parts of the configuration common to the first embodiment will be omitted. Note that the fifth embodiment is also applicable to the second, third, and fourth embodiments.

FIG. 22 is a schematic block diagram of a display device according to the fifth embodiment. As shown in FIG. 22, the control unit 32d of the display device 10d according to the fifth embodiment includes an age information acquisition unit 66, a physical information acquisition unit 68, and an output content determination unit 70.

Figure 23 is a flowchart explaining the processing contents of the display device according to the fifth embodiment. As shown in Figure 23, the display device 10d according to the fifth embodiment performs the same processing from step S10 to step S34 as in the first embodiment, and therefore the explanation is omitted. Meanwhile, the display device 10d acquires the age information and physical information of the user U by the age information acquisition unit 66 and the physical information acquisition unit 68 (step S60).

The age information acquisition unit 66 acquires age information indicating the age of the user U. The age information acquisition unit 66 may acquire the age information by any method. For example, the age information may be set in advance by input by the user U and stored in the storage unit 30, and the age information acquisition unit 66 may read the age information from the storage unit 30. Also, for example, the age information acquisition unit 66 may acquire the age information by estimating the age from biometric information.

The physical information acquisition unit 68 acquires physical information, which is information about the body of the user U. The physical information is information indicating the health condition of the user U, and is different from the biological information acquired by the biosensor 22, and is different from the information about the autonomic nerves. More specifically, the physical information is information about the performance of the five senses of the user U, such as information indicating eyesight and hearing. The physical information acquisition unit 68 may acquire the physical information by any method. For example, the physical information may be set in advance by input by the user U and stored in the storage unit 30, and the physical information acquisition unit 68 may read out the age information from the storage unit 30. Also, for example, the display device 10 may be provided with a body sensor that detects the physical information of the user U, and the physical information acquisition unit 68 may acquire the physical information detected by the body sensor.

Next, the display device 10d acquires restriction necessity information for restricting the control device based on the age information and physical information through the user state identification unit 46 (step S62). The user state identification unit 46 acquires age restriction necessity information as restriction necessity information based on the age information, and acquires physical restriction necessity information as restriction necessity information based on the physical information.

FIG. 24 is a table for explaining an example of age restriction necessity information. The age restriction necessity information is information indicating whether or not an output restriction is required for the output unit 26, and is information indicating whether or not the output unit 26 may be selected as a target device. In other words, the age restriction necessity information is information for selecting a target device from among the output units 26. The age restriction necessity information is set for each output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C, and the user state identification unit 46 can be said to acquire the age restriction necessity information for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C based on the age information. More specifically, the user state identification unit 46 acquires the age restriction necessity information based on age relationship information indicating the relationship between the age information and the age restriction necessity information. The age relationship information is information (table) in which the age information and the age restriction necessity information are stored in association with each other, and is stored, for example, in the specification setting database 30C. In the age-related information, age restriction necessity information is set for each type of output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C, for each predetermined age category. The user state identification unit 46 reads out the age-related information and selects the age restriction necessity information associated with the age information of the user U from the age-related information. In the example of FIG. 24, for age categories of 19 years or older, information is assigned as the age restriction necessity information that permits the selection of each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C as target devices. In addition, for age categories of 13 years or older and 18 years or younger, information is assigned as the age restriction necessity information that prohibits the selection of the display unit 26A as a target device and permits the selection of the audio output unit 26B and the tactile stimulation output unit 26C as target devices. Furthermore, for age categories of 12 years or younger, information is assigned as age restriction necessity information that does not permit the display unit 26A and the audio output unit 26B to be selected as target devices, and permits the tactile stimulation output unit 26C to be selected as target devices. However, FIG. 24 is only an example, and the relationship between the age category in the age-related information and the age restriction necessity information may be set as appropriate.

FIG. 25 is a table for explaining an example of the physical restriction necessity information. The physical restriction necessity information is information indicating whether or not the output of the output unit 26 is necessary, and is information indicating whether or not the output unit 26 may be selected as a target device. In other words, the physical restriction necessity information is information for selecting a target device from among the output units 26. The physical restriction necessity information is set for each output unit 26, that is, for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C, and the user state identification unit 46 can be said to acquire the physical restriction necessity information for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C based on the physical information. More specifically, the user state identification unit 46 acquires the physical restriction necessity information based on the physical relationship information indicating the relationship between the physical information and the physical restriction necessity information. The physical relationship information is information (table) in which the physical information and the physical restriction necessity information are stored in association with each other, and is stored, for example, in the specification setting database 30C. In the physical relationship information, physical restriction necessity information is set for each type of output unit 26, that is, for each physical information (physical state) for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C. The user state identification unit 46 reads out the physical relationship information and selects the physical restriction necessity information associated with the physical information of the user U from the physical relationship information. In the example of FIG. 25, for the physical information that the eyesight is weaker than a predetermined threshold, information is assigned as the physical restriction necessity information that the display unit 26A is not permitted to be selected as the target device and that the audio output unit 26B and the tactile stimulation output unit 26C are permitted to be selected as the target device. That is, in this case, whether or not the output unit 26 that stimulates the output for the five senses is set as the target device according to the physical information indicating the state of the five senses of the user U. For example, if the sense (for example, vision) of the user U is weaker than a predetermined threshold, it can be said that it is determined that the output unit 26 (here, the display unit 26A) that stimulates the output for that sense (here, vision) is excluded from the target devices. However, FIG. 25 is just an example, and the relationship between the physical information in the physical-related information and the physical restriction necessity information may be set as appropriate.

23, the display device 10d determines the output specifications based on the reference output specifications and the output specification correction degree by the output specification determination unit 50, while determining the target device based on the restriction necessity information by the output selection unit 48 (step S36d). The output specification determination unit 50 determines the output specifications in the same manner as in the first embodiment. On the other hand, in the fifth embodiment, the output selection unit 48 selects the target device based on the age restriction necessity information and the physical restriction necessity information. More specifically, even if the output unit 26 was selected as the target device based on the environmental score in step S26, the output selection unit 48 excludes it from the target devices if the age restriction necessity information or the physical restriction necessity information prohibits use. In addition, the output selection unit 48 sets the target device based on the output restriction necessity information set based on the bioinformation in step S32. Therefore, it can be said that the output selection unit 48 sets the target device based on the age information, the physical information, the bioinformation, and the environmental information.

In this way, the output selection unit 48 sets the target device based on the age restriction necessity information based on the age information, the physical restriction necessity information based on the physical information, the output restriction necessity information based on the bio-information, and the user state based on the environmental information. However, the method of setting the target device is not limited to this and may be arbitrary. The output selection unit 48 may set the target device in an arbitrary manner based on at least one of the age information, the physical information, the bio-information, and the environmental information. For example, the output selection unit 48 may set the target device in an arbitrary manner based on the age information, may set the target device in an arbitrary manner based on the age information and the physical information, may set the target device in an arbitrary manner based on the age information, the physical information, and the bio-information, may set the target device in an arbitrary manner based on the age information, the physical information, and the bio-information, may set the target device in an arbitrary manner based on the age information, the physical information, and the environmental information, or may set the target device in an arbitrary manner based on the age information, the physical information, the bio-information, and the environmental information.

23, the display device 10d uses the output content determination unit 70 to determine the output content (display content) of the sub-image PS by the output unit 26 based on the age information (step S37). The output content (display content) of the sub-image PS refers to the content of the sub-image PS, i.e., the content. Note that step S37 for determining the output content is not limited to being executed after step S36, and the order of execution is arbitrary.

Here, the sub-image PS is set with a content rating, which is information indicating whether the content of the sub-image PS may be provided. This content rating is set for each predetermined age category. In other words, the content rating can be said to be information that specifies the target age for which the content can be provided. An example of the content rating is the MPAA (Motion Picture Association of America) rating, but is not limited to this. In the fifth embodiment, the sub-image acquisition unit 52 acquires the content rating of the sub-image PS together with the image data of the sub-image PS. Then, the output content determination unit 70 determines whether or not the sub-image PS can be displayed based on the content rating of the sub-image PS and the age information of the user U. If the content rating of the sub-image PS indicates that the sub-image PS can be provided for the age of the user U, the output content determination unit 70 determines that the sub-image PS can be displayed and determines the content of the sub-image PS as the output content. On the other hand, if the content rating of the sub-image PS does not indicate that the sub-image PS can be provided for the age of the user U, the output content determination unit 70 determines that the display of the sub-image PS is not permitted, and does not set the content of the sub-image PS as the output content. For example, in this case, the output content determination unit 70 obtains the content rating of another sub-image PS obtained by the sub-image acquisition unit 52, and similarly determines whether or not the sub-image PS can be displayed.

Figure 26 is a table showing an example of content ratings. In the example of Figure 26, content rating CA3, for example, specifies that the target age for which the content can be provided is 19 years or older, content rating CA2, for example, specifies that the target age for which the content can be provided is 13 years or older, and content rating CA1, for example, specifies that the target age for which the content can be provided is unlimited, that is, that the content can be provided to all ages. For example, when the age information indicates 15 years old, the output content determination unit 70 does not permit the provision of a sub-image PS of content rating CA3, but permits the provision of sub-images PS of content ratings CA2 and CA1. Note that in Figure 26, the output content determination unit 70 permits or prohibits all output units 26, i.e., the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C, all at once. However, the output content determination unit 70 is not limited to managing all output units 26 collectively, and may determine the output of the content of the sub-image PS for each output unit 26, i.e., for each of the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C, based on the content rating and age information. By determining whether or not to output content for each output unit 26 in this manner, it is possible to flexibly respond to various situations, such as when an image is inappropriate but audio or tactile stimulation is appropriate.

In this way, the output content determination unit 70 determines the output content of the sub-image PS based on the age information and the content rating, but the method of determining the output content of the sub-image PS is not limited to the above and is arbitrary. The output content determination unit 70 may determine the output content of the sub-image PS by any method based on the age information.

Returning to FIG. 23, once the display device 10d has determined the output content, it causes the target device to output the determined output content based on the output specifications (step S38). That is, the display device 10d causes the sub-image PS of the determined output content (content) to be superimposed on the main image PM and displayed in accordance with the determined output specifications.

As described above, the display device 10d according to the fifth embodiment includes the display unit 26A for displaying an image, the audio output unit 26B for outputting audio, the tactile stimulation output unit 26C for outputting tactile stimulation to the user U, the age information acquisition unit 66, the output selection unit 48, and the output control unit 54. The age information acquisition unit 66 acquires the age information of the user U. The output selection unit 48 selects the target device to be used from among the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C based on the age information of the user U. The output control unit 54 controls the selected target device. Here, for humans, which sense is preferably stimulated may differ depending on age, due to reasons such as changes in the performance of the five senses depending on age. In contrast, the display device 10d according to the present embodiment selects the target device to be used from among the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C depending on the age of the user U. Therefore, the display device 10d can stimulate the senses of the user U appropriately according to the user's age, and can also provide the user U with an appropriate sub-image SP, for example.

The display device 10d further includes a physical information acquisition unit 68 that acquires physical information of the user U, and the output selection unit 48 selects the target device to be used from among the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C based on the physical information of the user U. The display device 10d according to this embodiment selects the target device to be used from among the display unit 26A, the audio output unit 26B, and the tactile stimulation output unit 26C according to the physical information of the user U, so that it is possible to appropriately stimulate the senses of the user U according to the physical condition of the user U, and for example, it is also possible to appropriately provide the sub-image SP to the user U.

Sixth Embodiment
Next, a sixth embodiment will be described. The display device 10e according to the sixth embodiment is different from the first embodiment in that the display device 10e determines the output contents (contents) of the sub-image PS based on age information indicating the age of the user U and position information of the user U. In the sixth embodiment, the description of the parts of the configuration common to the first embodiment will be omitted. Note that the sixth embodiment can also be applied to the second, third, fourth and fifth embodiments.

FIG. 27 is a schematic block diagram of a display device according to the sixth embodiment. As shown in FIG. 27, the control unit 32e of the display device 10e according to the sixth embodiment includes an age information acquisition unit 66 and an output content determination unit 70.

Figure 28 is a flowchart explaining the processing contents of the display device according to the sixth embodiment. As shown in Figure 28, the display device 10e according to the sixth embodiment performs the same processing from step S10 to step S36 as in the first embodiment, and therefore the explanation is omitted. Meanwhile, the display device 10e acquires the age information of the user U by the age information acquisition unit 66 (step S64).

The age information acquisition unit 66 acquires age information indicating the age of the user U. The age information acquisition unit 66 may acquire the age information by any method. For example, the age information may be set in advance by input by the user U and stored in the storage unit 30, and the age information acquisition unit 66 may read the age information from the storage unit 30. Also, for example, the age information acquisition unit 66 may acquire the age information by estimating the age from biometric information.

After acquiring the age information, the display device 10e uses the output content determination unit 70 to determine the output content (display content) of the sub-image PS by the output unit 26 based on the age information and location information of the user U (step S37e). The age information of the user U is acquired by the age information acquisition unit 66 as described above, and the location information of the user U is acquired by the environmental information acquisition unit 40 via the GNSS receiver 20C. The output content (display content) of the sub-image PS refers to the content of the sub-image PS, i.e., the content. Note that step S37e of determining the output content is not limited to being executed after step S36, and the order of execution is arbitrary.

Here, as in the fifth embodiment, a content rating indicating whether or not content should be provided according to age is set for the sub-image PS, and the sub-image acquisition unit 52 acquires the content rating of the sub-image PS along with the image data of the sub-image PS. In addition, in the sixth embodiment, a regional rating indicating whether or not content should be provided according to the location (earth coordinates) is set. The output content determination unit 70 sets a final rating indicating whether or not the final content should be provided to the user U based on the content rating and regional rating, and determines the output content of the sub-image PS based on the final rating and the age information of the user U. This will be explained in more detail below.

The output content determination unit 70 acquires regional rating information indicating the relationship between regional ratings and positions (earth coordinates). The regional rating information is set for each position. For example, for a predetermined range such as a 50 m radius from the position of an elementary school, the regional rating is set so as to impose strict restrictions on the content that can be provided, i.e., to reduce the amount of content that can be provided. For areas within a downtown area, the regional rating is set so as to ease restrictions on the content that can be provided, i.e., to prevent the amount of content that can be provided from decreasing. For other areas, the regional rating is set so as to impose intermediate restrictions on the content that can be provided. The output content determination unit 70 may acquire regional rating information by any method, but for example, the regional rating information may be included in the map data 30B, and the output content determination unit 70 may acquire regional rating information by reading the map data 30B.

Then, the output content determination unit 70 sets the regional rating to be applied based on the acquired location information of the user U and the regional rating information. The output content determination unit 70 sets the regional rating associated with the acquired location information of the user U in the regional rating information as the regional rating to be applied.

Figure 29 is a table showing an example of a final rating. The output content determination unit 70 sets the final rating of the sub-image PS based on the content rating of the acquired sub-image PS and the regional rating set from the location information of the user U. The output content determination unit 70 sets the content rating of the sub-image PS and the regional rating that places the strictest restrictions on the content that can be provided as the final rating. The example in Figure 29 illustrates the final ratings for each combination of content ratings CA1, CA2, CA3 and regional ratings CB1, CB2, CB3. In the example of Fig. 29, the content rating CA3 has strict restrictions on the content that can be provided, for example, the target age for which the content can be provided is 19 years or older, the content rating CA2 has looser restrictions than the content rating CA3, for example, the target age for which the content can be provided is 13 years or older, and the content rating CA1 has looser restrictions than the content rating CA2, for example, the target age for which the content can be provided is unlimited, i.e., the content can be provided to all ages. Also, in the example of Fig. 29, the area rating CB3 has strict restrictions on the content that can be provided, for example, the target age for which the content can be provided is 19 years or older, the area rating CB2 has looser restrictions than the area rating CB3, for example, the target age for which the content can be provided is 13 years or older, and the area rating CB1 has looser restrictions than the area rating CB2, for example, the target age for which the content can be provided is unlimited, i.e., the content can be provided to all ages. Furthermore, the final rating CC3 has strict restrictions on the content that can be provided, for example limiting the target age for which the content can be provided to those 19 years and older, the final rating CC2 has looser restrictions than the final rating CC3, for example limiting the target age for which the content can be provided to those 13 years and older, and the final rating CC1 has looser restrictions than the final rating CC2, for example limiting the target age for which the content can be provided to those with no restrictions, i.e., the content can be provided to all ages.

For ease of explanation, the combination of content rating CA1 and regional rating CB1 will be referred to as combination CA1-CB1, and the same applies to the others. As described above, the content rating and regional rating that places the strictest restrictions on the content that can be provided is set as the final rating. Therefore, in the example of FIG. 29, the final rating for combination CA-CB1 is CC1, the final rating for combinations CA1-CB2, CA2-CB1, and CA2-CB2 is CC2, and the final rating for combinations CA1-CB3, CA2-CB3, CA3-CB1, CA3-CB2, and CA3-CB3 is CC3.

After setting the final rating, the output content determination unit 70 determines the output content of the sub-image PS based on the final rating and the age information of the user U. The output content determination unit 70 determines whether or not to display the sub-image PS based on the final rating and the age information of the user U. If the final rating indicates that the content can be provided for the age of the user U, the output content determination unit 70 determines that the sub-image PS can be displayed and determines the content of the sub-image PS as the output content. On the other hand, if the final rating indicates that the content cannot be provided for the age of the user U, the output content determination unit 70 determines that the display of the sub-image PS is not permitted and does not set the content of the sub-image PS as the output content. For example, in this case, the output content determination unit 70 obtains a final rating for another sub-image PS obtained by the sub-image acquisition unit 52 and similarly determines whether or not to display the sub-image PS.

Figure 30 is a table illustrating an example of determining the output contents based on the final rating. As shown in Figure 30, for example, if the final rating is CC1, the display of the sub-image PS is permitted regardless of whether the user U is 10, 15, or 20 years old; if the final rating is CC2, the display of the sub-image PS is not permitted when the user U is 10 years old; and if the final rating is CC3, the display of the sub-image PS is not permitted when the user U is 10 or 15 years old.

Returning to FIG. 28, once the display device 10e has determined the output content, it causes the target device to output the determined output content based on the output specifications (step S38). That is, the display device 10d causes the sub-image PS of the determined output content (content) to be superimposed on the main image PM and displayed in accordance with the determined output specifications.

In this way, the output content determination unit 70 determines the output content of the sub-image PS based on the final rating set from the content rating and regional rating, and the age information, but the method of determining the output content of the sub-image PS is not limited to the above and is arbitrary. The output content determination unit 70 may determine the output content of the sub-image PS in an arbitrary manner based on the age information and age information of the user U. Furthermore, the output content determination unit 70 is not limited to using both the age information and age information of the user U, and may determine the output content of the sub-image PS in an arbitrary manner based on the age information of the user U.

As described above, the display device 10e according to the sixth embodiment includes the display unit 26A that displays an image, the age information acquisition unit 66, the output content determination unit 70, and the output control unit 54. The age information acquisition unit 66 acquires the age information of the user U. The output content determination unit 70 determines the display content (output content) of the sub-image PS to be displayed on the display unit 26A based on the age information of the user U. The output control unit 54 causes the display unit 26A to display the sub-image PS of the determined display content so as to be superimposed on the main image PM that is provided through the display unit 26A and is visible to the user U. Here, the content of the sub-image SP may include inappropriate content depending on the age of the user U. In contrast, the display device 10e according to this embodiment determines the content of the sub-image SP according to the age of the user U, and therefore can provide the sub-image SP appropriately according to the age.

The display device 10e further includes an environmental sensor 20 that detects the location information of the user U, and the output content determination unit 70 determines the display content of the sub-image PS based on the location information of the user U as well. The content of the sub-image SP may be inappropriate to provide depending on the area, for example, around an elementary school. In contrast, the display device 10e according to this embodiment determines the content of the sub-image SP according to the location information of the user U in addition to the age of the user U, and can therefore provide the sub-image SP appropriately according to the age and area of the user U.

The output content determination unit 70 also acquires regional rating information indicating the relationship between preset global coordinates and the display content that is permitted to be displayed (i.e., regional rating), and determines the display content of the sub-image PS based on the regional rating information and the location information of the user U. The display device 10e according to this embodiment sets a regional rating that restricts the provision of the sub-image PS, which is applied to the current location of the user U, based on the regional rating information and the location information of the user U, and determines the display content of the sub-image PS based on the regional rating. Therefore, the display device 10e according to this embodiment can provide an appropriate sub-image SP according to the age and region of the user U.

Although the embodiments and modifications of the present invention have been described above, the embodiments are not limited to the contents of these embodiments and modifications. The above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be combined as appropriate, and the configurations of each embodiment and modification can also be combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-mentioned embodiments and modifications.

REFERENCE SIGNS LIST 10 Display device 20 Environmental sensor 22 Biometric sensor 26 Output section 26A Display section 26B Audio output section 26C Tactile stimulation output section 40 Environmental information acquisition section 42 Biometric information acquisition section 44 Environmental determination section 46 User state determination section 48 Output selection section 50 Output specification determination section 52 Sub-image acquisition section 54 Output control section PM Main image PS Sub-image

Claims (4)

A display unit for displaying an image;
a sub-image acquiring unit that acquires image data of a sub-image including advertising information to be displayed on the display unit and advertising fee information to be paid for displaying the advertising information;
an output specification determination unit that determines a display mode indicating how the sub-image is to be displayed as a display specification of the sub-image based on the advertising fee information;
an output control unit that causes the display unit to display the sub-image on the basis of the display specifications and that is superimposed on a main image viewed through the display unit;
A display device comprising:
An environmental information acquisition unit that acquires environmental information around the display device;
an environment identification unit that identifies an environment pattern indicating an environment state based on the environment information;
and
the output specification determination unit determines a standard output specification indicating a standard output based on the environment pattern specified by the environment determination unit, determines an advertisement fee correction degree for correcting the standard output specification based on the advertisement fee information, and determines at least one of a display time, a display position, and a size of the sub-image by correcting the standard output specification with the advertisement fee correction degree;
Display device. A biosensor for detecting biometric information of a user ;
a user condition identification unit that identifies a condition of the user based on the biological information;
and
The display device described in claim 1, wherein the output specification determination unit determines an output specification correction degree, which is a value for correcting the standard output specification, based on the user state identified by the user state identification unit, and determines at least one of the display time, display position, and size of the sub-image by correcting the standard output specification with the output specification correction degree. acquiring image data of a sub-image including advertising information to be displayed on a display unit, and advertising fee information to be paid for displaying the advertising information;
determining a display mode indicating how the sub-image is to be displayed as a display specification of the sub-image based on the advertising fee information;
a step of displaying the sub-image on the display unit based on the display specification and superimposing the sub-image on the main image viewed through the display unit;
detecting environmental information around the display unit;
Identifying an environmental pattern indicative of an environmental state based on the environmental information;
determining a reference output specification indicating a reference output based on the environmental pattern;
determining an advertising fee correction rate for correcting the reference output specification based on the advertising fee information;
determining at least one of a display time, a display position, and a size of the sub-image by correcting the standard output specification with the advertisement fee correction rate;
Including,
Display method. acquiring image data of a sub-image including advertising information to be displayed on a display unit, and advertising fee information to be paid for displaying the advertising information;
determining a display mode indicating how the sub-image is to be displayed as a display specification of the sub-image based on the advertising fee information;
a step of displaying the sub-image on the display unit based on the display specification and superimposing the sub-image on the main image viewed through the display unit;
detecting environmental information around the display unit;
Identifying an environmental pattern indicative of an environmental state based on the environmental information;
determining a reference output specification indicating a reference output based on the environmental pattern;
determining an advertising fee correction rate for correcting the reference output specification based on the advertising fee information;
determining at least one of a display time, a display position, and a size of the sub-image by correcting the standard output specification with the advertisement fee correction rate;
A display method including the steps of:
program.

Images