JP6917427B2 - Display control device, display device, display control method, program - Google Patents

Description

The present invention relates to a display control device, a display device, a display control method, and a program.

Conventionally, there is a technique called VR (Virtual Reality) technique for changing the area of an image to be displayed on the HMD according to the inclination of the head-mounted display (HMD) worn by the user. In Patent Document 1, the tilt angle of the user's head is determined by the tilt sensor and the gyro sensor in the HMD, and the region of the translucent VR image is superimposed and displayed on the real space based on the tilt angle. Is described. According to this, it is possible to operate while checking the operation member and the like in the real space.

JP-A-2017-191474

However, in such a VR technique, the area of the image to be displayed is only changed according to the inclination (posture) of the HMD. Therefore, for example, when a user is viewing an image of an Antarctic animal in a cool area in a room with a high temperature, he / she may not be able to immerse (empathize) with the image.

Therefore, it is an object of the present invention to provide a technique for displaying an image that allows a user to be more immersive.

The first aspect of the present invention is
An acquisition method for acquiring the magnitude of external stimuli to the user,
A control means that controls the display image of the display means according to the magnitude of the stimulus, and
Have,
It is a display control device characterized by this.

By controlling the display image according to the magnitude of the stimulus from the outside, the display image can be displayed according to the stimulus (sense) felt by the user, so that the user is more immersive in the display image. be able to. The external stimulus is a physical stimulus that causes an energy change due to the stimulus.

Further, the external stimulus may include water pressure on the user, wind pressure on the user, or the temperature of an object in contact with the user.

The control means may determine whether or not the user is in water from the magnitude of the stimulus, and control the display image according to the result of the determination. In the real space, the space that the user can see differs depending on whether the user is underwater or not. Therefore, by changing the display image depending on whether or not the user is in the water, an image suitable for the space where the user is present can be displayed.

The control means may control to display an underwater image on the display means if the water pressure is larger than a predetermined value. If the water pressure is larger than a predetermined value, it can be determined that the user is located in the water. Therefore, by displaying the image in the water, an image suitable for the user's sense can be displayed.

The control means may control so that the larger the water pressure is, the deeper the water pressure in the underwater image is displayed on the display means. The higher the water pressure, the deeper the user feels to dive in the water, so it is possible to display an image that suits the user's senses.

The control means may control the display means to display an image that moves faster in a predetermined space as the wind pressure increases. The greater the wind pressure, the faster the user feels to move, so it is possible to display an image that suits the user's senses.

The display means may superimpose the display image on the real space and display it. According to this, since the user can also see the real space, the displayed image can be safely viewed.

Further having a position acquisition means for acquiring the position of the user, the control means may control the display image according to the magnitude of the stimulus and the position. By controlling the display image according to the position, for example, the image that the child likes can be displayed to the user located in the children's pool, and the drifting object flows to the user located in the flowing pool. Images can be displayed. That is, it is possible to display an image that matches the position of the user.

The display means for displaying the display image may be further provided, and the acquisition means may be a sensor for detecting the stimulus.

Further, it is a display device capable of communicating with the above-mentioned display control device.
A sensor that detects external stimuli to the user,
A transmission means for transmitting the magnitude of the stimulus to the display control device, and
An acquisition means for acquiring the display image from the display control device, and
A display means for displaying the display image and
Have,
The display device may be characterized in that.

The display device may further include mounting means for fixing the relative position of the user's head and the display means. According to this, the user can appreciate the displayed image even in an environment where the display device cannot be used, for example, when swimming in water.

The present invention is a display control method including at least a part of the above-mentioned functions and processes, a program for causing a computer to execute each step of the display control method, or a storage medium in which the program is stored non-temporarily. It can also be regarded as such.

According to the present invention, it is possible to provide a technique for displaying an image that allows a user to be more immersive.

It is a block diagram of the display system which concerns on Embodiment 1. FIG. It is a figure explaining the HMD which concerns on Embodiment 1. FIG. It is a flowchart which shows the display process which concerns on Embodiment 1. It is a figure explaining the water pressure which concerns on Embodiment 1. It is a figure explaining the display image data which concerns on Embodiment 1. FIG. It is a figure explaining the display image data which concerns on modification 1. FIG. It is a figure explaining another modification.

Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings. The embodiments are examples of embodiments for carrying out the invention, and the present invention is not limited to these descriptions.

<Embodiment 1>
The display system 1 according to the first embodiment will be described below. The display system 1 acquires an external stimulus to the user and controls the image (display image) to be displayed according to the magnitude of the stimulus to improve the immersive feeling of the user's display image. The external stimulus is, for example, water pressure, wind pressure, air pressure, heat (temperature), light, sound, or the like, and is a stimulus (physical stimulus) that causes an energy change. In the present embodiment, the external stimulus will be described as water pressure. Further, in the present embodiment, the water pressure corresponds to a value obtained by subtracting the air pressure from the pressure on the object, and is the pressure applied to the object only by water.

[Display system configuration]
First, the configuration of the display system 1 will be described with reference to the configuration diagram of the display system shown in FIG. The display system 1 has a display control device 100 and an HMD 200 (head-mounted display). The display control device 100 and the HMD 200 can communicate with each other via the network 300.

(Configuration of display control device)
The display control device 100 is an information processing device such as a PC, a smartphone, or a server. The display control device 100 includes a control unit 101, a storage unit 102, and a communication unit 103.

The control unit 101 is, for example, a CPU (Central Processing Unit) that controls each functional unit in the display control device 100. The control unit 101 controls each functional unit based on the program stored in the storage unit 102. Further, the control unit 101 determines (selects) image data (display image data) to be displayed on the HMD 200 from a plurality of image data stored by the storage unit 102. Here, the image data is data indicating an image that is a video or a still image.

The storage unit 102 stores a program for operating the control unit 101 and a plurality of image data used for displaying on the HMD 200. It is not necessary for the storage unit 102 to store a plurality of image data, and the control unit 101 may acquire the image data from, for example, an external server, if necessary, via the communication unit 103.

The communication unit 103 communicates with the HMD 200 via the network 300. More specifically, the communication unit 103 has an acquisition unit that acquires information on the magnitude of external stimuli (stimulation information) for the user from the HMD 200, and a transmission unit that transmits the display image data determined by the control unit 101 to the HMD 200. It is composed of and. The communication between the communication unit 103 and the network 300 may be wired communication or wireless communication, and the communication format may be arbitrary.

(HMD configuration)
The HMD 200 is a display device that displays a display image based on the display image data determined by the display control device 100. The HMD 200 has a control unit 201, a storage unit 202, a sensor 203, a communication unit 204, and a display unit 205.

The control unit 201 is, for example, a CPU (Central Processing Unit) that controls each functional unit in the HMD 200. The control unit 201 controls each functional unit based on the program stored in the storage unit 202.

The storage unit 202 stores a program for the control unit 201 to operate.

The sensor 203 is a detection unit that detects (acquires) an external stimulus to the user. Further, in the present embodiment, the sensor 203 is also a position acquisition unit that acquires the position of the user.

The communication unit 204 communicates with the display control device 100 via the network 300. More specifically, the communication unit 204 includes an acquisition unit that acquires display image data to be displayed on the display unit 205 from the display control device 100, and a transmission unit that transmits stimulus information to the display control device 100. .. The communication between the communication unit 204 and the network 300 may be wired communication or wireless communication, and the communication format may be arbitrary.

The display unit 205 displays a display image based on the display image data acquired from the display control device 100. The display unit 205 displays a display image on a display such as an LCD (Liquid Crystal Display) or an organic EL. The display unit 205 may be a transparent display having transparency, and when the display image is not displayed, the user sees the real space through the display unit 205 in consideration of the user's safety. You may be able to.

[Appearance of HMD]
Subsequently, the appearance of the HMD200 will be described with reference to FIGS. 2 (A) and 2 (B). The HMD 200 has a mounting portion 210, an eyepiece portion 211, and a housing portion 212.

The mounting portion 210 is a member for the user to mount the HMD 200 on the head. The mounting portion 210 has a contact portion 213 and a band 214. The contact portion 213 is formed of rubber or the like, and can be in contact with the peripheral portion of the user's eyes without a gap. The band 214 fixes the relative position of the user's head and the HMD 200 (display unit 205) by wrapping around the user's head.

The eyepiece 211 can show the display 205 to the user when the user looks into the eyepiece 211 while wearing the HMD200. That is, when the user wears the HMD 200, both eyes of the user are located at the eyepiece 211. Therefore, the user can appreciate the displayed image while the user wears the HMD 200 by the mounting portion 210.

The housing unit 212 is a housing that covers the control unit 201, the storage unit 202, the communication unit 204, and the display unit 205 described above. The housing portion 212 has waterproofness and airtightness, and in the present embodiment, the housing portion 212 is formed so that water does not enter the inside of the housing portion 212 even when the HMD 200 is located in water. The sensor 203 may be arranged inside the housing portion 212 or may be arranged outside. In the present embodiment, it is assumed that the sensor 203 is arranged outside the housing portion 212 in order to detect the water pressure.

As described above, by using the HMD200 as the display device for displaying the display image, in addition to being able to show the display image to the user without using the user's hands, it is possible to prevent water from entering the user's eyes. be able to.

FIG. 2B shows a state in which the user is wearing the HMD200. As shown in FIG. 2B, the HMD200 is located on the user's head, and the HMD200 covers both eyes of the user.

[Display processing of display image]
Next, the display processing of the display image will be described with reference to the flowchart shown in FIG. In the flowchart shown in FIG. 3, the process executed by the HMD 200 is executed by the control unit 201 based on the program stored in the storage unit 202. The process executed by the display control device 100 is executed by the control unit 101 based on the program stored in the storage unit 102. Further, it is assumed that the user is wearing the HMD200 on the head at the start of the flowchart shown in FIG. Further, each process of the flowchart shown in FIG. 3 is assumed to be repeatedly executed unless the user performs an end operation (end instruction) of the displayed image.

In step S1001, the control unit 201 of the HMD 200 controls the sensor 203 to acquire the position of the user. The sensor 203 acquires the position of the user by using, for example, GPS (Global Positioning System). The user's position may be a fine position indicated by latitude and longitude. For example, the user is located in any of the pool facilities such as a children's pool, a swimming pool, a pool where waves are generated, and a flowing pool. It may indicate whether to do it.

In step S1002, the control unit 201 controls the sensor 203 to detect (measure) the water pressure. Here, since the sensor 203 is provided in the HMD 200, the water pressure at the position of the user's head is detected. Therefore, as shown in FIG. 4A, when the HMD 200 is located in the air 400, the water pressure is 0 or undetectable. On the other hand, as shown in FIG. 4 (B), when the HMD 200 is located in the water 401, the water pressure is a value larger than 0.

In step S1003, the control unit 201 controls the communication unit 204 to transmit the user's position information and water pressure information to the display control device 100.

In step S1004, the control unit 101 of the display control device 100 controls the communication unit 103 to acquire the user's position information and water pressure information from the HMD 200.

In step S1005, the control unit 101 determines the display image data to be displayed on the HMD 200. For example, the control unit 101 selects display image data from a plurality of image data stored in the storage unit 102. Therefore, in the present embodiment, in step S1005, the control unit 101 controls the display image on the display unit 205 of the HMD 200. Here, the control unit 101 determines the display image data from the position information and the water pressure information.

First, the control unit 101 determines, for example, whether or not the user is located in the water, and if the user is located in the water, the image data for displaying the water as the display image data to be displayed on the HMD 200. To determine. For example, when the water pressure is greater than 0, it can be determined that the user's head is located in the water, so the control unit 101 selects display image data indicating an underwater image as shown in FIG. 5 (B). .. On the other hand, when the water pressure cannot be measured or is 0 or less, it can be determined that the user's head is located in the air, so that the control unit 101 can perform an image in the air (underwater) as shown in FIG. 5 (A). Select display image data indicating (an image that is not an image). In this way, by changing the display image data according to the determination result of whether or not the user is in the water, the display image matching the user's tactile sensation (sense) can be displayed on the HMD 200. Therefore, it is possible to improve the immersive feeling of the user's display image.

In the present embodiment, the control unit 101 determines whether or not the user is in the water based on whether or not the water pressure is greater than 0, but the user determines whether or not the water pressure is greater than a predetermined value (threshold value). May be determined whether or not is underwater. Further, the control unit 101 may determine display image data corresponding to a deeper position (deep place) of the underwater image as the water pressure increases. This is because the higher the water pressure, the deeper the user can be estimated to be in the water. Further, whether or not the user is in water may be determined by, for example, whether or not the temperature acquired by the sensor 203 is smaller than a predetermined value, or whether or not the amount of sound is smaller than a predetermined value.

Further, the control unit 101 determines the display image data according to the position of the user. For example, when the user is located in a children's pool in a pool facility, the control unit 101 estimates that the user is a child and determines display image data including an animated character preferred by the child. On the other hand, for example, when the user is in a pool where waves are generated in a pool facility, it is estimated that the user is seeking a thrill, and display image data including a shark that arouses fear is determined. .. By controlling in this way, for example, a display image suitable for the user's nature and emotion can be displayed on the HMD 200 according to the position of the user.

In step S1006, the control unit 101 controls the communication unit 103 to transmit the determined display image data to the HMD 200.

In step S1007, the control unit 201 controls the communication unit 204 to acquire display image data from the HMD 200.

In step S1008, the control unit 201 displays the display image on the display unit 205 based on the display image data. The display unit 205 may display, for example, so as to superimpose a display image based on the display image data on the real space, or display only the display image based on the display image data without displaying the real space. It may be displayed. By superimposing the display image on the real space and displaying it, the user can grasp the surrounding state, so that it is possible to improve the immersive feeling in the display image while ensuring safety. become.

In step S1005, when the user determines that the user is not located in the water, the control unit 101 does not have to determine the display image data to be displayed on the HMD 200. In this case, in step S1008, the HMD 200 may display an image showing the real space on the display unit 205, or the display unit 205 is a transmissive display that transmits light so that the user can see the real space. May be good.

Further, in step S1005, the control unit 101 may cut out a range corresponding to the posture of the HMD 200 from the determined display image data, and transmit the cut out area to the HMD 200 in step S1006. That is, the display image data may be so-called VR image (360 degree image) data indicating a virtual space. Then, the control unit 201 of the HMD 200 that has acquired the area may display the area as a display image on the display unit 205. The posture of the HMD 200 is detected by, for example, a posture detection unit (not shown).

Further, in the present embodiment, the sensor 203 is mounted on the user's head in the same manner as the housing portion 212, but the present invention is not limited to this. For example, the sensor 203 may be worn on the user's wrist or ankle. In this case, by changing the threshold value of the water pressure used for determining whether or not the user is located in the water in step S1005, whether or not the user is located in the water according to the water pressure detected by the sensor 203. Can be determined.

[effect]
As described above, in the present embodiment, the display control device 100 controls the display image according to the water pressure which is an external stimulus to the user. As a result, the display image that matches the stimulus felt by the user is displayed on the HMD 200, so that the user is more likely to be immersed in the display image.

<Modification example 1>
In the first embodiment, an example in which the external stimulus to the user is water pressure has been described, but in the modified example 1, a case where the stimulus is wind pressure will be described.

Hereinafter, the display processing of the display image will be described, but in the processing of the flowchart shown in FIG. 3, except for step S1002 and step S1005, it is the same as the processing in which "water pressure" in the first embodiment is read as "wind pressure". .. Therefore, in the following, step S1002 and step S1005 according to this modification will be described, and detailed description of the others will be omitted.

In step S1002, the control unit 201 controls the sensor 203 to detect the wind pressure. For example, it is assumed that the user wears the HMD200 while sliding on the waterslide. In this case, if the waterslide is installed indoors, the faster the user slides (moves), the greater the detected wind pressure. In the present embodiment, the sensor 203 is arranged outside the housing portion 212 in order to detect the wind pressure.

In step S1005, the control unit 201 determines the display image data to be displayed on the HMD 200 according to the wind pressure information and the user's position information. In this modification, for example, the control unit 201 determines display image data that moves faster in a predetermined space (VR space) as the wind pressure increases. More specifically, the control unit 201 determines the display image data that the higher the wind speed, the faster the river goes down from the viewpoint of the user in the river descent, as shown in FIG. That is, the control unit 201 controls the display image so that the faster the wind speed, the faster the movement in the predetermined virtual space. Further, the control unit 201 may determine the display image data that moves faster as the user's position change is larger, based on the user's position information.

[effect]
As described above, in this modification, the display control device 100 controls the display image according to the wind pressure which is an external stimulus to the user. For example, in an example in which a user is sliding on a waterslide in reality and the displayed image data indicates a river descent, the faster the user slides on the waterslide, the faster the user moves down the river. You can see it. As a result, the user's feeling and the displayed image are well matched, and it is possible to display an image that is easier for the user to immerse.

<Other variants>
In addition, the external stimulus to the user may be other than water pressure or wind pressure. For example, the sensor 203 detects the air temperature (the temperature of the object in contact with the user), and the control unit 101 determines the display image data according to the air temperature. Specifically, the control unit 101 determines display image data for displaying a cold region such as Antarctica if the detected temperature is close to 0 ° C. On the other hand, if the detected temperature is close to 40 degrees, the control unit 101 determines display image data for displaying a tropical area such as a desert. As a result, an image of an area close to the actual temperature is displayed on the HMD 200, so that the user can easily immerse himself in the displayed image.

Furthermore, the external stimulus to the user may be the pressure generated between a predetermined object and a person. For example, as shown in FIG. 7A, it is assumed that the user 701 wears the HMD200, rides on the surfboard 702, and simulates surfing in a pool where waves are generated. In this case, since it is a simulation, the actual position of the user 701 does not move significantly. Here, the sensor 203 detects the pressure (horizontal pressure and vertical pressure) between the sole of the foot of the user 701 and the surfboard. Then, for example, as shown in FIG. 7B, the control unit 201 determines the display image data that moves in the sea at a speed corresponding to the detected pressure. At this time, in consideration of safety, the HMD 200 may display a landscape image around the sea such as the sun and seagulls superimposed on the waves in the real space. According to this, since the image that matches the user's physical sensation in surfing is displayed on the HMD 200, the user can easily be immersed in the displayed image.

In the above-described embodiment and modification, the display system 1 determines the display image data in response to one stimulus, but the display image data may be determined in response to a plurality of stimuli. Further, although the HMD 200 has been described as a display device for displaying an image, the image does not necessarily have to be displayed on a display device worn by the user on the head. For example, the image may be displayed on a general display, a television, a smartphone, or the like.

Further, in the above-described embodiment and modification, the display system 1 is realized by having the display control device 100 and the HMD 200, but the present invention is not limited to this. For example, in the display system 1, the HMD 200 may have a display control device 100. Further, the display control device 100 may have a part of the functional part of the HMD200, and conversely, the HMD200 may have a part of the functional part of the display control device 100.

The above-described embodiment or modification can also be performed by transmitting a program that realizes one or more functions of the above-described embodiment or modification to an external computer or the like, and the processor or the like of the computer reads and executes the program. The example is feasible. Further, the above-described embodiment or modification can be realized not only by a processor but also by a circuit such as an ASIC that realizes one or more functions.

1: Display system, 100: Display control device, 101: Control unit, 103: Communication unit

Claims (15)

An acquisition method for acquiring the magnitude of external stimuli to the user,
A control means that controls the display image of the display means according to the magnitude of the stimulus, and
Have,
The magnitude of the stimulus includes water pressure and
The control means
From the magnitude of the water pressure, it is determined whether or not the user's head is located in the water.
When it is determined that the user's head is located in the water, the underwater image is controlled to be displayed on the display means.
When it is determined that the user's head is not located in the water, the display means is controlled to display the image in the air.
A display control device characterized by the fact that. The control means
If the water pressure is greater than a predetermined value, it is determined that the user's head is located in the water, and the user's head is determined to be located in the water.
If the water pressure is equal to or less than a predetermined value, it is determined that the user's head is not located in the water.
The display control device according to claim 1. The control means controls so that the larger the water pressure is, the deeper the water pressure in the underwater image is displayed on the display means.
The display control device according to claim 2. The stimulus includes sound and
The control means determines whether or not the user's head is located in the water based on the magnitude of the water pressure and whether or not the amount of the sound is smaller than a predetermined value.
The display control device according to claim 1. The stimulus includes the temperature of the object in contact with the user.
The control means determines whether or not the user's head is located in water based on the magnitude of the water pressure and whether or not the temperature of the object is smaller than a predetermined value.
The display control device according to claim 1 or 4. The stimulus includes wind pressure on the user.
The display control device according to any one of claims 1 to 5 , wherein the display control device is characterized. The control means controls the display means to display an image that moves faster in a predetermined space as the wind pressure increases.
The display control device according to claim 6. The display means superimposes and displays the display image on the real space.
The display control device according to any one of claims 1 to 7, wherein the display control device is characterized. Further having a position acquisition means for acquiring the user's position,
The control means controls the display image according to the magnitude of the stimulus and the position.
The display control device according to any one of claims 1 to 8, wherein the display control device is characterized by the above. The position acquisition means determines which of the plurality of pools the user is located in, and determines which pool the user is located in.
The control means controls the display image so as to display an image according to the pool in which the user is located.
The display control device according to claim 9. Further having a display means for displaying the display image,
The acquisition means is a sensor that detects the stimulus.
The display control device according to any one of claims 1 to 10. A display device capable of communicating with the display control device according to any one of claims 1 to 10.
A sensor that detects external stimuli to the user,
A transmission means for transmitting the magnitude of the stimulus to the display control device, and
An acquisition means for acquiring the display image from the display control device, and
A display means for displaying the display image and
Have,
A display device characterized by that. Further, it has a mounting means for fixing the relative position between the user's head and the display means.
The display device according to claim 12. The acquisition step to acquire the magnitude of the external stimulus to the user,
A control step that controls a display image of the display means according to the magnitude of the stimulus, and
Have,
The magnitude of the stimulus includes water pressure and
In the control step,
From the magnitude of the water pressure, it is determined whether or not the user's head is located in the water.
When it is determined that the user's head is located in the water, the underwater image is controlled to be displayed on the display means.
When it is determined that the user's head is not located in the water, the display means is controlled to display the image in the air.
A display control method characterized by that. A program for causing a computer to function as each means of the display control device according to any one of claims 1 to 10.

Images