JP7580834B2 - Program and image display system - Google Patents

Description

The present invention relates to technology for displaying images captured in a virtual space.

Technology has been proposed for displaying on a display device an image captured by a virtual camera (hereinafter referred to as a "virtual camera") installed in a virtual space. For example, Patent Document 1 discloses a configuration for displaying a menu image in a virtual space according to the tilt of a head-mounted display.

JP 2016-115122 A

Content such as games that allow communication between objects such as characters in a virtual space has been proposed. When this type of content is displayed using a head-mounted display, there is a problem in that it is difficult for the user to grasp the positional relationship between a virtual straight line (hereinafter referred to as a "reference line") such as the optical axis of a virtual camera or the user's line of sight, and an object such as a character in the virtual space. In consideration of the above circumstances, the present invention aims to make it easier for the user to grasp the positional relationship between an object in a virtual space and a reference line.

In order to solve the above problems, a program according to a preferred embodiment of the present invention causes a computer to function as a display control unit that causes a display device of the head-mounted display to display a stereoscopic image utilizing binocular parallax, the stereoscopic image being an image captured of a virtual space by a virtual camera whose direction is controlled according to the direction of a head-mounted display, the display control unit causing an instruction image to be displayed in the direction of a reference line whose direction in the virtual space changes according to the direction of the head-mounted display, and changing the display mode of the instruction image according to the positional relationship between the reference line and an object in the virtual space.

An image display system according to a preferred embodiment of the present invention is an image display system including a head-mounted display and an information processing device, the information processing device including a display control unit that causes a display device of the head-mounted display to display a stereoscopic image utilizing binocular parallax, the stereoscopic image being an image captured of a virtual space by a virtual camera whose direction is controlled according to the direction of the head-mounted display, the display control unit causing a pointer image to be displayed in the direction of a reference line whose direction in the virtual space changes according to the direction of the head-mounted display, and changing the display mode of the pointer image according to the positional relationship between the reference line and an object in the virtual space.

1 is a perspective view illustrating an example of the appearance of an image display system according to a first embodiment of the invention. FIG. 1 is a block diagram illustrating a configuration of an image display system. FIG. 2 is a block diagram illustrating an example of a functional configuration of the image display system. FIG. 1 is an explanatory diagram of a virtual space. FIG. 2 is an explanatory diagram of an image displayed by a display device. 4 is a flowchart illustrating the contents of a process executed by a control device. FIG. 13 is an explanatory diagram of the rotation of the HMD in the roll direction. FIG. 2 is an explanatory diagram of an image displayed by a display device. FIG. 13 is an explanatory diagram of the movement of the virtual camera in modification example A4. FIG. 13 is an explanatory diagram of a change in the imaging range in modification example A5. FIG. 11 is a block diagram illustrating a functional configuration of an image display system according to a second embodiment. 13 is an explanatory diagram of an image displayed when the reference line does not intersect with the character in the second embodiment. FIG. 13 is an explanatory diagram of an image displayed when a reference line intersects with a character in the second embodiment. FIG. 13 is an explanatory diagram of the relationship between a character and an instruction image in the second embodiment. FIG. 10 is a flowchart illustrating the contents of a process executed by a control device of a second embodiment. FIG. 13 is an explanatory diagram of a change over time in an instruction image in modification example B2. FIG. 13 is an explanatory diagram of an instruction image in modification example B3. FIG. 13 is an explanatory diagram of an instruction image in modification example B4. FIG. 13 is a perspective view illustrating an example of the appearance of an image display system in modification C5.

[A: First embodiment]
FIG. 1 is a perspective view illustrating an example of the appearance of an image display system 1A according to a first embodiment of the present invention. The image display system 1A is a video device for displaying an image, and includes a terminal device 10 and a wearing tool 20. The terminal device 10 is a portable information terminal such as a smartphone or a tablet terminal. The terminal device 10 is detachably installed on the wearing tool 20. The wearing tool 20 is a tool for wearing the terminal device 10 on the head of a user U. For example, as illustrated in FIG. 1, a goggle-type attachment having a belt wrapped around the head of the user U is suitable as the wearing tool 20.

Figure 2 is a block diagram illustrating the configuration of the terminal device 10. As illustrated in Figure 2, the terminal device 10 is a computer system including a control device 11, a storage device 12, a display device 13, and a detection device 14. The control device 11 is a processor such as a CPU (Central Processing Unit), and controls each element of the image display system 1A in an integrated manner. The storage device 12 stores programs executed by the control device 11 and various data used by the control device 11. For example, the storage device 12 is configured from a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of multiple types of recording media.

The display device 13 displays an image under the control of the control device 11. For example, a flat display such as a liquid crystal display panel or an organic EL (Electroluminescence) display panel is used as the display device 13. As can be seen from FIG. 1, when the wearing device 20 is worn on the head of the user U, the display device 13 is disposed in front of both eyes of the user U with the display surface facing the head of the user U. The display device 13 of the first embodiment displays a stereoscopic image that allows the user U to perceive a three-dimensional effect. The stereoscopic image is composed of an image for the right eye and an image for the left eye to which binocular parallax has been imparted. The user U perceives a three-dimensional effect by having the right eye of the user U view the image for the right eye and the left eye of the user U view the image for the left eye.

As illustrated in FIG. 2, the display device 13, the detection device 14, and the wearing device 20 constitute a head mounted display (hereinafter referred to as "HMD") 30, and the control device 11 and the storage device 12 constitute an information processing device 40 that displays an image on the HMD 30. That is, the image display system 1A includes the HMD 30 and the information processing device 40. As described above, in the first embodiment, the information processing device 40, the display device 13, and the detection device 14 are realized by a single terminal device 10. Note that the information processing device 40 may be understood as an element of the HMD 30.

The detection device 14 in FIG. 2 is a sensor that outputs a detection signal corresponding to the attitude of the HMD 30. Specifically, the detection device 14 is composed of one or more types of sensors selected from a plurality of types of sensors, such as a gyro sensor that detects angular velocity, an acceleration sensor that detects acceleration, an inclination sensor that detects tilt angle, and a geomagnetic sensor that detects direction using geomagnetism. By analyzing the detection signal output from the detection device 14, it is possible to identify the attitude or a change in attitude of the HMD 30. The detection device 14 can also be described as a sensor that outputs a detection signal corresponding to the attitude of the display device 13 or the terminal device 10. Note that for convenience, an amplifier that amplifies the detection signal and an A/D converter that converts the detection signal from analog to digital are omitted from the illustration.

1, the posture of the HMD 30 is defined by three mutually orthogonal axes (X-axis, Y-axis, Z-axis) at the origin O. The X-axis is an axis perpendicular to the display surface of the display device 13 and corresponds to the front-rear direction of the user U. The Y-axis and Z-axis are axes parallel to the display surface of the display device 13. The Y-axis corresponds to the left-right direction of the user U, and the Z-axis corresponds to the up-down direction of the user U. Focusing on the display surface of the display device 13, the X-axis corresponds to the depth direction of the display surface, the Y-axis corresponds to the horizontal direction of the display surface, and the Z-direction corresponds to the vertical direction of the display surface. The direction of the circumference around the X-axis is the roll direction, the direction of the circumference around the Y-axis is the pitch direction, and the direction of the circumference around the Z-axis is the yaw direction. The direction of the X-axis is determined by the angle in the pitch direction and the angle in the yaw direction. Rotation in the roll direction corresponds to the action of tilting the head left and right while the user U faces forward. That is, for example, when the user U tilts his/her head, the HMD 30 rotates in the roll direction. In the following description, as illustrated in FIG. 1, one side in the roll direction is referred to as the "first side" and the other side is referred to as the "second side." One of the clockwise and counterclockwise directions around the X-axis is the first side, and the other is the second side.

The control device 11 in FIG. 2 executes a program stored in the storage device 12, and functions as a posture analysis unit 41, a display control unit 42A, and a motion control unit 43A, as illustrated in FIG. 3. Note that some or all of the functions of the control device 11 may be realized by a dedicated electronic circuit.

The posture analysis unit 41 identifies the posture of the HMD 30 by analyzing the detection signal output by the detection device 14. Specifically, the posture analysis unit 41 sequentially generates posture data related to the posture of the HMD 30 at a predetermined cycle. The posture data is data that represents the posture of the HMD 30 or a change in posture.

The display control unit 42A displays an image captured by a virtual camera E in a virtual space V on the display device 13. FIG. 4 is an explanatory diagram of the virtual space V. As illustrated in FIG. 4, the virtual camera E is installed in the virtual space V and captures an image of a specific range (hereinafter referred to as an "imaging range") R in the virtual space V. The imaging range R is a range that spans a predetermined angle in the vertical and horizontal directions with respect to the optical axis of the virtual camera E. The optical axis of the virtual camera E corresponds to the virtual line of sight of the user U in the virtual space V. The display control unit 42A of the first embodiment displays a stereoscopic image (an image for the right eye and an image for the left eye) using binocular parallax on the display device 13. Therefore, the virtual camera E includes a first virtual camera that captures an image for the left eye and a second virtual camera that captures an image for the right eye. Image data representing the stereoscopic image captured by the virtual camera E is sequentially supplied from the display control unit 42A to the display device 13, and the stereoscopic image is displayed on the display device 13.

The display control unit 42A controls the direction of the virtual camera E in the virtual space V according to the direction of the HMD 30 identified by the posture analysis unit 41. Specifically, the direction of the optical axis of the virtual camera E (and further the imaging range R) changes in conjunction with the rotation of the X-axis around the origin O. For example, when the HMD 30 rotates in the yaw direction, the optical axis of the virtual camera E rotates left and right around the origin O, and when the HMD 30 rotates in the pitch direction, the optical axis of the virtual camera E rotates up and down around the origin O. In the first embodiment, even when the HMD 30 rotates in the roll direction, the posture of the virtual camera E (angle around the optical axis) does not change. However, the inclination of the virtual camera E around the optical axis may be changed according to the rotation of the HMD 30 in the roll direction.

As illustrated in FIG. 4, a character C is placed in a virtual space V in which a virtual camera E is installed. In the first embodiment, the character C is an object representing a virtual creature (e.g., a human, animal, or monster) active in the virtual space V, and is configured to include a head and a torso. The image display system 1A provides a user U with a game in which the user can interact with the character C by making contact (skinship) with the character C at any time in the virtual space V.

A reference line Q is set within the virtual space V. The reference line Q is a straight line whose direction within the virtual space V changes depending on the direction of the HMD 30 (the direction of the X-axis). In the first embodiment, as illustrated in FIG. 4, the optical axis of the virtual camera E is exemplified as the reference line Q. For example, if the user U turns his head to the right, the reference line Q rotates to the right, and if the user U turns his head to the left, the reference line Q rotates to the left. As can be understood from the above explanation, the reference line Q is the virtual line of sight of the user U within the virtual space V.

Figure 5 is a schematic diagram of an image displayed on the display device 13. As illustrated in Figure 5, the display control unit 42A causes the display device 13 to display a stereoscopic image captured by the virtual camera E in the virtual space V. When a character C is located within the imaging range R, the character C is displayed on the display device 13. The display control unit 42A also causes the display device 13 to display an image (hereinafter referred to as an "indication image") G indicating the direction of the reference line Q. The indication image G is placed at a predetermined position on the reference line Q in the virtual space V. Since the reference line Q in the first embodiment is the optical axis of the virtual camera E, the indication image G is displayed at a predetermined position on the display surface of the display device 13.

The movement control unit 43A in FIG. 3 controls the movement of character C within virtual space V. In addition to controlling the movement of character C in accordance with the progress of the game, the movement control unit 43A also causes character C to perform a movement (hereinafter referred to as a "reaction movement") in response to contact by user U within virtual space V. Specifically, the movement control unit 43A determines whether or not user U has contacted character C within virtual space V, and causes character C to perform a reaction movement in response to the contact by user U. For example, various movements such as a change in posture, a change in facial expression, or the pronunciation of a specific line are typical examples of reaction movements.

The motion control unit 43A judges whether the user U has contacted the character C in the virtual space V according to the change in the posture of the HMD 30 identified by the posture analysis unit 41. In the first embodiment, the motion control unit 43A judges that the user U has contacted the character C in the virtual space V when the HMD 30 has rotated in the roll direction. That is, the rotation of the HMD 30 in the roll direction corresponds to an instruction (motion instruction) for the user U to contact the character C in the virtual space V. Specifically, when the HMD 30 has rotated in the roll direction, the motion control unit 43A judges that the user U has contacted a point P where the reference line Q and the surface of the character C intersect (hereinafter referred to as the "contact point") when the HMD 30 has rotated in the roll direction. The contact point P is located on the surface of the character C. Note that when the reference line Q intersects with the surface of the character C at multiple points, the point closest to the virtual camera E (i.e., the user U) among the multiple points is selected as the contact point P.

As described above, in the first embodiment, the movement of character C in virtual space V is controlled in response to the rotation of HMD 30 in the roll direction. Therefore, instructions from user U can be reflected in the movement of character C without the need to prepare an operation device separate from HMD 30 through which user U inputs instructions to control the movement of character C. In particular, in the first embodiment, it is determined that user U has come into contact with character C when reference line Q and character C intersect, so that user U can operate character C while keeping character C within his or her field of vision. In other words, there is an advantage in that user U can operate character C more intuitively.

Figure 6 is a flow chart illustrating the specific content of the process (hereinafter referred to as the "first control process") executed by the operation control unit 43A. The operation control unit 43A repeatedly executes the first control process of Figure 6 at a predetermined cycle. The storage device 12 stores contact determination data indicating whether or not the user U is in contact with the character C in the virtual space V (hereinafter referred to as the "contact state"). The contact determination data is, for example, a flag indicating either a contact state or a non-contact state.

When the first control process is started, the operation control unit 43A determines whether or not the contact determination data indicates a contact state (Sa1). If the contact determination data indicates a non-contact state (Sa1: NO), the operation control unit 43A determines whether or not the HMD 30 has rotated in the roll direction (Sa2). Specifically, the operation control unit 43A determines whether or not the HMD 30 has rotated to the first side in the roll direction (e.g., clockwise) by referring to the posture data sequentially supplied from the posture analysis unit 41.

7 is an explanatory diagram of a process for determining whether the HMD 30 has rotated in the roll direction. As illustrated in FIG. 7, the operation control unit 43A determines that the HMD 30 has rotated in the roll direction when the angle θ at which the HMD 30 has rotated to the first side in the roll direction exceeds a threshold θt. The angle θ is the angle at which the Z axis or the Y axis has rotated around the X axis, and is defined, for example, with the vertical direction as a reference (θ=0). On the other hand, when the rotation angle θ of the HMD 30 is below the threshold θt, the operation control unit 43A does not determine that the HMD 30 has rotated in the roll direction. As described above, in the first embodiment, the HMD 30 is not determined to have rotated in the roll direction when the rotation is below the threshold θt. Therefore, the possibility that the HMD 30 is determined to have rotated excessively frequently can be reduced. In addition, there is also an advantage that the possibility that the HMD 30 is determined to have rotated even when the user U does not intend to rotate the HMD 30 (i.e., the possibility of erroneous operation) can be reduced.

In addition, not only when the HMD 30 rotates in the roll direction alone, but also when the HMD 30 rotates in the roll direction as well as in the pitch or yaw direction, it is determined that the HMD 30 has rotated in the roll direction if the angle of rotation in the roll direction exceeds the threshold value θt.

In the first control process of FIG. 6, when it is determined that the HMD 30 has rotated to the first side in the roll direction (Sa2: YES), the operation control unit 43A determines whether the reference line Q intersects with the character C in the virtual space V (Sa3). That is, it is determined whether the character C exists on the reference line Q. As illustrated in FIG. 8, when the reference line Q intersects with the character C in the virtual space V (Sa3: YES), the operation control unit 43A determines that the user U is in a contact state in which the user U has contacted the contact point P of the character C (Sa4). Specifically, the contact determination data is changed to a numerical value indicating the contact state. As can be understood from the above explanation, when the HMD 30 has rotated in the roll direction, the operation control unit 43A determines that the user U has contacted the contact point P of the character C in the virtual space V. That is, the user U can instruct contact with the contact point P of the character C by rotating the HMD 30 in the roll direction.

When the contact state is entered, the motion control unit 43A causes the character C to perform a reaction action (Sa5). Specifically, the motion control unit 43A causes the character C to perform a reaction action according to the position of the contact point P where the reference line Q intersects with the character C. That is, the reaction action performed by the character C changes according to the position of the contact point P. For example, a table that specifies the type of reaction action for each of a plurality of regions that divide the surface of the character C is stored in the storage device 12. The motion control unit 43A causes the character C to perform a reaction action specified in the table for the region that includes the contact point P among the plurality of regions on the surface of the character C. For example, when the contact point P is located on the head of the character C, the motion control unit 43A causes the character C to perform a reaction action that accepts contact by the user U (e.g., a favorable action such as being happy, laughing, or approaching). On the other hand, when the contact point P is located on the torso of the character C, the motion control unit 43A causes the character C to perform a reaction action that rejects contact by the user U (e.g., a negative action such as being angry, sad, or moving away). With the above configuration, it is possible to vary the reaction behavior of character C in a variety of ways depending on the position of contact point P.

On the other hand, if the HMD 30 does not rotate to the first side in the roll direction (Sa2: NO), or if the reference line Q does not intersect with the character C (Sa3: NO), the motion control unit 43A maintains the contact determination data at a value indicating a non-contact state. In other words, even if the HMD 30 rotates to the first side in the roll direction, if the reference line Q does not intersect with the character C as in the example of FIG. 5, it is determined that the user U is not in contact with the character C.

When the user U comes into contact with the character C in the virtual space V, in step Sa1 of the first control process, it is determined that the contact determination data indicates a contact state. When the contact determination data indicates a contact state (Sa1: YES), the operation control unit 43A determines whether the HMD 30 has rotated to the second side in the roll direction (e.g., counterclockwise) by referring to the posture data sequentially supplied from the posture analysis unit 41. Specifically, the operation control unit 43A determines that the HMD 30 has rotated to the second side in the roll direction when the rotation angle θ with respect to the second side in the roll direction exceeds a predetermined threshold value θt. When the HMD 30 has rotated to the second side in the roll direction (Sa6: YES), the operation control unit 43A determines that the contact with the character C has been released (Sa7). Specifically, the operation control unit 43A changes the contact determination data to a value indicating a non-contact state. On the other hand, if the HMD 30 does not rotate to the second side in the roll direction (Sa6: NO), the operation control unit 43A maintains the contact determination data at a numerical value indicating a contact state.

Note that, when contact with character C is released, it is preferable that the action control unit 43A causes character C to perform an action that responds to the release of contact, but a configuration in which character C does not perform an action that responds to the release of contact is also envisioned.

As can be understood from the above explanation, the operation control unit 43A determines that the user U has contacted the character C (Sa4) when the HMD 30 has rotated to the first side in the roll direction (Sa2: YES). On the other hand, when the HMD 30 has rotated to the second side in the roll direction (Sa6: YES) while the user U is in contact with the character C (Sa1: YES), it determines that the contact with the character C has been released. With the above configuration, the user U can instruct the generation and release of contact with the character C in the virtual space V according to the direction of rotation of the HMD 30 in the roll direction. Since rotations in opposite directions in the roll direction correspond to the opposite actions of the generation and release of contact with the character C, there is also the advantage that the user U can easily intuitively grasp the generation and release of contact with the character C.

In the first embodiment, the following configuration may be adopted, for example:

[Modification A1]
The motion control unit 43A in the modified example A1 causes the character C to execute a reaction action according to the time length during which contact with the character C is maintained in the virtual space V. Specifically, the motion control unit 43A changes the reaction action over time during the period during which the contact state continues (hereinafter referred to as the "contact period"). The contact period is the period from when the HMD 30 rotates to the first side in the roll direction to when it rotates to the second side. The contact period may be the period from when the contact determination data is set to a value indicating a contact state to when it is changed to a value indicating a non-contact state. The motion control unit 43A changes the reaction action of the character C over time from the start point to the end point of the contact period. Therefore, the reaction action of the character C changes depending on the time length of the contact period. According to the above configuration, the reaction action of the character C can be varied in a variety of ways depending on the time length during which the user U contacts the character C.

[Modification A2]
In the storage device 12 in the modification A2, parameters related to the character C are stored. Specifically, the favorability rating of the character C to the user U is stored as a parameter in the storage device 12. The favorability rating is a parameter indicating the degree of favorable feelings from the character C to the user U. The operation control unit 43A changes the favorability rating according to the progress of the game, and also changes the favorability rating from the character C to the user U according to contact with the character C. Specifically, the greater the degree of contact (e.g., number of times or time) of the user U with the character C, the greater the favorability rating from the character C to the user U is set to. Note that, in the above example, the favorability rating of the character C to the user U has been described, but the favorability rating stored in the storage device 12 may be the favorability rating from the character C to the character (player character) used by the user U. The user U may have multiple player characters in the virtual space V.

The action control unit 43A also changes the reaction action according to the likeability level stored for the character C. That is, the type of reaction action that the character C executes (Sa4) when the user U comes into contact with the character C (Sa2: YES, Sa3: YES) changes according to the likeability level of the character C. For example, when the number of contacts or time with the character C is insufficient and the likeability level is low, the character C executes a reaction action to reject the contact when the user U touches the head. On the other hand, when the number of contacts or time with the character C is sufficient and the likeability level is high, the character C executes a reaction action to accept the contact when the user U touches the head. With the above configuration, the reaction action of the character C can be varied in a variety of ways according to the parameters related to the character C.

The parameters that affect the reaction behavior of character C are not limited to the favorability rating exemplified above. For example, various parameters such as the growth level of character C or the intimacy between user U and character C change in response to user U's contact with character C, and the reaction behavior of character C is controlled in response to these parameters.

[Modification A3]
The motion control unit 43A in the modified example A3 causes the character C to perform a reaction action according to the angle θ at which the HMD 30 rotates in the roll direction. For example, the rotation angle θ of the HMD 30 corresponds to the virtual pressure of the user U contacting the character C in the virtual space V. For example, when the rotation angle θ is small, it means that the user U is lightly contacting the character C, and when the rotation angle θ is large, it means that the user U is strongly pressing the character C. For example, when the rotation angle θ is small, the motion control unit 43A causes the character C to perform a reaction action to accept the contact by the user U, and when the rotation angle θ is large, it causes the character C to perform a reaction action to reject the contact by the user U. According to the above configuration, the reaction action of the character C can be varied in various ways according to the rotation angle θ of the HMD 30 (i.e., the virtual pressure of the user U contacting the character C).

[Modification A4]
The display control unit 42A in the modification A4 moves the virtual camera E (i.e., the virtual viewpoint in the virtual space V) in the virtual space V in response to the rotation of the HMD 30 in the roll direction. Specifically, the display control unit 42A changes the positional relationship between the virtual camera E and the character C in the virtual space V in conjunction with the rotation of the HMD 30. For example, the distance between the virtual camera E and the character C is controlled in accordance with the rotation angle θ of the HMD 30. That is, the virtual camera E moves toward the object in conjunction with the rotation of the HMD 30 in the roll direction.

For example, when the HMD 30 rotates to the first side in the roll direction, the display control unit 42A moves the virtual camera E closer to the character C in the virtual space V by an amount corresponding to the rotation angle θ (θ>0) of the HMD 30, as shown by the arrow a1 in FIG. 9. On the other hand, when the HMD 30 rotates to the second side in the roll direction, the display control unit 42A moves the virtual camera E away from the character C in the virtual space V by an amount corresponding to the rotation angle θ (θ<0) of the HMD 30, as shown by the arrow a2 in FIG. With the above configuration, the user U can move the virtual camera E in the virtual space V by a simple operation of rotating the HMD 30 in the roll direction.

[Modification A5]
The display control unit 42A in the modified example A5 changes the imaging range R (i.e., the angle of view of the virtual camera E) by the virtual camera E according to the rotation of the HMD 30 in the roll direction. That is, the range displayed on the display device 13 in the virtual space V changes according to the rotation of the HMD 30 in the roll direction. For example, when the HMD 30 rotates to the first side in the roll direction, the display control unit 42A reduces the imaging range R at a ratio according to the rotation angle θ of the HMD 30 as shown by the arrow b1 in FIG. 10. Therefore, for example, the character C in the virtual space V is zoomed in (enlarged). On the other hand, when the HMD 30 rotates to the second side in the roll direction, the display control unit 42A expands the imaging range R at a ratio according to the rotation angle θ of the HMD 30 as shown by the arrow b2 in FIG. Therefore, for example, the character C in the virtual space V is zoomed out (reduced). According to the above configuration, the user U can change the imaging range R of the virtual camera E in the virtual space V by simply rotating the HMD 30 in the roll direction.

[Modification A6]
In the first embodiment, the contact with the character C is determined to be released when the HMD 30 rotates to the second side in the roll direction (Sa6: YES), but the condition for determining that the contact with the character C is released is not limited to the above example. For example, the operation control unit 43A may determine that the contact with the character C is released when the HMD 30 returns to its original state from the state rotated in the roll direction (Sa2: YES). Specifically, in step Sa6 of FIG. 6, the operation control unit 43A determines that the contact with the character C is released when the rotation angle θ of the HMD 30 changes to the initial angle (e.g., θ=0) at the time when the rotation for contacting the character C is started (Sa7). On the other hand, in step Sa2, the operation control unit 43A determines that the user U has contacted the character C when the rotation angle θ changes, regardless of the direction of rotation of the HMD 30 (first side/second side). As can be understood from the above description, by monitoring the rotation angle θ of the HMD 30, it is possible to determine the occurrence and release of contact with the character C without determining the direction of rotation (first side/second side) of the HMD 30. In other words, in determining contact with the character C, the operation control unit 43A does not necessarily need to determine the direction of rotation of the HMD 30.

[Modification A7]
In the first embodiment, when the reference line Q intersects with the character C in the virtual space V (Sa3: YES), the character C is made to perform a reaction action to the contact by the user U, but the condition regarding the positional relationship between the reference line Q and the character C is not limited to the above example. For example, the character C may be made to perform a reaction action on the condition that the distance between the reference line Q and the character C is below a predetermined threshold. When the distance between the reference line Q and the character C is below a predetermined threshold, this includes not only the case where the reference line Q intersects with the character C, but also the case where the reference line Q is separated from the character C within a range below the threshold. In addition, the character C may be made to perform a reaction action on the condition that the reference line Q intersects with a specific part of the character C. When the reference line Q intersects with a part other than the specific part of the character C, the movement control unit 43A does not make the character C perform a reaction action. As can be understood from the above description, the movement control unit 43A in the first embodiment is comprehensively expressed as an element that controls the movement of the character C according to the positional relationship between the reference line Q and the character C (for example, the relationship where the reference line Q intersects with the character C).

[B: Second embodiment]
A second embodiment of the present invention will be described. In the following examples, the reference numerals used in the description of the first embodiment will be used for elements whose functions are similar to those of the first embodiment, and detailed descriptions of each will be omitted as appropriate.

The image display system in the second embodiment has the same configuration as the first embodiment illustrated in Figs. 1 and 2. That is, the HMD 30 in the second embodiment includes the display device 13 and the detection device 14 of the terminal device 10 and the wearing device 20, and the information processing device 40 includes the control device 11 and the storage device 12 of the terminal device 10.

Figure 11 is a block diagram illustrating an example of the functional configuration of the terminal device 10 of the second embodiment. As illustrated in Figure 11, the control device 11 of the second embodiment functions as a posture analysis unit 41, a display control unit 42B, and a motion control unit 43B by executing a program stored in the storage device 12. Note that a part or all of the control device 11 may be realized by a dedicated electronic circuit. As in the first embodiment, the posture analysis unit 41 sequentially generates posture data related to the posture of the HMD 30 by analyzing the detection signal output by the detection device 14.

Similar to the display control unit 42A in the first embodiment, the display control unit 42B causes the display device 13 to display an image within the imaging range R captured by the virtual camera E in the virtual space V, and an indication image G indicating the direction of the reference line Q. As described above in the first embodiment, the reference line Q is a virtual straight line whose direction in the virtual space V changes depending on the orientation of the HMD 30.

The display control unit 42B in the second embodiment changes the display mode of the instruction image G according to the positional relationship between the reference line Q and a character C (an example of an object) in the virtual space V. Specifically, the display control unit 42B changes the display mode of the instruction image G between a case where the reference line Q intersects with the character C and a case where the reference line Q does not intersect with the character C. The display mode refers to the characteristics of an image that can be visually distinguished by the user U. For example, the three attributes of color, that is, hue (tone), saturation, and brightness (tone), as well as size and image content (e.g., pattern or shape) are also included in the concept of the display mode.

When the reference line Q does not intersect with the character C, the display control unit 42B causes the display device 13 to display the image G1 as the instruction image G, as illustrated in FIG. 12. The image G1 is a circular or dot-shaped object arranged in the virtual space V, and is arranged on the reference line Q in the virtual space V. On the other hand, when the reference line Q intersects with the character C, the display control unit 42B causes the display device 13 to display an image G2 different from the image G1 as the instruction image G, as illustrated in FIG. 13. The image G2 is a planar object that typically represents the hand of the user U. The display control unit 42B places the image G2 at a point (i.e., contact point P) on the surface of the character C in the virtual space V that intersects with the reference line Q, as illustrated in FIG. 14. That is, the image G2 contacts the contact point P on the surface of the character C in the virtual space V. The contact point P can also be said to be a point where the image G2 contacts the surface of the character C.

In the above explanation, switching between images G1 and G2 has been exemplified, but changes in the display mode of the instruction image G also include switching between display and non-display. That is, the display control unit 42B may display image G2 as the instruction image G when the reference line Q intersects with the character C, and may not display the instruction image G when the reference line Q does not intersect with the character C.

As can be understood from the above explanation, in the second embodiment, the display mode of the indication image G, which indicates the direction of the reference line Q in the virtual space V, changes according to the positional relationship between the reference line Q and the character C. Therefore, the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V. In addition, since the indication image G (image G2) is placed on the surface of the character C in the virtual space V, there is an advantage that the user U can easily perceive the state of contact with the surface of the character C in the virtual space V. Another advantage is that the indication image G is placed on the surface of the character C, so that the user U can easily and accurately visually grasp the contact point P.

The movement control unit 43B in FIG. 11 controls the movement of the character C in the virtual space V, similar to the movement control unit 43A in the first embodiment. Specifically, the movement control unit 43B causes the character C to perform a movement (i.e., a reaction movement) that reacts to contact by the user U in the virtual space V. Typical examples of reaction movements are a change in posture, a change in facial expression, or the pronunciation of a specific line.

Specifically, the movement control unit 43B causes the character C to perform a movement according to the position of the contact point P where the reference line Q intersects with the character C. For example, when the contact point P is located on the head of the character C, the movement control unit 43B causes the character C to perform a reaction movement to accept contact by the user U, and when the contact point P is located on the torso of the character C, the movement control unit 43B causes the character C to perform a movement to reject contact by the user U. With the above configuration, it is possible to vary the movement of the character C in a variety of ways according to the position of the contact point P.

Figure 15 is a flowchart illustrating the specific content of the process (hereinafter referred to as the "second control process") executed by the display control unit 42B and the operation control unit 43B in the second embodiment. The second control process in Figure 15 is executed repeatedly at a predetermined cycle.

When the second control process starts, the display control unit 42B determines whether the reference line Q intersects with the character C in the virtual space V (Sb1). If the reference line Q intersects with the character C (Sb1: YES), the display control unit 42B causes the display device 13 to display the image G2 placed at the contact point P where the reference line Q and the character C intersect as the pointing image G (Sb2). On the other hand, if the reference line Q does not intersect with the character C (Sb1: NO), the display control unit 42B causes the display device 13 to display the image G1 placed on the reference line Q as the pointing image G (Sb3).

When image G2 is displayed overlapping character C, the action control unit 43B determines whether an action instruction (hereinafter referred to as "action instruction") for character C has been received from user U (Sb4). The action instruction is an instruction to cause an action to occur on character C in virtual space V. In the second embodiment, the action instruction means an instruction for user U to touch character C in virtual space V.

The motion control unit 43B in the second embodiment accepts a specific change in the posture of the HMD 30 as a motion instruction from the user U. Specifically, the motion control unit 43B determines that a motion instruction has been received when the change in the posture of the HMD 30 satisfies a predetermined condition (hereinafter referred to as the "instruction determination condition"). For example, similar to the first embodiment, the motion control unit 43B determines that a motion instruction has been given from the user U when the HMD 30 has rotated in the roll direction. That is, the rotation of the HMD 30 in the roll direction is the instruction determination condition. On the other hand, when the change in the posture of the HMD 30 does not satisfy the instruction determination condition, the motion control unit 43B determines that a motion instruction has not been received. Note that, as described above with reference to FIG. 7, a configuration in which it is determined that the HMD 30 has rotated when the rotation angle θ of the HMD 30 exceeds the threshold value θt is preferable.

When an action instruction is received from the user U (Sb4: YES), the action control unit 43B determines that the user U has contacted the character C in the virtual space V, and causes the character C to perform an action in response to the contact by the user U (Sb5). Specifically, as described above, the action control unit 43B causes the character C to perform an action according to the position of the contact point P where the reference line Q intersects with the character C. As can be understood from the above explanation, in the second embodiment, the user U can contact the desired position of the character C in the virtual space V by giving an action instruction in a state where the reference line Q has been moved so that it intersects with the desired position of the character C.

Note that in a configuration in which changes in the posture of the HMD 30 are recognized as operational instructions, there is a problem in that since direct operations such as touch operations on the display surface are not required, the user U has difficulty in feeling the sensation of contact. In the second embodiment, the display mode of the instruction image G changes depending on the positional relationship between the reference line Q and the character C (specifically, whether or not the reference line Q intersects with the character C), so compared to a configuration in which the instruction image G is displayed in a fixed manner, there is an advantage in that the user U can easily feel the sensation of contacting the character C in the virtual space V.

In the second embodiment, the following configuration may be adopted, for example:

[Modification B1]
In the second embodiment, the character C is caused to perform a reaction action according to the position of the contact point P, but similarly to the first embodiment, the character C may be caused to perform a reaction action according to the change in the attitude of the HMD 30 (for example, the rotation angle θ in the roll direction). For example, the change in the attitude of the HMD 30 is assumed to be a virtual pressure of the user U contacting the character C in the virtual space V. The action control unit 43B of the modification B1 causes the character C to perform a reaction action to accept contact by the user U when the change in the attitude of the HMD 30 is small, and causes the character C to perform a reaction action to reject contact by the user U when the change in the attitude is large. According to the above aspect, it is possible to cause the character C to perform a variety of reaction actions according to the attitude of the HMD 30.

[Modification B2]
The movement control unit 43B of the modified example B2 accepts movement instructions from the user U within a specific period on the time axis (hereinafter referred to as the "instruction receiving period"). That is, the movement instructions given by the user U during the instruction receiving period are effectively reflected in the movement of the character C, but the movement instructions given by the user U after the instruction receiving period has elapsed are ignored. The instruction receiving period is, for example, a period of a predetermined length of time (for example, several seconds) starting from the point at which the reference line Q begins to intersect with the character C. Note that the start point or end point of the instruction receiving period is not limited to the above examples. For example, the start point of the instruction receiving period may be the point at which a specific event begins in the game. By limiting the acceptance of movement instructions to the instruction receiving period as described above, a moderate sense of tension is given to the user U, and as a result, it is possible to suppress boredom of the game.

The display control unit 42B changes the display mode of the instruction image G (image G2) over time during the instruction acceptance period. For example, as illustrated in FIG. 16, assume that the image G2 displayed as the instruction image G when the reference line Q intersects with the character C includes the image G21 and the image G22. The image G21 is an image representing the hand of the user U, similar to the image G2 in the first embodiment, and the image G22 is a circular image arranged behind the image G21. The display control unit 42B reduces the size of the image G22 over time from the start point to the end point of the instruction acceptance period. The size of the image G21 does not change. The above configuration has the advantage that the user U can visually grasp the instruction acceptance period (e.g., the remaining time) during which the acceptance of the operation instruction is permitted from the display mode of the instruction image G.

[Modification B3]
The display control unit 42B of the modification B3 changes the display mode of the instruction image G according to the position of the contact point P of the character C in the virtual space V where the reference line Q intersects. Specifically, as illustrated in Fig. 17, the display mode of the instruction image G differs between a case where the contact point P is at a first position on the surface of the character C in the virtual space V and a case where the contact point P is at a second position on the surface of the character C. The first position is, for example, a position where the character C accepts contact by the user U (for example, the torso of the character C), and the second position is, for example, a position where the character C rejects contact by the user U (for example, the head of the character C).

Note that while FIG. 17 illustrates an example in which the contact point P is at the first or second position, the display control unit 42B may change the display mode of the instruction image G over time in parallel with the change in the position of the contact point P on the character C. That is, when the user U moves the contact point P on the surface of the character C in accordance with the posture of the HMD 30, the display control unit 42B changes the display mode of the instruction image G over time in conjunction with the movement of the contact point P. For example, during the period in which the contact point P moves from the first position to the second position in FIG. 17, the display control unit 42B changes the display mode of the instruction image G continuously or stepwise from the display mode at the first position to the display mode at the second position.

In variant B3, the display mode of the instruction image G changes depending on the position of the contact point P. Therefore, by visually checking the display mode of the instruction image G, the user U can guess the action of the character C at the time of contact. In other words, by checking the display mode of the instruction image G, the user U can gradually move the contact point P while guessing the action of the character C at the time of contact, and provide an action instruction while maintaining the contact point P in the position where the desired action is expected, providing the user U with the excitement of doing so.

[Modification B4]
The display control unit 42B of the modification B4 controls the character C so that the line of sight of the character C tracks the reference line Q. For example, the character C is controlled so that the line of sight is directed toward the contact point P where the reference line Q intersects with the character C. Specifically, as illustrated in FIG. 18, the eyeballs (e.g., pupils) and head of the character C rotate so as to track the reference line Q. Note that only one of the eyeballs or the head of the character C may track the reference line Q. The modification B4 has the advantage that the user U can easily grasp the sense of the influence that the user U is having on the character C.

[Modification B5]
In the second embodiment, the display mode of the instruction image G is changed depending on whether the reference line Q intersects with the character C, but the conditions for changing the display mode of the instruction image G are not limited to the above examples. For example, the display control unit 42B may change the display mode of the instruction image G continuously or stepwise depending on the distance between the reference line Q and the character C. The intersection of the reference line Q and the character C is not an essential condition for changing the display mode of the instruction image G. In addition, the display mode of the instruction image G may be changed depending on whether the reference line Q intersects with a specific part of the character C. In a state in which the reference line Q intersects with a part other than the specific part of the character C, the instruction image G is displayed in the same display mode as when the reference line Q does not intersect with the character C. As can be understood from the above description, the operation control unit 43B of the second embodiment is comprehensively expressed as an element that changes the display mode of the instruction image G depending on the positional relationship between the reference line Q and the character C.

[Modification B6]
The configuration of the second embodiment in which the display mode of the instruction image G is changed according to the positional relationship between the reference line Q and the character C is also applied to the case where the user U uses the terminal device 10 (e.g., a smartphone) while holding it in his/her hand. As in the second embodiment, the imaging range R of the virtual space V displayed on the display device 13 changes according to the attitude of the terminal device 10.

The terminal device 10 of the modified example B6 is equipped with a touch panel that detects the contact of the user U with the display surface of the display device 13. In the second embodiment, the position of the contact point P in the virtual space V is changed according to the attitude of the HMD 30, but in the modified example B6, the point where the user U contacts the display surface of the display device 13 is set as the contact point P, and an instruction image G is displayed. For example, a reference line Q passing through the contact point P where the user U contacts is set in the virtual space V. That is, a virtual straight line in the direction indicated by the user U by contacting (i.e., touch operation) the display surface of the display device 13 is set as the reference line Q in the virtual space V. The display control unit 42B causes the display device 13 to display the image G2 as the instruction image G when the reference line Q intersects with the character C, and causes the display device 13 to display the image G1 as the instruction image G when the reference line Q does not intersect with the character C. The above configuration also has the advantage that the user U can easily grasp the positional relationship between the reference line Q and the character C in the virtual space V (for example, the state in which the user U is in contact with the character C in the virtual space V).

As can be understood from the example of variant example B6, in the second embodiment, the configuration in which the display device 13 is attached to the head of the user U may be omitted. In a configuration that does not assume that the display device 13 will be attached to the head of the user U, the display control unit 42B is expressed as an element that causes the display device 13 to display an image of the virtual space V captured by a virtual camera E whose attitude is controlled according to the attitude of the display device 13. Furthermore, in a display device 13 that is used while being held in the hand of the user U, a virtual straight line in the direction indicated by an operation (e.g., a touch operation) on the display device 13 is the "reference line."

[C: Other Modifications]
Specific examples of modifications to the first or second embodiment are given below. Two or more examples selected from the following examples may be combined as appropriate within a range that is not mutually contradictory.

[Modification C1]
The reference line Q is not limited to the optical axis of the virtual camera E exemplified in each of the above-mentioned embodiments. For example, a straight line perpendicular to the display surface of the display device 13 is used as the reference line Q. Also, a straight line forming a predetermined angle with respect to the optical axis of the virtual camera E or a straight line perpendicular to the display surface may be used as the reference line Q. In addition, in the HMD 30 equipped with an eye tracking (gaze measurement) function for estimating the direction of the gaze of the user U, the gaze estimated by the function may be used as the reference line Q. As can be understood from the above examples, the reference line Q is comprehensively expressed as a virtual straight line set in the virtual space V.

[Modification C2]
In each of the above-mentioned embodiments, a character C is exemplified as an object placed in the virtual space V, but the object that the user U comes into contact with in the virtual space V is not limited to the character C. For example, inanimate elements such as buildings and natural objects in the virtual space V are also included in the concept of an object. An "action" of a living object (character C) is, for example, the behavior, action, appearance, or attitude of the object. Also, an "action" of an inanimate object is, for example, a change in the form of the element. For example, the action of opening a door of a building, or the action of a natural object such as a rock deforming or moving are exemplified as the action of an object.

[Modification C3]
In each of the above-mentioned embodiments, when the HMD 30 rotates in the roll direction, it is determined that the user U has contacted the character C in the virtual space V, and the character C is made to perform a reaction action, but the trigger for the reaction action (i.e., the action instruction) is not limited to the above examples. For example, the rotation of the HMD 30 in the pitch direction or yaw direction may be regarded as a contact by the user U, and the character C may be made to perform a reaction action. In addition, the character C may be made to perform a reaction action on the condition that, for example, the reference line Q intersects within a specific range of the character C (i.e., the user U stares within the range for a long time) continues for a predetermined time. The character C may be made to perform a reaction action on the condition that the user U operates an operation device (not shown) connected to the HMD 30. As can be understood from the above examples, the configuration in which the change in the attitude of the HMD 30 (particularly the rotation in the roll direction) is used as a trigger for the reaction action of the character C may be omitted.

[Modification C4]
In each of the above-described embodiments, the rotation of the HMD 30 in the roll direction is accepted as a motion instruction (an instruction to contact the character C), but the change in the attitude of the HMD 30 that is determined as a motion instruction is not limited to the rotation in the roll direction. For example, the rotation of the HMD 30 in the pitch direction or the yaw direction may be accepted as a motion instruction.

[Modification C5]
The modified examples (Modified Example A1 to Modified Example A7) exemplified for the first embodiment are similarly applied to the second embodiment, and the modified examples (Modified Example B1 to Modified Example B6) exemplified for the second embodiment are similarly applied to the first embodiment.

[Modification C6]
In each of the above-described embodiments, the display device 13 and the detection device 14 of the HMD 30 and the information processing device 40 that controls the HMD 30 are realized by a single terminal device 10, but the HMD 30 and the information processing device 40 may be realized as separate devices as in the image display system 1B in FIG. 19. The HMD 30 and the information processing device 40 can communicate with each other by wire or wirelessly. Any method of communication between the HMD 30 and the information processing device 40 may be used, but short-range wireless communication such as Bluetooth (registered trademark) is preferable.

The HMD 30 is equipped with a display device 13, a detection device 14, and a wearing device 20, as in the first embodiment, and is worn on the head of the user U. The information processing device 40 is a control device that displays various images on the HMD 30 by communicating with the HMD 30, and is equipped with a control device 11 and a storage device 12. For example, various information terminals such as mobile phones, smartphones, tablet terminals, personal computers, or home game devices are used as the information processing device 40. It does not matter whether the information processing device 40 is portable or stationary. The control device 11 executes a program stored in the storage device 12 to function as a posture analysis unit 41, a display control unit 42 (42A, 42B), and a motion control unit 43 (43A, 43B), as in the first or second embodiment. Therefore, the same effects as those of the first or second embodiment are achieved in the configuration of FIG. 19.

[Modification C7]
A preferred embodiment of the present invention is realized by cooperation between a computer (specifically, the control device 11) and a program, as exemplified in each of the above-mentioned embodiments. The program according to each of the above-mentioned embodiments is provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and is exemplified by an optical recording medium (optical disk) such as a CD-ROM, but includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except a transient, propagating signal, and does not exclude a volatile recording medium. It is also possible to provide the program to the computer in the form of distribution via a communication network.

[D: Supplementary Note]
From the above description, for example, preferred embodiments of the present invention can be understood as follows. In order to facilitate understanding of each embodiment, reference numerals in the drawings are written in parentheses for convenience, but the present invention is not limited to the illustrated embodiments.

[Aspect 1]
A program according to a preferred embodiment (embodiment 1) of the present invention is a program that causes a computer (11) to function as a display control unit (42B) that causes a display device (13) of the head-mounted display (30) to display a stereoscopic image using binocular parallax, the stereoscopic image being an image captured in a virtual space (V) by a virtual camera (E) whose direction is controlled according to the direction of a head-mounted display (30), and the display control unit (42B) displays an instruction image (G) in the direction of a reference line (Q) whose direction in the virtual space (V) changes according to the direction of the head-mounted display (30), and changes the display mode of the instruction image (G) according to the positional relationship between the reference line (Q) and an object (C) in the virtual space (V). In the above configuration, the display mode of the instruction image (G) that represents the direction of the reference line (Q) in the virtual space (V) changes according to the positional relationship between the reference line (Q) and the object (C), so that the user (U) can easily grasp the positional relationship between the reference line (Q) and the object (C) in the virtual space (V).

The "reference line" is a virtual straight line set in the virtual space (V). For example, the optical axis of the virtual camera (E), the center line of the display screen of the display device (13), or a straight line that forms a predetermined angle with these straight lines are suitable examples of the reference line (Q). In a configuration that can use an eye tracking function that estimates the gaze of the user (U), in addition to the straight lines exemplified above, the gaze of the user (U) may also be used as the reference line (Q).

An "object" is a virtual object placed in the virtual space (V). A typical example of an object (C) is a character (e.g., a human, animal, or monster), but the concept of an object can also include inanimate elements such as buildings or natural objects in the virtual space (V).

"Display mode" refers to the visually distinguishable characteristics of an image. For example, the three attributes of color - hue (tone), saturation, and brightness (gradation) - as well as size and image content (e.g., pattern or shape) are all included in the concept of display mode. Changes in display mode also include switching between display and non-display.

[Aspect 2]
In a preferred example (Aspect 2) of Aspect 1, the display control unit (42B) displays the instruction image (G) at a point (P) on the surface of the object (C) that intersects with the reference line (Q). According to the above aspect, since the instruction image (G) is disposed on the surface of the object (C) in the virtual space (V), there is an advantage that the user (U) can easily perceive the state of contact with the surface of the object (C) from the instruction image (G).

[Aspect 3]
A program according to a preferred example (Aspect 3) of Aspect 1 or Aspect 2 causes the computer (11) to function as a motion control unit (43B) that controls the motion of the object (C) in accordance with the positional relationship between the reference line (Q) and the object (C). According to the above aspect, it is possible to cause the object (C) to execute various motions that reflect instructions from the user (U).

The "movement of an object" refers to, for example, the behavior, action, appearance, or attitude of a character, which is an example of an object (C). With respect to an inanimate element, the "movement of an object" refers to, for example, a change in the form of the element (for example, a door or window of a building opening, or a natural object such as a rock deforming or moving, etc.).

[Aspect 4]
In a preferred example (Aspect 4) of Aspect 3, when the motion control unit (43B) receives a motion instruction from the user (U), it controls the motion of the object (C) in accordance with the positional relationship between the reference line (Q) and the object (C). According to the above aspect, the user (U) can execute a motion corresponding to the desired position on the object (C) by giving a motion instruction in a state where the reference line (Q) has been moved to the desired position.

An "action instruction" is an instruction to cause an action (e.g., contact) to occur on an object (C) in the virtual space (V). It can also be said as an instruction to confirm the provisional direction represented by the instruction image (G) as the direction desired by the user (U). For example, it is determined that an action instruction has been received when a specific change (e.g., rotation in the roll direction) occurs in the direction of the head-mounted display (30). It may also be determined that an action instruction has been received when, for example, the reference line (Q) remains stationary in the virtual space (V) for a predetermined period of time (e.g., 2 seconds).

[Aspect 5]
In a preferred example (aspect 5) of aspect 4, the action control unit (43B) determines that the action instruction has been received when a change in the attitude of the head mounted display (30) satisfies a predetermined condition, and causes the object (C) to execute an action corresponding to the amount of change in the attitude, and the display control unit (42B) changes the display mode of the instruction image (G) according to the change in the attitude of the head mounted display (30) before receiving the action instruction. According to the above aspect, it is possible to cause the object (C) to execute various actions according to the attitude of the head mounted display (30). In addition, since the display mode of the instruction image (G) changes according to the attitude of the head mounted display (30), there is an advantage that the user (U) can easily grasp the sense of the user (U) influencing the object (C) (for example, the sense of contact). Note that the "predetermined condition" is a condition of the attitude change that is determined to be an action instruction by the user (U), and is, for example, a rotation in the roll direction in the embodiment.

[Aspect 6]
In a preferred example (Aspect 6) of Aspect 4 or Aspect 5, the display control unit (42B) changes the display mode of the instruction image (G) over time during an instruction acceptance period during which acceptance of the operation instruction by the user (U) is permitted. According to the above aspect, the user (U) can visually grasp the instruction acceptance period during which acceptance of the operation instruction is permitted (e.g., the remaining time of the instruction acceptance period) from the display mode of the instruction image (G). Note that the "instruction acceptance period" is, for example, the remaining time during which one contact with a character is permitted, or the remaining time of an operation mode during which contact with a character is permitted.

[Aspect 7]
In a preferred example (Aspect 7) of any one of Aspects 3 to 6, the display control unit (42B) changes the display mode of the instruction image (G) according to the position of the point (P) of the object (C) where the reference line (Q) intersects. According to the above aspects, since the display mode of the instruction image (G) changes according to the position of the point (P) of the object (C) where the reference line (Q) intersects, the user (U) can infer the movement of the object (C) from the display mode of the instruction image (G).

[Aspect 8]
In a preferred example (aspect 8) of aspect 7, the display control unit (42B) changes the display mode of the indication image (G) over time in parallel with the change in the position of the point (P) of the object (C) where the reference line (Q) intersects. According to the above aspect, the display mode of the indication image (G) changes over time in parallel with the change in the position of the point (P) of the object (C) where the reference line (Q) intersects, so that the user (U) can be provided with the interest of gradually changing the reference line (Q) while checking the display mode of the indication image (G). Note that "change in the display mode over time" means that the display mode does not change only in a binary manner at a specific point in time, but rather the display mode changes gradually over time. Note that it does not matter whether the display mode changes continuously or in stages.

[Aspect 9]
In a preferred example (Aspect 9) of any one of Aspects 1 to 8, the motion control unit (43B) controls the object (C) so that the line of sight of the object (C) tracks the reference line (Q). According to the above aspect, the user (U) can easily grasp the sensation of the user (U) influencing the object (C) (e.g., the sensation of contact). Note that "the line of sight of the object tracks the reference line" includes not only the case where the eyeball (e.g., the pupil) of the object (C) rotates to track the reference line (Q), but also the case where the head of the object (C) rotates according to the reference line (Q).

[Aspect 10]
An image display system (1A, 1B) according to a preferred embodiment (embodiment 10) of the present invention is an image display system (1A, 1B) comprising a head mounted display (30) and an information processing device (40), wherein the information processing device (40) comprises a display control unit (42B) that causes a display device (13) of the head mounted display (30) to display a stereoscopic image utilizing binocular parallax, the stereoscopic image being an image captured of a virtual space (V) by a virtual camera (E) whose direction is controlled according to the direction of the head mounted display (30), and the display control unit (42B) causes an instruction image (G) to be displayed in the direction of a reference line (Q) whose direction in the virtual space (V) changes according to the direction of the head mounted display (30), and changes the display mode of the instruction image (G) according to the positional relationship between the reference line (Q) and an object (C) in the virtual space (V). In the above configuration, the display mode of the instruction image (G) indicating the direction of the reference line (Q) in the virtual space (V) changes depending on the positional relationship between the reference line (Q) and the object (C), which has the advantage that the user (U) can easily grasp the positional relationship between the reference line (Q) and the object (C) in the virtual space (V). Note that the head mounted display (30) and the information processing device (40) may be either integrated or separate.

1A, 1B...image display system, 10...terminal device, 11...control device, 12...storage device, 13...display device, 14...detection device, 20...wearing device, 30...HMD, 40...information processing device, 41...posture analysis unit, 42A, 42B...display control unit, 43A, 43B...motion control unit, V...virtual space, E...virtual camera, C...character, R...imaging range, Q...reference line, G...pointed image, P...contact point.

Claims (6)

a display control unit that causes an image of a virtual space captured by a virtual camera whose direction is controlled according to a direction of a head mounted display to be displayed on the head mounted display; and
a program for causing a computer to function as an action control unit that causes an object to execute an action according to a position where a reference line, the direction of which in the virtual space changes according to a direction of the head mounted display, intersects with the object in the virtual space, the program comprising:
The display control unit displays a first pointing image in a direction of the reference line when the reference line does not intersect with the object, and displays a second pointing image having a display mode different from that of the first pointing image in the direction of the reference line when the reference line intersects with the object.
program. The program of claim 1 , wherein the second instruction image contacts a surface of the object in the virtual space. a display control unit that causes an image of a virtual space captured by a virtual camera whose direction is controlled according to a direction of a head mounted display to be displayed on the head mounted display; and
a program for causing a computer to function as an action control unit that causes an object to execute an action according to a position where a reference line, the direction of which in the virtual space changes according to a direction of the head mounted display, intersects with the object in the virtual space, the program comprising:
The action control unit causes the object to execute the action when a change in the direction of the head mounted display satisfies a predetermined condition.
program. The program of any one of claims 1 to 3, wherein the action control unit causes the object to perform a first action when the reference line intersects a first portion of the object, and causes the object to perform a second action different from the first action when the reference line intersects a second portion of the object that is different from the first portion. An image display system including a head mounted display and an information processing device,
The information processing device includes:
a display control unit that causes the head mounted display to display an image of a virtual space captured by a virtual camera whose direction is controlled according to a direction of the head mounted display;
and an action control unit that causes the object to perform an action according to a position where a reference line, the direction of which in the virtual space changes according to a direction of the head mounted display, intersects with the object in the virtual space ,
The display control unit displays a first pointing image in a direction of the reference line when the reference line does not intersect with the object, and displays a second pointing image having a display mode different from that of the first pointing image in the direction of the reference line when the reference line intersects with the object.
Image display system. An image display system including a head mounted display and an information processing device,
The information processing device includes:
a display control unit that causes the head mounted display to display an image of a virtual space captured by a virtual camera whose direction is controlled according to a direction of the head mounted display;
and an action control unit that causes the object to perform an action according to a position where a reference line, the direction of which in the virtual space changes according to a direction of the head mounted display, intersects with the object in the virtual space ,
The action control unit causes the object to execute the action when a change in the direction of the head mounted display satisfies a predetermined condition.
Image display system.

Images