JP7271251B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing apparatus and information processing method used in a VR (Virtual Reality) system.

In a conventional VR system, an ideal state of a virtual object operated by a VR user is set, and if there is a large difference between the state of the virtual object and the ideal state, the display of the virtual object is corrected so as to approach the ideal state. , is displayed on a display device (Patent Document 1).

JP 2017-188041 A

However, in the VR system described in Patent Literature 1, when the display of the virtual object is corrected, the user viewing the virtual object may feel uncomfortable with the posture and movement of the virtual object.

The problem to be solved by the present invention is to provide an information processing apparatus and an information processing method that can reduce the unnaturalness that a user viewing a virtual object feels about correcting the posture of the virtual object. be.

An information processing apparatus and an information processing method according to the present invention distinguish a portion of a virtual object output by an output unit that does not match the posture of at least a part of a user's body from other portions. The above problem is solved by outputting.

According to the present invention, it is possible to recognize a portion of a virtual object whose orientation does not match. I don't feel ugly.

1 is a schematic diagram showing the configuration of an information processing apparatus and a VR system using the information processing apparatus according to an embodiment of the present invention; FIG. 2 is a flow chart showing a method of displaying a virtual object by the information processing apparatus shown in FIG. 1;

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, the information processing device 2 used in the VR system 1 includes a communication device 101 in a user real space 100, VR bases 102 and 108, an optical sensor 103, a controller 104, VR trackers 105 and 107, and VR goggles 106. have Further, the information processing device 2 of the VR system 1 has AR goggles 302 for the vehicle real space 300 , a vehicle controller 303 and a vehicle communication device 304 .

Here, it is assumed that the first user 109 is lying on a bed or a futon in the user real space 100 . A user real space 100 is provided with a pair of VR bases 102, 108 for generating VR motion ranges. Also, in the user real space 100, an optical sensor 103 for detecting the posture of the first user 109, a controller 104 for integrating information from various sensors and generating a virtual space 200, and a controller 104 running in a remote location. A communication device 101 is provided for communicating with a vehicle 10 as a mobile object. Optical sensor 103 is, for example, a camera.

Also, the first user 109 holds a pair of VR trackers 105 and 107 in each hand and wears VR goggles 106 on the head. The VR trackers 105 and 107 have acceleration sensors, gyro sensors, etc., and detect the posture and movement of the arms and hands of the first user 109 . The VR goggles 106 are also provided with an acceleration sensor, a gyro sensor, etc., and detect the orientation of the head of the first user 109 . Furthermore, the information of the virtual space 200 generated by the controller 104 is displayed on the VR goggles 106 so as to be visible to the first user 109 . The controller 104 also stores the orientation of the first user 109 detected by the optical sensor 103, the orientation of the arms and hands of the first user 109 detected by the VR trackers 105 and 107, and the orientation of the hands detected via the VR goggles 106. The posture of the head of the first user 109 is input. Furthermore, the VR bases 102 and 108 detect location information of the VR trackers 105 and 107 and the VR goggles 106 and input it to the controller 104 .
Here, the optical sensor 103, the VR bases 102 and 108, the VR trackers 105 and 107, and the VR goggles 106 constitute a posture detection unit. Also, the controller 104 constitutes an output unit. Also, the communication device 101 constitutes a transmission unit.

On the other hand, the vehicle real space 300 is a space inside the vehicle 10 traveling in a remote location distant from the user real space 100 . The vehicle 10 includes a vehicle-side communication device 304 that communicates with the communication device 101 in the user's real space 100 and a vehicle-side controller 303 . The vehicle controller 303 integrates the information of the first user 109 and the information of the virtual space 200 transmitted from the communication device 101 of the user real space 100 and superimposes them on the vehicle real space 300 . A second user 301 who is a driver who drives the vehicle 10 exists in the vehicle real space 300 and sits on the driver's seat 11 . In the vehicle real space 300, the front passenger seat 12 next to the driver seat 11 is unoccupied. Also, the second user 301 wears AR goggles 302 on the head. The AR goggles 302 receive information about the virtual space 200 generated by the controller 104 via the vehicle-side communication device 304 and the vehicle-side controller 303 . Thereby, the virtual space 200 is displayed so as to be visible to the second user 301 .
Note that the AR goggles 302 constitute a display unit. Further, the vehicle 10 constitutes a moving body.

A virtual space 200 generated by the controller 104 corresponds to a vehicle real space 300 . In the virtual space 200 , an avatar 201 of the second user 301 is sitting on the driver's seat 11 and an avatar 202 of the first user 109 is sitting on the front passenger seat 12 . Avatars 201 and 202 are virtual objects output by the controller 104 . Here, for the first user 109, the virtual space 200 is virtual reality (VR). That is, the first user 109 has an avatar 202 that is the virtual object of the first user 109 and an avatar 201 that is the virtual object of the second user 301 in the virtual space 200 corresponding to the vehicle real space 300 . It looks like there is On the other hand, for the second user 301, the virtual space 200 is augmented reality (AR). That is, for the second user 301, the virtual space 200 appears to be superimposed on the actual vehicle real space 300 and the avatar 202 of the first user 109, which does not actually exist, exists. The communication device 100 transmits information about the avatar 202 to the AR goggles 302 via the vehicle-side communication device 304 and the vehicle-side controller 303 .

In the user real space 100, the first user 109 is lying down. On the other hand, the posture of the avatar 202 in the virtual space 200 has been corrected by the controller 104 to the state of sitting on the front passenger seat 12 . Therefore, the actual posture of the first user 109 and the posture of the avatar 202 in the virtual space 200 are different from the neck down. Therefore, the body 202a, legs 202b, and feet 202c of the avatar 202 are displayed in a different color to indicate that their postures do not match, and can be distinguished from other parts. Since the arm 202d and hand 202e of the avatar 202 move in conjunction with the movement of the actual arm and hand of the first user 109 holding the VR trackers 105 and 107, the display mode is not changed. In addition, since the posture of the avatar 202 is adjusted with the head 202f as the reference position, the display mode of the head 202f of the avatar 202 is changed assuming that the posture and movement of the head 202f of the avatar 202 match those of the head of the first user 109. do not have.
Note that the legs of the avatar 202 refer to parts corresponding to parts of the human body from the pelvis to the ankles. Also, the feet of the avatar 202 refer to parts corresponding to parts of the human body from ankles to toes.
In addition, the display mode of the arms 202d, hands 202e, head 202f, legs 202b, legs 202c, and body 202a of the avatar 202 is changed, such as by changing the color of each part so that it can be distinguished from other parts. can do.

Next, a method of displaying the avatar 202 in the virtual space 200 on the AR goggles 302 of the second user 301 using the information processing device 2 will be described with reference to FIGS. 1 and 2. FIG.
First, in step S<b>1 , the first user 109 starts using the VR system 1 . Next, optical sensor 103 , VR bases 102 and 108 , VR trackers 105 and 107 and VR goggles 106 of information processing device 2 detect the posture of first user 109 . Then, in step S<b>2 , the controller 104 corrects the posture of the first user 109 so as to be suitable for the virtual space 200 and reflects it in the posture of the avatar 202 .

Next, in step S<b>3 , controller 104 determines whether or not the posture of first user 109 in user real space 100 matches the posture of avatar 202 in virtual space 200 . That is, when the first user 109 in the user real space 100 is sitting on a chair, the posture of the first user 109 is determined to match the posture of the avatar 202 sitting on the front passenger seat 12 in the virtual space 200. . In this case, the image of the avatar 202 is output without changing the display mode and displayed on the AR goggles 302 .

Here, in the real user space 100, three-dimensional coordinates are set with the head of the first user 109 as a reference. Similarly, in the virtual space 200, three-dimensional coordinates are set with the head 202f of the avatar 202 as a reference. The three-dimensional coordinates of the user's real space 100 and the three-dimensional coordinates of the virtual space 200 correspond to each other. The controller 104 determines whether or not the postures match, depending on how much the coordinate positions of the body parts of the avatar 202 deviate from the coordinate positions of the body parts of the first user 109 . Note that the posture of the first user 109 in the user's real space 100 and the posture of the avatar 202 in the virtual space 200 do not need to be perfect matches, and may be within a range where they can be recognized as approximately the same posture. Further, among the differences in posture between the first user 109 and the avatar 202, the allowable range in which the posture of the first user 109 and the posture of the avatar 202 are considered to match is It is set wider than the position. The movable part is a part of the human body that is easy to move, and includes arms, hands, and the head. Also, the trunk position includes the torso with little movement.

On the other hand, when it is determined that the posture of the first user 109 in the user real space 100 does not match the posture of the avatar 202 in the virtual space 200, the second user 301 asks the first user 109 to interact in the next step S4. It is determined whether or not the call has been made. For example, when it is detected that the second user 301 is gazing at and speaking to the avatar 202 of the first user 109 via various sensors (not shown) provided in the vehicle or the AR goggles 302, It is determined that the second user 301 is interacting with the first user 109 . Also, even if the second user 301 is not looking at the avatar 202 of the first user 109, if the name of the first user 109 is at the end of the utterance of the second user 301, the second user 301 It is assumed that the first user 109 is addressed. That is, it is determined that the second user 301 is interacting with the avatar 202 of the first user 109 by calling the avatar 202 . Furthermore, in the virtual space 200, when the body of the avatar 202 of the first user 109 and the body of the avatar 201 of the second user 301 are close to each other until they are within a certain distance, the avatars 202 and 201 are considered to be in contact. When the avatar 202 of the first user 109 and the avatar 201 of the second user 301 are considered to be in contact, it is determined that the second user 301 is interacting with the first user 109 . Then, while the second user 301 is interacting with the first user 109, the control moves to step S6, and the portion of the avatar 202 that does not match the actual posture of the body of the first user 109 is displayed. The aspect is changed and it is displayed in a different color than other parts. As a result, while the second user 301 is communicating with the first user 109, discomfort caused by the correction of the posture of the avatar 201 is reduced, so that the second user's 301 sense of immersion in augmented reality is enhanced. can be done.

If the second user 301 has not interacted with the first user 109, it is determined in step S5 whether or not the first user 109 can perform the intended action. When the first user 109 cannot perform the intended action, for example, even if the first user 109 extends the arm 202d in the virtual space 200 and tries to touch an arbitrary target object such as a switch, the user's real space 100 does not move. For example, it is too narrow to actually extend the arm sufficiently. In such a case, although the first user 109 in the user real space 100 is in a state of not being able to perform the intended action of “stretching his arm,” the avatar 201 in the virtual space 200 stretches his arm according to the operation of the first user 109. It is possible to take a posture in which 202d is extended. Specifically, when the first user 109 continues to gaze at the target object for a predetermined time or longer, the target object displayed on the VR goggles 106 blinks. When the first user 109 confirms that the target object is blinking and inputs "Yes" to the VR trackers 105 and 107, the avatar 202 in the virtual space 200 can extend the arm 202d and touch the target object. Even in such a case, the posture of first user 109 and the posture of avatar 201 do not match, so the portion that does not match the actual posture of first user 109, that is, the display mode of arm 202d of avatar 201 is changed. and displayed in a color different from that of other portions (step S6). On the other hand, when the second user 301 does not interact with the first user 109 and the first user 109 can perform the intended action, the display mode of the avatar 202 is not changed.

Next, in step S7, whether or not the second user 301 has completed the interaction with the first user 109, or whether the action of the avatar 201 operated while the first user 109 cannot perform the intended action has been completed. It is determined whether or not If the interaction by the second user 301 has not been completed, or the action of the avatar 201 operated in a state in which the first user 109 cannot perform the intended action has not been completed, control returns to step S6. On the other hand, when the interaction by the second user 301 is completed, or the action of the avatar 201 operated in a state in which the first user 109 cannot perform the intended action is completed, in step S8, the first user 109 moves to the VR system 1 It is determined whether or not the use of the Then, when the first user 109 finishes using the VR system 1, the control ends. On the other hand, if the first user 109 has not finished using the VR system 1, the control returns to step S4.

As described above, when the posture of the first user 109 and the posture of the avatar 202 in the virtual space 200 do not match, the controller 104 of the information processing device 2 according to this embodiment replaces the mismatched portion with another Output in a color different from that of the As a result, when the second user 301 in the interior space of the vehicle 10 sees the avatar 202 of the first user 109 through the AR goggles 302, he/she can grasp at a glance the portion of the posture mismatch between the first user 109 and the avatar 202. can do. Thereby, the second user 301 can enhance the sense of immersion in augmented reality without feeling that the correction of the posture of the avatar 202 is unnatural.

The information processing device 2 also includes a communication device 101 that transmits information on the avatar 202 so that the AR goggles 302 worn by the second user 301 can receive the information. As a result, the second user 301 at a remote location distant from the user's real space 100 can visually recognize the avatar 202 of the first user 109 .

Also, the second user 301 is traveling in the vehicle 10 and the first user 109 is in a place other than the vehicle 10 . This enables a VR experience in which the first user 109 at home drives together in the virtual space 200 while communicating with the avatar 201 of the second user 301 . Also, for the second user 301, it is possible to have an AR experience of driving together while communicating with the avatar 202 of the first user 109 who does not actually exist in the virtual space 200 corresponding to the vehicle real space 300. Become.

Further, assuming that the postures of the first user 109 and the avatar 202 do not match, the parts whose display mode is changed so as to be distinguishable from the other parts are the arms 202d, the hands 202e, the head 202f, and the legs. 202b, legs 202c and/or torso 202a. By associating the parts whose display mode is changed with each part of the human body in this manner, the display of the avatar 202 can be adapted to various movements of the first user 109 . Also, the second user 301 can more clearly grasp the corrected portion of the avatar 202 of the first user 109 .

In addition, among the differences in posture between the first user 109 and the avatar 202, the allowable range in which the posture of the first user 109 and the posture of the avatar 202 are considered to match is The movable part is set wider than the trunk position. In this way, the second user 301 can visually recognize the correction of the posture of the avatar 202 by appropriately changing the allowable range of posture matching according to the ease of movement of each part of the body, that is, the ease of displacement of the position. It is possible to further reduce the unnaturalness felt when doing so.

In this embodiment, when the posture of the first user 109 and the posture of the avatar 202 do not match, the controller 104 outputs the mismatched portion in a different color from the other portions. is not limited to That is, the controller 104 may display a portion where the posture of the first user 109 and the posture of the avatar 202 do not match with a different transmittance than other portions. Also, the controller 104 may highlight only the portion where the posture of the first user 109 and the posture of the avatar 202 match. Also, the display mode of the avatar 202 of the first user 109 displayed on the AR goggles 302 may be switched at regular intervals.

Also, in this embodiment, the second user 301 travels on the vehicle 10, but is not limited to this, and may travel on a mobile such as a train or an airplane. Further, the space where the second user 301 is present is not limited to the internal space of the moving body, and may be a concert hall, a movie theater, a theater, or the like.

Also, in this embodiment, the first user 109 in the user real space 100 is lying down, but this is not the only option. For example, when the first user 109 is in a standing posture, the legs 202b and feet 202c of the avatar 202 in the virtual space 200 are displayed in a manner distinguishable from other parts as the postures do not match. .

The vehicle 10 described above corresponds to a moving body according to the present invention. The communication device 101 described above corresponds to a transmission unit according to the present invention. The optical sensor 103, the VR bases 102 and 108, the VR trackers 105 and 107, and the VR goggles 106 correspond to the attitude detection section according to the present invention. The avatar 202 described above corresponds to the virtual object of the first user 109 according to the present invention. The AR goggles 302 described above correspond to the display unit according to the present invention.

2... Information processing device 10... Vehicle (moving body)
101...Communication device (transmitting unit)
102, 108 ... VR base (posture detection unit)
103... Optical sensor (posture detection unit)
104... Controller (output unit)
105, 107... VR tracker (posture detection unit)
106 ... VR goggles (posture detection unit)
109... First user 200... Virtual space 202... Avatar (first user's virtual object)
202a... body 202b... leg 202c... leg 202d... arm 202e... hand 202f... head 301... second user 302... AR goggles (display unit)

Claims (8)

a posture detection unit that detects the posture of the first user;
an output unit that corrects the posture of the first user detected by the posture detection unit according to the virtual space and outputs it as a virtual object corresponding to the first user;
The output unit
When the posture of at least part of the first user's body detected by the posture detection unit does not match the posture of the first user's virtual object,
outputting a portion of the virtual object of the first user that does not match the posture of at least a part of the body of the first user in a manner distinguishable from other portions;
Of the difference in posture between the first user and the first user's virtual object, the allowable range in which the posture of the first user and the posture of the first user's virtual object are considered to match is An information processing apparatus in which a movable portion of a first user's body is set wider than a trunk portion . a posture detection unit that detects the posture of the first user;
an output unit that corrects the posture of the first user detected by the posture detection unit according to the virtual space and outputs a virtual object corresponding to the first user;
a display unit that displays the first user's virtual object so that it can be viewed by a second user different from the first user;
a transmission unit that transmits information on the first user's virtual object to the display unit ;
The output unit
When the posture of at least part of the first user's body detected by the posture detection unit does not match the posture of the first user's virtual object,
outputting a portion of the virtual object of the first user that does not match the posture of at least a part of the body of the first user in a manner distinguishable from other portions;
The information processing apparatus , wherein the second user is a passenger of a mobile body, and the first user is a person outside the mobile body . a display unit that displays the first user's virtual object so that it can be viewed by a second user different from the first user;
The information processing apparatus according to claim 1 , further comprising a transmission section that transmits information on said first user's virtual object to said display section. 4. The information processing apparatus according to claim 3, wherein said second user is a passenger of a mobile body, and said first user is a person outside said mobile body. 5. The information processing apparatus according to claim 4, wherein said mobile body is a vehicle. Of the difference in posture between the first user and the first user's virtual object, the allowable range in which the posture of the first user and the posture of the first user's virtual object are considered to match is 6. The information processing apparatus according to any one of claims 2 to 5 , wherein the first user's body has a movable portion wider than a trunk portion. detecting the posture of the first user;
A step of correcting the posture of the first user according to the virtual space and outputting it as a virtual object corresponding to the first user;
In the step of outputting a virtual object corresponding to the first user, when the posture of at least part of the body of the first user and the posture of the virtual object of the first user do not match, outputting a portion of the virtual object that does not match the posture of at least part of the body of the first user in a manner distinguishable from other portions ;
Of the difference in posture between the first user and the first user's virtual object, the allowable range in which the posture of the first user and the posture of the first user's virtual object are considered to match is An information processing method in which a movable portion of a first user's body is set wider than a trunk portion . detecting the posture of the first user;
a step of correcting the posture of the first user according to the virtual space and outputting it as a virtual object corresponding to the first user;
visibly displaying the first user's virtual object to a second user different from the first user;
transmitting information of the first user's virtual object ;
In the step of outputting a virtual object corresponding to the first user, when the posture of at least part of the body of the first user and the posture of the virtual object of the first user do not match, outputting a portion of the virtual object that does not match the posture of at least part of the body of the first user in a manner distinguishable from other portions ;
The information processing method , wherein the second user is a passenger of a mobile body, and the first user is a person outside the mobile body .

Images