JP3140708B2 - Hand shape input device and virtual reality generation device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a hand shape input device used for grasping and manipulating a virtual object in a virtual reality technology, and a virtual reality generation device using the same.

[0002]

2. Description of the Related Art In a virtual environment system using a large display, computer graphics (hereinafter, referred to as computer graphics) are used.
Called CG. ) Generates a virtual three-dimensional space, and the observer stereoscopically views the virtual space. Here, for such a stereoscopic display, for example, a configuration in which a binocular parallax image displayed in a time-division manner on a large screen screen is stereoscopically viewed through liquid crystal shutter glasses using a binocular stereoscopic display (binocular parallax method). Is used.

That is, the liquid crystal shutter glasses have a structure in which the liquid crystal shutter opens and closes alternately at predetermined time intervals for both eyes, and this opening and closing operation is synchronized with the display of the binocular parallax image displayed on the large screen screen. I have. For this reason, if the image on the large screen viewed from the right eye and the image on the large screen viewed from the left eye are different from each other by the parallax of the two, the observer may not be able to see the stereoscopic image. You will feel like that.

[0004] At that time, D-2-10 "Real-time hand shape recognition using a pipeline type image processing device" (Koichi Ishibuchi et al.)
As disclosed in Japanese Unexamined Patent Application Publication No. 2003-168, a special glove (for example, Cy) capable of capturing hand shape and section position data in an operation for assembling or deforming an object in a virtual environment.
It was possible to visually grasp and move a virtual object by using berGlove (registered trademark)) and input image processing of a hand image by a plurality of cameras.

Also, at the 11th Human Interface Symposium, 2121, p. 491-496 (19
95) As disclosed in "Haptic Interface for Object Manipulation Using Small Display" (Haruo Noma et al.), Using a haptic device, the sense of weight of a grasped virtual object can be sensed happily. Rather than combining a haptic device with a small display device that can be held with both hands, it may be presented, and after capturing the image of the virtual object in the display device, the virtual object in the virtual space in conjunction with the movement of the display device Moving was also done. At that time, when the virtual object collides with another object, the repulsive force at the time of the collision is reproduced by the haptic device.

[0006]

However, the conventional hand shape input device and the virtual reality generation device have the following problems.

[0007] First, when operating a virtual object displayed in three dimensions, if one's hand is positioned between the image display surface and the viewpoint, the field of view is obstructed. In addition, when a haptic device is used to present a sense of weight of a grasped virtual object, the device enters the entire field of view of the image display surface and obstructs the view.

[0008] Second, when the user's own hand or haptic device is present between the image display surface and the viewpoint, the mixture of the stereoscopic image and the real object (own hand or haptic device) is a fusion of the stereoscopic image. In the fusion region in the binocular stereoscopic display in some cases, making fusion difficult. This becomes remarkable when the distance between the display surface of the stereoscopic image and the real object is large.

FIG. 6 is a conceptual diagram for explaining binocular stereoscopic display when fusion is difficult as described above.
FIG. 7 is a conceptual diagram showing the line of sight of both eyes when an actual object is gripped by a hand as compared with FIG.

First, referring to FIG. 7, when a real object is gripped by a hand, a human perceives a three-dimensional effect of the target object due to parallax when the object existing at the position of the real hand is viewed from both eyes. be able to.

On the other hand, consider a case where the observer wears the special glove 10 as described above in his hand and looks at the stereoscopic image 30 projected on the video display surface. Here, it is assumed that the stereoscopic image 30 is configured to output an image reflecting parallax from both eyes in a time-division manner and observe the image through the liquid crystal shutter glasses 40.

FIG. 6 shows a case where the glove 10 is located between the right eye and the stereoscopic image 30. In this case, the stereoscopic image 30 can be seen through the left eye, whereas the stereoscopic image 30 cannot be seen through the right eye. It will not be fused and will not be perceived as a stereoscopic image.

Third, when a small display device (having a size large enough to be held by both hands) and a haptic device are combined, when a grasped virtual object is grasped, an object search / It is necessary to sequentially press the switch attached to the display device to switch the mode of gripping / moving / release of gripping,
Since the operation for realizing that the observer grasps the target object is not intuitive, there is a problem that it is insufficient to give the observer a virtual reality.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to solve the problem even when the observer's hand exists between the observer's viewpoint and the display device. The present invention provides a hand shape input device and a virtual reality generation device capable of giving an observer a three-dimensional effect of a target object without obstructing the field of view.

Another object of the present invention is to provide a hand-shaped input device and a virtual reality input device which make the operation of gripping a target object intuitive, and which can make the virtual reality given to the observer more realistic. It is to provide a generating device.

[0016]

The hand shape input device according to claim 1 is mounted on the observer's hand to give the observer a virtual reality, and the position of the observer's hand in the virtual space. and shape a hand shape input device for data you get the sensor for detecting the hand shape data is arranged in a predetermined joint portion, and gloves to be attached to the observer's hand, at a predetermined position of the glove 3 placed in the virtual section of the observer's hand
Position detecting means for detecting a three-dimensional position; position and inclination information in the virtual space of the hand shape input device obtained from three-dimensional position information of the observer's hand detected by the position detecting means and hand shape data from the sensor; , Based on information on the position of the eyes of the observer and the gaze direction given from the outside,
In the part blocked by gloves existing in the virtual space in the line of sight
And a control means for outputting the image information of the corresponding object, is fixed at a predetermined position of the glove surface, the output video from the control means
Based on the information, the image corresponding to the part blocked by the glove
And first display means for displaying.

According to a second aspect of the present invention, there is provided a virtual reality generating apparatus for giving a virtual reality to an observer, wherein the second apparatus outputs three-dimensional image information corresponding to a virtual space image. Outputting three-dimensional image information to the display means, the eye-gaze detecting means attached to both eyes of the observer, and detecting the eye position and the eye-gaze direction in the observer's virtual space, and the second display means Control means to be mounted on the observer's hand,
A hand shape input means for you get the data of the position and shape of the observer's hand in the virtual space, hand shape input means, a sensor for detecting the hand shape data is arranged in a predetermined joint portion, A glove to be worn on the observer's hand, and three gloves arranged in a predetermined position of the glove and within a virtual section of the observer's hand.
Position detecting means for detecting a three-dimensional position, and first display means fixed to a predetermined position on the surface of the glove and outputting an image given from the control means, wherein the control means comprises an observer detected by the position detecting means. The position and inclination information of the hand shape input device in the virtual space obtained from the three-dimensional position information of the hand and the hand shape data from the sensor, and the information of the eye position and the line of sight of the observer given from the line of sight detection means based on, among the three-dimensional image information to be output to the second display means, line-of-sight direction and the cross vital observation first display means
The three-dimensional image information of the area corresponding to the part blocked by the hand of the user is output to the first display means.

According to a third aspect of the present invention, there is provided a virtual reality generating apparatus according to the second aspect, wherein
Is a display device fixed to the upper part of the glove.

According to a fourth aspect of the present invention, in the configuration of the second aspect, the control means includes: a control unit configured to control the position information of the object in the virtual space and a hand from the hand shape detection unit; Based on the shape data and the three-dimensional position information of the hand, it is detected that the observer's hand has performed the gripping operation of the object existing in the virtual space.

[0020]

FIG. 1 is a schematic block diagram showing the configuration of a hand shape input device 10 and a virtual reality generation device 1000 using the same according to an embodiment of the present invention.

The virtual reality generating apparatus 1000 includes a computer 100 for controlling the operation of the virtual reality generating apparatus 1000,
The liquid crystal shutter glasses 40 worn by the observer to perceive a stereoscopic image, and the shutter operation of the liquid crystal shutter glasses 40 controlled by the computer 100 in synchronization with the switching of the stereoscopic image to images for the right and left eyes. A shutter controller 50, a hand shape input device 10 attached to the hand of the observer, and providing the computer 100 with information on the hand shape of the observer and information on the position of the hand in the virtual space;
Large-screen monitor 20 for displaying a stereoscopic image output from 0
And

The hand shape input device 10 includes three bending sensors arranged for each finger corresponding to each joint of each finger and a sensor for measuring an angle between the fingers. Special gloves 12 that output the data of
And a small display 14 fixed to the upper part of the special glove 12 and outputting a stereoscopic image from the computer 100, and a three-dimensional space position detector 16 for outputting position data in the virtual space of the hand to the computer 100.

The liquid crystal shutter glasses 40 are controlled by a shutter controller 50 to open and close the conduction of light corresponding to each eye, position data of the observer's viewpoint in virtual space, and the observer's line of sight. And a three-dimensional position detector 44 that outputs direction data to the computer 100.

FIG. 2 is an enlarged view showing the main part of the hand shape input device 10 in more detail. The operation of the virtual reality generation device 1000 will be described in more detail with reference to FIGS.

The computer 100 outputs a virtual space image that can be viewed by the observer to the large screen monitor 20 based on the data of the observer's viewpoint and line of sight from the shutter glasses 40.

Here, the virtual space image includes a time-divided binocular parallax image. On the other hand, the computer 100 gives the time-divided binocular parallax image to the shutter controller 50, and the shutter controller 50 controls the liquid crystal shutter glasses 40 in synchronization with switching of the binocular parallax image, and The liquid crystal shutter is opened and closed alternately at predetermined time intervals. Since this opening and closing operation is synchronized with the display of the binocular disparity image displayed on the large screen screen, the image on the large screen screen seen from the right eye differs from the image on the large screen screen seen from the left eye by the disparity between the two. Thus, the observer feels as if he / she is viewing a stereoscopic image.

The computer 100 further includes shutter glasses 40
As shown in FIG. 2, based on the observer's viewpoint and line-of-sight direction data given at regular time intervals, and the observer's hand position data similarly given from the hand shape input device 10 at regular time intervals. And a part of the image on the large screen monitor which can be viewed by the observer using the small display 14 as a window.

FIGS. 3 and 4 are conceptual diagrams showing a state in which an observer looks at a virtual object corresponding to the stereoscopic image 30 using the hand shape input device 10 as described above.

FIG. 3 shows the line of sight of the observer in the initial stage when the observer starts to grasp the target object based on the stereoscopic image 30 projected on the large screen monitor 20.
5 shows the observer's line of sight at the time when the grasping operation is completed.

First, referring to FIG. 3, an observer stereoscopically views stereoscopic image 30 on large screen monitor 20 through liquid crystal shutter glasses 40, and tries to grasp a virtual object corresponding to stereoscopic image 30. I do. At this time, the right hand is present between the observer's right eye and the stereoscopic image 30, and the left eye can see the stereoscopic image 30 on the large screen monitor 20,
For the right eye, the small display 14 in the hand shape input device 10
It is possible to see the stereoscopic image projected above. However, in actuality, as a function of the human eye, it does not occur that objects located at different distances are viewed with the same weight by the left and right eyes. Therefore, at this moment,
Image information from either the left or right eye is prioritized, and the state of stereoscopic vision is temporarily disrupted.

However, at the moment after the completion of the gripping operation, the eyes of the observer's eyes look at the image of the virtual object displayed on the small display 14 with reference to FIG. The person will again stereoscopically view the virtual object.

That is, in this state, when observing the stereoscopic image 30 through the shutter glasses 40, the observer sees the image of the virtual object while feeling as if his / her hand is partially transparent. This image can be viewed in three dimensions.

Therefore, when the small display 14 does not exist, if the user grasps a virtual object having a size such that it can enter the hand, the object disappears with his own hand, whereas the virtual reality generation device 1000 So, as if your hands were partially transparent,
The observer can stereoscopically view the virtual object in the hand.

In this case, for example, based on the data of the position of the hand in the virtual space and the data of the position of the target object in the virtual space, the computer 100 determines that the hand is at the position of the target object,
In addition, when it is determined from the hand shape data that the hand has gripped the target object, the stereoscopic image 30 corresponding to the virtual object is moved in accordance with the movement of the hand.

At this time, the virtual object in hand is displayed on the small display 14. FIG. 5 is a schematic diagram showing a relationship between a stereoscopic image of a virtual object projected on the large screen screen 20 in such a state and an observer.

Referring to FIG. 5, the virtual reality generating apparatus 1
000 performs the above-described operations as described with reference to FIG. 3, so that the observer has performed an operation of grasping the target object with his / her hand in an extremely intuitive manner. Can be grasped, and the virtual reality can be accepted without discomfort.

[0037]

As described above, according to the present invention,
Since a part of one's hand can be made transparent, the field of view is not obstructed even if one's hand is positioned between the image display surface for stereoscopic display and the viewpoint. Furthermore, since the distance between the display surface of the stereoscopic image and the real hand is short, even when the stereoscopic image and the real object coexist, it is possible to reduce the interference of fusion of the stereoscopic image. In addition, a special glove can be used as a mode switching switch, so that the operation for the observer can be made intuitive. That is, when grasping a grasped virtual object,
For example, if the hand is closed while the object is displayed on the small display on the back of the hand, the virtual reality generation device can detect that the object has been gripped. For this reason, it is possible to make the virtual reality sense for the observer closer to the real sense.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram illustrating a configuration of a virtual reality generation device 1000 of the present invention.

FIG. 2 is an enlarged view showing the configuration of the hand shape input device 10.

FIG. 3 is a first diagram illustrating the operation of the virtual reality generation device 1000;
FIG.

FIG. 4 is a second diagram illustrating the operation of the virtual reality generation device 1000;
FIG.

FIG. 5 is a schematic diagram illustrating a grasping operation of an observer in the virtual reality generation device 1000.

FIG. 6 is a conceptual diagram illustrating the operation of a conventional virtual reality generation device.

FIG. 7 is a conceptual diagram for explaining an operation in which an observer grasps a real object.

[Explanation of symbols]

 REFERENCE SIGNS LIST 10 hand shape input device 12 special gloves 14 small display 16 3D position detector 20 large screen monitor 30 stereoscopic image 40 liquid crystal shutter glasses 42 liquid crystal shutter 44 3D position detector 50 shutter controller 100 computer 1000 virtual reality generation device

Continuation of the front page (51) Int.Cl. ⁷ Identification symbol FI G06F 3/14 350 G06F 3/14 350A G06T 17/40 G06T 17/40 A (56) References JP-A-8-314621 (JP, A) JP-A-4-216118 (JP, A) JP-A-6-12181 (JP, A) JP-A-7-239750 (JP, A) JP-A-5-75834 (JP, U) (58) Fields investigated (Int.Cl. ⁷ , DB name) G06F 3/02-3/033 G06F 3/00 601-680 G06F 3/14 G06T 15/00-17/50

Claims (4)

(57) [Claims] To 1. A gives the virtual reality to the viewer, the viewer is mounted in the hand, the hand shape input for you get the data of the position and shape of the observer's hand in the virtual space An apparatus, wherein a sensor for detecting the hand shape data is arranged at a predetermined joint, a glove worn on the hand of the observer, and a glove arranged at a predetermined position of the glove, the hand of the observer. Position detecting means for detecting a three-dimensional position in the virtual section; and 3 of the hand of the observer detected by the position detecting means
Based on three-dimensional position information and the position and inclination information in the virtual space of the hand shape input device obtained from the hand shape data from the sensor, and information on the position of the observer's eyes and gaze direction given from outside. Corresponding to the part blocked by the glove existing in the virtual space in the line of sight direction .
Control means for outputting video information of an object to be fixed; and
Based on the output video information, the part blocked by the gloves
A hand shape input device, comprising: first display means for displaying an image corresponding to a minute . 2. A virtual reality generation device for giving a virtual reality to an observer, comprising: a second device for outputting three-dimensional image information corresponding to a virtual space image;
Display means mounted on both eyes of the observer and detecting a position and an eye direction of an eye in a virtual space of the observer; and a control means for outputting a dimension image information, is attached to the hand of the observer, and the hand shape input means for you get the data of the position and shape of the observer's hand in the virtual space, the A hand shape input unit, a sensor for detecting the hand shape data is arranged at a predetermined joint, a glove worn on the observer's hand, and a glove placed at a predetermined position on the glove; Position detecting means for detecting a three-dimensional position in the virtual section, and first display means fixed to a predetermined position on the surface of the glove and outputting an image given from the control means, wherein the control means comprises: By the position detecting means 3 of the issued the observer's hand
Position and inclination information in the virtual space of the hand shape input device obtained from three-dimensional position information and hand shape data from the sensor, and information on the position and gaze direction of the observer's eyes given from the gaze detection means. Of the three-dimensional image information output to the second display means based on
The first display means and cross vital the said viewing direction
A virtual reality generation device that outputs three-dimensional image information of a region corresponding to a portion blocked by a viewer's hand to the first display means. 3. The virtual reality generation device according to claim 2, wherein the first display means is a display device fixed to an upper part of the glove. 4. The method according to claim 1, wherein the control unit determines whether the observer's hand is virtual based on the position information of the object in the virtual space, the hand shape data from the hand shape detection unit, and the three-dimensional position information of the hand. The virtual reality generation device according to claim 2, wherein the virtual reality generation device detects that a grasping operation of an object existing in the space has been performed.