JP6905384B2 - Force presenter and virtual force presenter - Google Patents

Description

The present invention relates to a force sense presenter and a virtual force sense presenter that function as a user interface that interacts with a virtual space model and present a force sense.

With the development of virtual reality technology in recent years, technology that presents information to the sense of tactile force in addition to visual (video) and auditory (audio) is drawing attention. So far, various methods for presenting the sensation of virtually touching or touching an object have been proposed, and some of them have been put into practical use as user interfaces such as controllers for video game machines.

For example, a joystick controller can be used to configure a force sense presenter to present vibrations to the hands and fingers to make them feel a pseudo force, or an actuator such as a motor can be used to configure a force sense presenter to physically react. There is a method of presenting the hardness by presenting.

For example, as a conventional technique, a virtual force sense presenting device is configured by installing a base of a force sense presenter on a fixture such as a desk, and a force that generates vibration or a reaction force in the force sense presenter using this base as a substrate. There is a first method in which a sense presentation unit is provided and the surface of a wall or an object is virtually perceived by changing the vibration amount or reaction force amount of the force sense presentation unit (see, for example, Non-Patent Document 1). This first method has a long history and is characterized by being able to present a physically accurate reaction force to some extent. However, since the force sense presenter that realizes this first method requires furniture such as a desk as a base thereof, the movable range is limited, and the virtual force sense presenter including the force sense presenter is used. There is a problem that the scale becomes large as a whole.

Therefore, in recent vertical reality technology, as a simpler technique, the force sense presenter is fixed to the palm of the user such as wearing a glove on the hand, or the force sense presenter is fixed to the user's body or the like. Two methods are known (see, for example, Non-Patent Documents 2 and 3).

Furthermore, as a simpler technique, the main body of the force sense presenter is configured so that the user can freely grasp and release it without wearing gloves or the like, and the main body of the force sense presenter is configured. A force sense presenting unit composed of a switch or the like that can be pushed with a fingertip is provided, and a control unit that controls the force sense presentation unit to generate vibration or reaction force is provided separately from the main body of the force sense presenter. Therefore, the third method for configuring the virtual force sense presentation device has become mainstream (see, for example, Patent Document 1, Non-Patent Documents 4, 5 and 6). In this third method, the volume and mass of the housing constituting the main body of the force sense presenter are designed to be relatively large so that the main body side can be naturally gripped and supported by a hand or finger. Therefore, the foundation of the force that presents the sense of tactile force can be obtained.

In order to detect the position and posture when the body of the force presenter is gripped, the position when the body of the force presenter is gripped by imaging and analyzing with one or more motion capture cameras. A real-time three-dimensional motion measurement system that outputs coordinate / angle information of a camera or a posture is known (see, for example, Non-Patent Document 7).

Also known is a three-dimensional position measurement system that measures the three-dimensional position of each magnetic sensor mounted on a fingertip based on the magnetic field generated by the magnetic field generation unit and outputs the coordinate information of the position of each magnetic sensor. (See, for example, Non-Patent Document 8).

Japanese Unexamined Patent Publication No. 2016-900219

“Geomagic Touch Haptic Device,” 3D SYSTEMS, [online], [Searched April 3, 2017], Internet <URL: http://www.geomagic.com/ja/products/phantom-omni/overview> “CyberGrasp,” CyberGlove Systems Inc., [online], [Searched April 3, 2017], Internet <URL: http://www.cyberglovesystems.com/cybergrasp/> M.Hirose, K.Hirota, T.Ogi, H.Yano, N.Kakehi, M.Saito and M.Nakashige, “HapticGEAR: the development of a wearable force display system for immersive projection displays,” In Proceedings of the IEEE Virtual Reality 2001 Conference, pp.123-129, 2001. Iemuro, Minamizawa, Arai, Kawakami, Tate, "Tactile Interaction with Virtual Objects Using Portaple Pen Type Force Display," The 14th Annual Meeting of the Virtual Reality Society of Japan, 1A2-4, September 2009. “Oculus Touch,” Oculus VR, LLC, [online], [Searched April 3, 2017], Internet <URL: https://www.oculus.com/rift/> “VIVE,” HTC Corporation, [online], [Search on April 3, 2017], Internet <URL: https://www.vive.com/jp/> “Real-time 3D motion measurement system L1 Tracker,” Detect Co., Ltd., [online], [Search on April 3, 2017], Internet <URL: http://www.ditect.co.jp/products/L1tracker. php> "3D position measurement system POLHEMUS," 3D Co., Ltd., [online], [Search on April 3, 2017], Internet <URL: http://www.ddd.co.jp/polhemus/ >

As described above, the force sense presenter is fixed to the palm of the user such as wearing a glove on the hand, or the second method of fixing the force sense presenter to the user's body or the like, or wearing a glove or the like. A third method of configuring a force presenter and its virtual force presenter that the user can freely grab and release without having to do so, for example, when hitting a virtual wall or object. It is possible to present the existence of walls and objects by simulating the reaction force.

However, techniques that provide a user interface that interacts with the virtual space model to present forces that cannot physically penetrate the wall or object, or to prevent physical intrusion into the virtual wall. Not realized.

For example, in a controller such as a video game that applies virtual reality technology, when invading a virtual object or wall, the wall or object actually exists in the real space by presenting vibration to the user. It is a technique that makes you feel illusion. However, even if a wall or an object is expressed by the presentation of physical vibration or the pressure on the user's finger using an actuator or the like, if there is no foundation for generating the force, the force of the user's finger is completely applied. It was not possible to generate enough force to keep the fingers in place and completely prevent the invasion of the virtual wall.

On the other hand, as described above, if it is the first method of constructing a virtual force sense presenting device by installing a base of a force sense presenter on a fixture such as a desk and using the base for presenting force. By presenting more force than the user's fingers, it is possible to prevent intrusion into the virtual wall. However, since the virtual force sense presentation device becomes large-scale, it is not suitable for application in a video game controller or the like.

That is, the virtual force sense presenter is composed of a force sense presenter that can be grasped and a control unit that controls the force sense presenter so that the user can freely grasp and release the force sense presenter without wearing gloves or the like. At the same time, it presents a force or impact that cannot be physically penetrated to a virtual contact body such as a wall or an object in the virtual space model, or prevents physical intrusion into a virtual wall. A technique that presents a more accurate sense of force to the virtual space model is desired.

In view of the above problems, the present invention provides a grippable force sense presenter that presents a more accurate force sense to the virtual space model and a force sense presenter thereof as a user interface that interacts with the virtual space model. The purpose is to provide a virtual force sense presentation device to be controlled.

The force sense presenter of the present invention is a grippable force sense presenter that presents force sense as a user interface that interacts with a virtual space model, and by expanding and contracting from the grippable main body portion and the main body portion. It is provided with one or a plurality of force sense presenting units for presenting a force sense, and the force sense presenter is associated with a virtual object in a virtual space model including a virtual contact body, and the one or a plurality of force senses are associated with the virtual object. Each of the presenting portions restricts the movement or posture of the finger or palm that grips the main body portion in the real space and the finger to contact the virtual object with the virtual contact body so as to follow the virtual space model. Alternatively, each of them has a uniaxial or multi-axis actuator that expands and contracts linearly in one direction or both directions so as to be able to present a sense of force by applying an impact to the palm, and in the area of the virtual contacted body. On the other hand, the spatial position on the surface of the palm holding the force sensor is set to the real space position physically corresponding to the virtual contact body so that the force sense presentation perceives that the virtual object is inaccessible. The actuator for restricting movement or posture so as to hold it in place is configured to extend to a position where the palm cannot be grasped, which makes it impossible for the palm to enter the real space area corresponding to the virtual contact body. It is characterized by being done.

Further, in the force sense presenter of the present invention, the force point at the tip of each of the one or more force sense presenters always corresponds to the surface of the contact target in the virtual contacted body. It is characterized in that it is configured to present the force sense of contact of the virtual object with respect to the virtual contacted body by being driven and controlled so as to remain in a spatial position.

Further, in the force sense presenter of the present invention, the maximum extension amount of each of the one or a plurality of force sense presenters is individually changed and controlled at the same time, so that the virtual object is brought into contact with the virtual object. It is characterized in that it is configured to present the force sense of contact of the virtual object with respect to the virtual contacted body according to the position with respect to the body.

Further, the force sense presenter of the present invention is further provided with one or more tactile force sense presenters for perceiving the shape and hardness of the virtual object.

Further, in the force sense presenter of the present invention, a control means for controlling the force sense presenter so as to interact with the virtual space model is provided in the main body, and the control means displays the virtual space model on the display device. It is characterized in that it is configured to be interlocked with the presented virtual space model display control unit by short-range wireless communication.

Further, the virtual force sense presenting device of the present invention uses the force sense presenter of the present invention, a control means for controlling the force sense presenter so as to interact with the virtual space model, and the virtual space model as a display device. The virtual space model display control unit for presenting is provided, and the control means makes contact of the virtual object with the virtual contact body with respect to each of the one or a plurality of force sense presentation units in the virtual space. The control means includes means for controlling the movement or posture of the finger or palm that grips the main body in accordance with the model, or applying an impact to the finger or palm, and the control means is the movement restriction or the movement restriction or the palm. The force sense presenter is controlled by extending the actuator for restricting the posture to a position where the palm cannot enter the real space area corresponding to the virtual contacted body so that the palm cannot be grasped. The space position on the surface of the palm to be gripped is restricted in movement or posture so as to be physically held at the real space position corresponding to the virtual contact body, and the virtual object is restricted with respect to the area of the virtual contact body. It is characterized in that it is made to perceive that it is inaccessible by presenting force.

Further, the virtual force sense presenting device of the present invention uses the force sense presenter of the present invention, a control means for controlling the force sense presenter so as to interact with the virtual space model, and the virtual space model as a display device. A virtual space model display control unit for presenting is provided, and the control means makes contact of the virtual object with the virtual contacted body with respect to each of the one or a plurality of force sense presenting units in the virtual space. A means for controlling the movement or posture of the finger or palm that grips the main body so as to follow the model, or to give an impact to the finger or palm, and the presentation of one or more tactile force senses. Each of the parts is provided with a means for controlling the expansion and contraction in the contraction direction from the initial position and presenting a reaction force in order to perceive the shape and hardness of the virtual object, and the control means is said to move. The force sense is presented by controlling the actuator for restricting or posture restriction to extend to a position where the palm cannot enter the real space region corresponding to the virtual contacted body so that the palm cannot be grasped. The space position on the surface of the palm that grips the vessel is restricted in movement or posture so as to be physically held at the real space position corresponding to the virtual contact body, and the area of the virtual contact body is restricted. It is characterized in that the virtual object is made to perceive that it is inaccessible by force presentation .

Further, the virtual force sense presenting device of the present invention includes the force sense presenter of the present invention and a virtual space model display control unit that presents the virtual space model to the display device, and the control means is the one or the above. For each of the plurality of force sense presenting portions, the contact of the virtual object with the virtual contact body is restricted, or the movement of the finger or palm holding the main body portion is restricted in the real space so as to follow the virtual space model. A means for controlling the posture to be restricted or giving an impact to a finger or a palm is provided, and the control means is a real space region in which the palm corresponds to the virtual contact body with an actuator for restricting the movement or the posture. By controlling the extension to a position where it becomes impossible to grasp the force sense presenter, the spatial position of the surface of the palm that grips the force sense presenter physically corresponds to the virtual contact body. It is characterized in that the movement is restricted or the posture is restricted so as to keep it in the real space position, and the virtual object is perceived by the force sense presentation that the virtual object cannot enter the area of the virtual contact body. ..

Further, in the virtual force sense presenting device of the present invention, in the control means, the force point at the tip of each of the one or a plurality of force sense presenting portions is always the contact target in the virtual contacted body. It is characterized in that the drive is controlled so as to continue to stay in the real space position corresponding to the surface of the.

Further, in the virtual force sense presenting device of the present invention, the control means is characterized in that each maximum extension amount of each of the one or a plurality of force sense presentation units is individually changed and controlled at the same time.

According to the present invention, a virtual force sense is achieved by a force sense presenter that can be grasped and a control unit that controls the force sense presenter so that the user can freely grasp and release the force sense without wearing gloves or the like. While configuring the presentation device, a user interface that interacts with the virtual space model is realized so that the virtual object corresponding to the force sense presenter is restricted in movement or posture with respect to the virtual contact body in the virtual space model. can do.

For example, in a virtual reality environment, in a grippable force sense presenter that does not require a base that fits in one hand, a force that cannot physically invade a virtual contact object such as a wall or an object in the virtual space model. It is possible to illusionally control the position of a virtual contact body (wall, etc.) that can invade in real space, such as by presenting an impact or preventing physical intrusion into a virtual wall. It will be possible.

It is a figure which illustrates the schematic structure of the force sense presenter, the virtual force sense presenter, and the peripheral device of one Embodiment by this invention. (A) and (b) are diagrams showing an example of 3-axis position detection and an example of 6-axis position / posture detection according to the virtual force sense presentation device of one embodiment according to the present invention, respectively. It is a figure which illustrates the schematic structure of the force sense presenter of one Embodiment by this invention. It is a flowchart which shows the control example of the force sense presenter control part in the control unit in the virtual force sense presenter of one Embodiment by this invention. (A) and (b) are diagrams showing operation examples of the force sense presenter of one embodiment according to the present invention, respectively. (A) and (b) are diagrams showing the schematic configuration of the force sense presenter of the modified examples 1 and 2 according to the present invention, respectively. It is a figure which shows the operation example of the force sense presenter of the modification 2 which concerns on this invention from (A) visual | displacement by (B) visual | visual | view. (A) and (b) are diagrams showing the schematic configuration of the force sense presenter of the modified examples 3 and 4 according to the present invention, respectively. (A), (b), and (c) are diagrams showing application examples of the virtual force sense presentation device of one embodiment according to the present invention, respectively.

Hereinafter, the force sense presenter 2 and the virtual force sense presenter 1 according to the embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a schematic configuration of a force sense presenter 2 and a virtual force sense presenter 1 according to an embodiment of the present invention and peripheral devices thereof. FIG. 2 is a diagram illustrating a schematic configuration of a force sense presenter 2 according to an embodiment of the present invention.

[Virtual force presentation device]
First, as shown in FIG. 1, the virtual force sense presenting device 1 includes a force sense presenter 2 and a control unit 3. The control unit 3 displays a virtual space model (in this example, a video game of a two-dimensional maze) on the display device 5, and the force sense presenter 2 is associated with the virtual object Obj in the virtual space model. That is, the movement of the virtual object Obj in the virtual space model is associated with the movement of the force sense presenter 2 in the real space, and the virtual object Obj is configured as a virtual body of the force sense presenter 2.

Then, the virtual space model is provided with a virtual contact body W (wall in this example) capable of contacting the virtual object Obj, and the virtual object Obj corresponds to the virtual contact body W on the virtual space model. When touched, it should be prevented from physically invading the virtual wall or should be shocked. Therefore, it is desired that the force sense presenter 2 corresponding to the virtual object Obj in the virtual space model also accurately presents the force sense so as to follow the virtual space model, and the force sense presenter 2 and the virtual according to the present invention are desired. The force sense presentation device 1 realizes this as described below.

The force sensation presenter 2 is a device that is configured to be graspable with one hand H and presents the force sensation as a user interface that interacts with the virtual space model. The force sensation is expanded and contracted from the main body 2a and the main body 2a. The force sense presenting part 2b is provided, and the main body part 2a and the force sense presentation part 2b are covered with a stretchable flexible cover 2c. A specific configuration example and operation of the force sense presenter 2 will be described later.

The position and posture of the force sense presenter 2 are 6 of the real space position coordinates (x, y, z) of the force sense presenter 2 and the angle (θx, θy, θz) indicating the aerial posture of the force sense presenter 2. The axis coordinate / angle information (see FIG. 2A) is input to the control unit 3 in real time. In the example shown in FIG. 1, real-time three-dimensional motion is obtained by capturing images with one or more motion capture cameras 41 and outputting coordinate / angle information of the position and posture when the main body of the force sensor is grasped by the signal processing unit 42. The measurement system 4 (see, for example, Non-Patent Document 7) is used. In order to detect the position and attitude of the force presenter 2, the real space position coordinates (x, y, z) of the force presenter 2 and the angle (θx, θx,) indicating the aerial attitude of the force presenter 2 are shown. Any 6-axis sensor obtained as coordinate / angle information of θy, θz) can be used. Further, when it is sufficient to detect only the position information of the force sensor 2 due to the configuration of the virtual space model, the information of the real space position coordinates (x, y, z) (see FIG. 2B) is arbitrarily output. For example, a three-dimensional position measurement system using a magnetic sensor (see, for example, Non-Patent Document 8) or the like can be used. As the virtual space model, in the case of a two-dimensional maze video game as in this example, information on only the two axes of x and y is used.

The control unit 3 includes a force sense presenter control unit 31, an application execution unit 32, and a storage unit 33.

The force sense presenter control unit 31 is a function unit that controls the force sense presentation in the force sense presenter 2 corresponding to the virtual object Obj in the virtual space model, and is configured by dedicated hardware or central processing processing. The unit (CPU) can be configured as a computer that reads and executes a dedicated program stored in the storage unit 33.

The application execution unit 32 includes a virtual space model display control unit 321 that presents the virtual space model to the display device 5, and the central processing unit (CPU) reads and executes the dedicated application software stored in the storage unit 33. Can be configured as a computer. The storage unit 33 includes a recording medium such as a cassette or a card on which application software for presenting such a virtual space model to the display device 5 is recorded.

The force sense presenter control unit 31 includes a body position / posture detection unit 311, a relative position calculation unit 312, a force sense presentation control unit 313, and a force sense presentation drive unit 314.

In this example, the body position / posture detection unit 311 uses the real-time three-dimensional motion measurement system 4 to indicate the real space position coordinates (x, y, z) of the force sense presenter 2 to be grasped and the angle (θx, θx, aerial posture). The 6-axis coordinate / angle information of θy, θz) is acquired in real time, the position and orientation of the force sense presenter 2 are detected, and the actual position / attitude information indicating the position and attitude of the force sense presenter 2 is obtained. It is generated and output to the virtual space model display control unit 321 and the relative position calculation unit 312. When the x-axis direction is the right direction for the user and the y-axis direction is the forward direction for the user, the posture data are θx = pitch (Pitch), θy = roll (Roll), and θz = yaw (, respectively). It may be acquired as Yaw).

The virtual space model display control unit 321 displays and controls the posture and position of the virtual object Obj in the virtual space model based on the input physical position / posture information. Further, the virtual space model display control unit 321 includes a case where the virtual space model is displayed on the display device 5 so that the screen can be switched according to the movement of the virtual object Obj, and the display device 5 uses the position of the virtual object Obj as a reference. Virtual contact object information indicating the position of the virtual contact object W of the displayed virtual space model is generated and output to the relative position calculation unit 312.

The relative position calculation unit 312 bases the virtual object Obj in real time based on the actual position / attitude information obtained from the actual position / attitude detection unit 311 and the virtual contacted body information obtained from the virtual space model display control unit 321. Calculates the relative position (relative displacement) between the actual position according to the posture of and the position of the virtual contact body W based on the position of the virtual object Obj, and outputs the calculation result to the force sense presentation control unit 313. do.

When the calculation result of the relative position (relative displacement) acquired in real time from the relative position calculation unit 312 is a positive value, the force sense presentation control unit 313 does not contact the virtual object Obj and the virtual contact body W. The force sense is presented via the drive control of the force sense presentation drive unit 314 so that it is determined that the state is in the state and the force sense presentation part 2b is set to the initial position where the force sense presentation part 2b is not extended from the main body part 2a of the force sense presenter 2. The position of the force sense presentation unit 2b of the vessel 2 is controlled.

On the other hand, when the calculation result of the relative position (relative displacement) acquired in real time from the relative position calculation unit 312 by the force sense presentation control unit 313 is a value of zero or less than zero, the virtual object Obj and the virtual hit are applied. It is determined that the contact body W is in contact with the contact body W. Then, the force sense presentation control unit 313 shifts the force sense presentation unit 2b from the main body part 2a of the force sense presenter 2 toward a minus value according to the calculation result of the relative position (relative displacement) which is a value equal to or less than zero. It is extended and the position of the force sense presenting unit 2b of the force sense presenter 2 is controlled so that the actual position of the force sense presentation unit 2b holds the contact position through the drive control of the force sense presentation drive unit 314. ..

The force sense presentation drive unit 314 is a driver that generates a control signal for driving and controlling the extension amount of the force sense presentation unit 2b from the main body unit 2a in the force sense presenter 2, preferably the force sense presentation unit 2b. The expansion amount of the above is servo-controlled via the output signal (pulse signal) of the encoder built in the main body 2a.

[Force sense presenter]
As described above, the force sense presenter 2 is a device that is configured to be graspable with one hand H and presents force sense as a user interface that interacts with the virtual space model. The force sense presenter 2 has a main body portion 2a and a force sense presentation unit 2b that presents a force sense by expanding and contracting from the main body portion 2a, and the main body portion 2a and the force sense presentation unit 2b can be expanded and contracted. Is covered with a flexible cover 2c. That is, the force sense presenter 2 is mounted in a size that fits in the hand.

A specific configuration example of the force sense presenter 2 is shown in FIG. In the configuration illustrated in FIG. 3, a servomotor 21 with an encoder is installed in the main body 2a, and a pinion gear 22 provided on the motor shaft of the servomotor 21 meshes with a flat plate rod-shaped rack 23 having meshing teeth formed therein. It is designed to do. One end 23a of the rack 23 is formed with a retaining protrusion, and the other end 23b of the rack 23 projects outward from the opening 25 formed on the flat surface 24 of the main body 2a and is fixed to the flat surface 26 of the force sense presenting portion 2b. Will be done. Then, the position where the flat surface 24 of the main body 2a and the flat surface 26 of the force sense presenting unit 2b abut is set as the initial position, and the servomotor 21 is rotationally driven to a position where one end 23a of the rack 23 engages with the pinion gear 22. Therefore, the force sense presentation unit 2b can be extended from the main body unit 2a.

The servomotor 21 is driven and controlled by a control signal from the force sense presentation drive unit 314. Preferably, the servomotor 21 has a configuration in which the extension amount of the force sense presenting unit 2b is servo-controlled via the output signal (pulse signal) of the encoder built in the servomotor 21. However, it is not always necessary to perform servo control, and a stepping motor may be used. Further, in addition to forming an actuator by a motor such as the servo motor 21, a linear actuator or, for example, a multi-stage extension type actuator having a nested structure as disclosed in Patent Document 1 may be used. Further, the flexible cover 2c does not have to be provided.

[Control example of force sense presenter control unit]
FIG. 4 is a flowchart showing a control example of the force sense presenter control unit 31 in the control unit 3 in the virtual force sense presentation device 1 according to the present invention. First, the force sense presenter control unit 31 associates the actual position and posture of the force sense presenter 2 with the virtual object Obj of the virtual space model, and sets the force sense presenter 2b to the initial position (step S1).

Next, the force sense presenter control unit 31 is grasped by the body position / posture detection unit 311 from the real-time three-dimensional motion measurement system 4 in this example, and the real space position coordinates (x, y) of the force sense presenter 2. , Z) and the 6-axis coordinates / angle information of the angles (θx, θy, θz) indicating the aerial attitude are acquired in real time to detect the position and orientation of the force sense presenter 2 and to detect the position and attitude of the force sense presenter 2. Generate real position / posture information indicating the position and posture (step S2).

Next, the force sense presenter control unit 31 displays the virtual space model on the display device 5 by the relative position calculation unit 312 so that the screen can be switched from the virtual space model display control unit 321 according to the movement of the virtual object Obj. Including the case, the virtual contact body information indicating the position of the virtual contact body W of the virtual space model displayed on the display device 5 with reference to the position of the virtual object Obj is acquired (step S3).

Next, the force sense presenter control unit 31 receives the actual position / attitude information obtained from the actual position / attitude detection unit 311 and the virtual contact obtained from the virtual space model display control unit 321 by the relative position calculation unit 312. Based on the body information, the relative position (relative displacement) between the actual position according to the posture of the virtual object Obj and the position of the virtual contact body W based on the position of the virtual object Obj is calculated in real time (step). S4).

Next, the force sense presenter control unit 31 sets the virtual object Obj and the virtual hit based on the calculation result of the relative position (relative displacement) acquired in real time from the relative position calculation unit 312 by the force sense presentation control unit 313. It is determined whether the contact W is in contact or non-contact (step S5).

Here, in the force sense presentation control unit 313, when the calculation result of the relative position (relative displacement) acquired in real time from the relative position calculation unit 312 is a positive value, the virtual object Obj and the virtual contact body W are not. It is determined that the device is in contact (step S5: No), and the process proceeds to step S1 so that the force sense presenting unit 2b does not extend from the main body 2a of the force sense presenter 2.

On the other hand, when the calculation result of the relative position (relative displacement) acquired in real time from the relative position calculation unit 312 is zero or a value of zero or less, the force sense presentation control unit 313 becomes a virtual object Obj and a virtual hit. It is determined that the contact W is in contact (step S5: Yes). Then, the force sense presentation control unit 313 shifts the force sense presentation unit 2b from the main body part 2a of the force sense presenter 2 toward a minus value according to the calculation result of the relative position (relative displacement) which is a value equal to or less than zero. The position of the force sense presenting unit 2b in the force sense presenter 2 is controlled via the drive control of the force sense presentation drive unit 314 so that the force sense presentation unit 2b is extended and the actual position of the force sense presentation unit 2b holds the contact position. Step S6).

After that, when the process is continued until the end (step S7: Yes of ending the execution of the application), the process proceeds to step S2 (step S7: No).

With this configuration, it is determined in real time whether or not the virtual object Obj is in contact with the virtual contact body W (wall in this example) provided in the virtual space model, and the virtual contact body W is determined. When the virtual object Obj comes into contact with (in this example, the wall) (including the impact of the contact), the force sense presenter 2 corresponding to the virtual object Obj in the virtual space model is also a virtual object. The contact with the virtual contact object of the above is to restrict the movement or posture of the finger or palm holding the force sense presenter in the real space so as to follow the virtual space model, or to give an impact to the finger or palm. It becomes.

That is, the user's hand or finger cannot physically enter the real space area corresponding to the virtual contact body W (wall in this example) area (forced physical movement is a force sensation). The presenter 2 cannot be grasped). Hereinafter, this operation will be described in more detail.

5 (a) and 5 (b) are diagrams showing operation examples of the force sensor 2 according to the present invention, respectively. Various methods of gripping the force sensor 2 are allowed, and two of them are shown separately in FIGS. 5 (a) and 5 (b).

That is, in FIG. 5 (a) (FIGS. 5 (a-1) to (a-3)), the force sense presenter 2 is located at the force point P1 with 1 to 4 fingers (index finger, middle finger, ring finger, little finger). With respect to the virtual contacted body W (wall in this example) region while being gripped by the entire one hand H using the thumb located at the force point P2 facing these four fingers and the palm located at the force point dtp. A state in which the virtual object Obj (corresponding to the force sense presenter 2) is not invaded vertically (in the x-axis direction) (FIG. 5 (a-1)) is attempted to be invaded (FIG. 5 (a-2)). , (A-3)) is shown.

On the other hand, in FIG. 5 (b) (FIGS. 5 (b-1) to (b-3)), 1 to 4 fingers (index finger, middle finger, ring finger, little finger) in which the force sense presenter 2 is located at the force point P. The virtual object Obj (in the x-axis direction) perpendicular to the virtual contact body W (in this example, the wall) region while being held by one hand H with the thumb located at the force point dtp facing these four fingers. An operation example of a state in which (corresponding to the force sense presenter 2) is not invaded (FIG. 5 (b-1)) and is attempted to be invaded (FIGS. 5 (b-2) and (b-3)). Shown.

Representatively, the operation in the case of the example shown in FIG. 5A will be described in detail.

In the state shown in FIG. 5A-1, the user holds the force sense presenter 2 with one hand H and sets the force sense presenter 2b to an initial position where the force sense presenter 2b does not extend from the main body 2a of the force sense presenter 2. The virtual object Obj (corresponding to the force sense presenter 2) does not invade the virtual contact body W (wall in this example) region. At this time, as the value of the relative position (relative displacement) between the actual position according to the posture of the virtual object Obj and the position of the virtual contacted body W with reference to the position of the virtual object Obj, the force sense presenter 2 It is assumed that the relative displacement between the center of gravity point d0 and the force point dtp at the tip of the force sense presentation unit 2b + Da.

Subsequently, as shown in FIG. 5A-2, the virtual object Obj (force sense presenter 2) is perpendicular to the virtual contact body W (wall in this example) region (in the x-axis direction). When trying to invade (corresponding), as the value of the relative position (relative displacement) between the actual position according to the posture of the virtual object Obj and the position of the virtual contact body W based on the position of the virtual object Obj. , -Db (| Db |> | Da |), and the force point dtp, which is the tip of the force sense presentation unit 2b, is driven and controlled so as to always stay at the real space position corresponding to the surface of the virtual contact object W. .. At this time, the combined force of the force applied to the force point P1 of the four fingers and the force applied to the force point P2 of the thumb is in a state of being balanced with the force applied to the palm of the force point dtp.

Further, as shown in FIG. 5A-3, it corresponds to the virtual object Obj (force sense presenter 2) perpendicularly (x-axis direction) to the virtual contact body W (wall in this example) region. The value of the relative position (relative displacement) between the actual position according to the posture of the virtual object Obj and the position of the virtual contacted body W based on the position of the virtual object Obj, even if the virtual object Obj is further intruded. As a result, -Dc (| Dc |> | Dc |) is obtained, and the force point dtp, which is the tip of the force sense presentation unit 2b, is driven and controlled so as to always stay at the real space position corresponding to the surface of the virtual contact object W. NS. Also at this time, the combined force of the force applied to the force point P1 of the four fingers and the force applied to the force point P2 of the thumb is in a state of being balanced with the force applied to the palm of the force point dtp.

By the way, in the conventional technique, it is possible to perceive whether or not the virtual object Obj is in contact with the virtual contact body W (in this example, the wall) region, but the virtual contact body W (in this example) In this example, it was not possible to perceive the virtual object Obj as inaccessible to the wall) region by force presentation.

On the other hand, in the force sense presenter 2 and the virtual force sense presenter 1 according to the present invention, the spatial position (position of the force point dtp) on the surface of the palm holding the force sense presenter 2 is physically applied virtually. It is kept in the real space position corresponding to the contact body W (in this example, the wall). That is, in the force sense presenter 2, the force sense presenter 2b extends from the main body 2a as long as the palm keeps trying to invade the real space region corresponding to the virtual contact body W (in this example, the wall), and the force is applied. The total length of the sensation presenter 2 itself continues to grow.

Therefore, in the force sense presenter 2 and the virtual force sense presenter 1 according to the present invention, as long as the user continues to hold the force sense presenter 2, the palm corresponds to the virtual contact body W (wall in this example). It becomes impossible to invade the real space area. Conversely, in order for the palm to enter the real space area corresponding to the virtual contact body W (in this example, the wall), the user has to part with the force sense presenter 2. In the illustrated example, a part such as a finger other than the palm invades the real space area corresponding to the virtual contact body W (in this example, the wall), but the user can use the tip of the finger and the palm or the palm. Which boundary, such as the center position of the device, is perceived as a wall can be arbitrarily adapted to the method of superimposing or presenting video information and audio information.

The operation of the example shown in FIG. 5 (b) is the same as that of the example shown in FIG. 5 (a), and the force point dtp at the tip of the force sense presenting unit 2b is always the surface of the virtual contact body W. The drive is controlled so as to continue to stay in the real space position corresponding to. At this time, the force applied to the force point P of the four fingers and the force applied to the force point dtp of the thumb are in a balanced state. Therefore, in the force sense presenter 2 and the virtual force sense presenter 1 according to the present invention, as long as the user continues to hold the force sense presenter 2, the entire thumb is on the virtual contact body W (in this example, the wall). It becomes impossible to invade the corresponding real space area.

The maximum displacement amount of the force sense presentation unit 2b does not have to be the physical maximum length of the force sense presentation unit 2b, and exceeds the maximum width that can be grasped by an individual user or the majority of users. It may be set, and for example, an extension amount of about 10 cm is sufficient.

[Modifications 1 and 2 of the force sense presenter]
In the configuration example of the force sense presenter 2 shown in FIG. 3 and the like described above, an example in which the force sense presentation unit 2b that expands and contracts from the main body 2a is configured as one has been described, but the force sense presentation unit that expands and contracts from the main body 2a has been described. It is also possible to configure 2b as a plurality.

For example, as in the modified example 1 shown in FIG. 6A, it is possible to have a configuration having two force sense presenting portions 2b-1, 2b-2 that expand and contract in both directions in one axis from the main body portion 2a. The control unit 3 can be configured to individually change the maximum extension amount of each of the force sense presenting units 2b-1 and 2b-2 and control them at the same time. Further, as in the modified example 2 shown in FIG. 6A, four force sense presenting portions 2b-1, 2b-2, 2b-3, 2b- that expand and contract in both directions in two axes orthogonal to each other from the main body portion 2a. It can be configured to have 4. Hereinafter, when they are not individually distinguished and described, they are also collectively referred to as "force sense presentation unit 2b". Therefore, the control unit 3 individually changes the maximum extension amount of each of the one or the plurality of force sense presenting units 2b and controls them at the same time, so that the virtual object can be controlled according to the position of the virtual object with respect to the virtual contact body. It can be configured to present the force of contact of the virtual object with respect to the virtual contact body.

Further, the number of axes may be increased to have a configuration having five or more force sense presentation units. In both of the modified examples 1 and 2, the force sensation presenter 2 can be mounted in a size that fits in the hand. By increasing the number and number of force sense presentation units in the force sense presenter 2 and changing the maximum extension amount of each force sense presentation unit, more diverse force sense presentation is possible.

For example, FIG. 7 shows an operation example of the force sense presenter 2 of the modification 2 shown in FIG. 6 (b). The expansion and contraction directions of the four force sense presenters 2b-1, 2, b-2, 2b-3, and 2b-4 in the force sense presenter 2 of the modified example 2 have two degrees of freedom on the x and y axes, and are in the center of FIG. As shown in (A), the force sense presenter 2 of the modified example 2 is grasped with one hand and surrounded on all sides by the real space position corresponding to the surface of the virtual contact body W (for example, a wall). It shows the control when it is.

Based on the operating principle described with reference to FIG. 5, the force sense presenter 2 of the modified example 2 shown in FIG. 6 (b) in the state of (A) vision shown in the center of FIG. ) The control unit 3 can be configured to simultaneously control the amount of extension of the four force sense presenting units 2b-1, 2b-2, 2b-3, and 2b-4 according to the movement positions in the eight directions of vision. ..

For example, at the moving positions of (B) and (H) vision, in the force sense presenter 2 of the modified example 2, the control unit 3 has the maximum extension amount of each of the force sense presentation units 2b-1, 2, b-3, and 2b-4. Are individually changed and controlled at the same time. At the moving positions of (C) and (G) vision, the control unit 3 individually changes the maximum extension amount of the force sense presenting units 2b-3 and 2b-4 in the force sense presenter 2 of the modification 2. Control at the same time. At the moving positions of (D) and (F) vision, the control unit 3 individually sets the maximum extension amount of the force sense presenting unit 2b-2, 2b-3, 2b-4 in the force sense presenter 2 of the modification 2. And control at the same time. At the moving positions of (E) and (I) vision, in the force sense presenter 2 of the modified example 2, the control unit 3 individually changes the maximum extension amount of the force sense presentation units 2b-1, 1 and 2b-2. Control at the same time.

That is, the force point at the tip of any of the force sense presenting portions 2b-1, 2b-2, 2b-3, and 2b-4 is always in all directions at any of the moving positions in the eight directions from B to I. The drive is controlled so as to continue to stay at the real space position corresponding to the surface of any of the contact objects W of the virtual contact body W. Therefore, as long as the user continues to hold the force sense presenter 2 of the modified example 2, a part of the finger or the palm does not invade the real space region corresponding to the virtual contact body W (in this example, the wall). It becomes impossible.

In this way, by increasing the number and number of force sense presenting portions in the force sense presenter 2 and changing the maximum extension amount of each force sense presenting portion, control is performed not only in the two-dimensional space but also in the three-dimensional space. The control unit 3 can be expanded. When controlling in a three-dimensional space, it is desirable to hold the hand so that it covers all directions comprehensively.

[Modifications 3 and 4 of the force sense presenter]
In the configuration example of the force sensation presenter 2 shown in FIG. 3 and the like described above, an example in which the force sensation presenting unit 2b that expands and contracts from the main body 2a is configured as one is described. It is also possible to add a tactile force sense presentation unit 2d for perceiving the shape and hardness of the object. For example, as in the modified example 3 shown in FIG. 8 (a), the main body portion 2a has one force sense presentation unit 2b, and the tactile force sense presentation for causing the thumb to perceive the shape and hardness of the virtual object. Part 2d-1 and one or more tactile force sense presentation parts 2d (example shown) for making one or four fingers (index finger, middle finger, medicine finger, small finger) perceive the shape and hardness of a virtual object. Then, it is possible to have a configuration having two tactile force sense presentation units 2d-2, 2d-3).

Further, as in the modified example 4 shown in FIG. 8B, the main body portion 2a has a plurality of force sense presentation units 2d (in the illustrated example, two force sense presentation units 2b-1, 2b-2). The shape and hardness of the virtual object are applied to the tactile force sense presentation unit 2d-1 for allowing the thumb to perceive the shape and hardness of the virtual object, and to 1 to 4 fingers (index finger, middle finger, medicine finger, small finger). It can be configured to have one or more tactile force sense presentation units 2d (two tactile force sense presentation units 2d-2, 2d-3 in the illustrated example) for perceiving. Hereinafter, when they are not individually distinguished and described, they are also collectively referred to as "tactile force sense presentation unit 2d".

Such a tactile force sense presentation unit 2d is expanded and contracted by controlling the contraction amount and its reaction force, unlike the force sense presentation unit 2b that expands and contracts by controlling the extension amount. By doing so, it is possible to distinguish and perceive the force sense to be presented. The structure of the tactile force sense presentation unit 2d can be an actuator having a rack and pinion structure as shown in FIG. 3, and in addition to this, a linear actuator or a nesting as disclosed in Patent Document 1, for example, can be used. It may be a multi-stage extension type actuator having a structure.

When the tactile force sense presenting unit 2d is provided, it is preferable to operate it in a control cycle by a feedback loop of about 1 kHz in order to perceive the shape and hardness of the virtual object. Further, when the force sense presenting unit 2b wants to express the hardness of the virtual contact body W (wall in this example), it can be expressed by a control cycle by a feedback loop of about 1 kHz, and the control cycle is particularly changed. The hardness of the wall may be controlled by this.

As described above, the force sense presenter 2 and the virtual force sense presenter 1 according to the present invention are the force sense presenter 2 when the user grips the force sense presenter 2 so as to wrap it with one or both hands. Taking advantage of the fact that there is a certain limit on the degree of opening of the hand or finger that can hold the hand or finger according to the size of the user's hand, the force sense presentation unit 2b reaches the limit of the size at which the hand can be opened. By controlling the width or the overall size of the force sense presenter 2 by expansion and contraction in real time, the real space area where the fingers holding the force sense presenter 2 can enter is controlled in association with the virtual space model. ..

As a result, in the virtual reality environment as described above, even when a device that does not require a base that fits in one hand is used, the device can invade in the real space. ) Area can be controlled and reproduced illusionally.

[Application example]
9 (a), 9 (b), and 9 (c) are diagrams showing application examples of the virtual force sense presentation device 1 of the embodiment according to the present invention, respectively.

When the force sense presenter 2 and the virtual force sense presenter 1 are applied to contents such as virtual reality and games, for example, as shown in FIG. 9A, a bat is used in a baseball game as a virtual space model. The virtual object Obj corresponding to the force sense presenter 2 can be used, and the ball can be used as the virtual contact body W. In this case, the impact when the bat collides with the ball can be physically reproduced. On the contrary, the ball may be a virtual object Obj corresponding to the force sense presenter 2, and the bat may be a virtual contact body W. In this case, the impact when the ball collides with the bat or the ground can be physically reproduced.

Further, as shown in FIG. 9B, in a three-dimensional maze or dungeon as a virtual space model, a moving object such as a ball is set as a virtual object Obj corresponding to the force sense presenter 2, and walls, floors, and ceilings are set. It can be a virtual contact body W. In this way, it is possible to present a sense of force that makes it impossible to invade not only the walls but also the bowls and floors that exist three-dimensionally above and below. At this time, when passing through a tunnel or a hole that can be physically passed through, the force sense is not presented, so that the user can freely proceed only in this direction. Become.

As shown in FIG. 9C, in a story game in which virtual reality goggles (also referred to as a head-mounted display) are used as a display device 5 to confront a virtual character as a virtual space model. For example, an item such as a sword held in a hand can be a virtual object Obj corresponding to the force sense presenter 2, and an animal body such as a monster or a static object such as a rock can be a virtual contact body W.

In this way, the virtual force sense presenter 1 makes the force sense presenter 2 corresponding to the virtual object Obj, and the force sense presenter 2 so as to make the contact of the virtual object with the virtual contact body according to the virtual space model. The finger or palm to be gripped is restricted in movement or posture in real space, or an impact is applied to the finger or palm.

The present invention is not limited to the examples of the above-described embodiments, but is limited only by the description of the scope of claims. For example, in the example of the above-described embodiment, the example in which the force sense presenter control unit 31 is provided in the control unit 2 configured separately from the force sense presenter 2 has been described, but the force sense presenter control unit has been described. The force sense presenter 2 is provided inside the force sense presenter 2 to be battery-powered, and the force sense presenter control unit 31 can be interlocked with the virtual space model display control unit 321 by short-range wireless communication. As a result, the force sense presenter 2 with further improved usability can be configured.

Further, in the example of the above-described embodiment, the virtual contact body information will be described as indicating the position of the virtual contact body W of the virtual space model, but simply, the virtual object Obj and the virtual contact body W will be described. It may be configured as a flag indicating whether or not the body W is in contact with the virtual space model. That is, while the virtual object Obj and the virtual contact body W indicate that they are in contact with each other on the virtual space model, the force sense presentation unit 2b in the force sense presenter 2 performs the force sense presentation, and the flag is set. By controlling the force sense presenting unit 2b in the force sense presenter 2 to be set to the initial position when the non-contact is shown, the same actions and effects as those of the above-described embodiment can be obtained.

According to the present invention, a virtual force sense is achieved by a force sense presenter that can be grasped and a control unit that controls the force sense presenter so that the user can freely grasp and release the force sense without wearing gloves or the like. While configuring the presentation device, a user interface that interacts with the virtual space model is realized so that the virtual object corresponding to the force sense presenter is restricted in movement or posture with respect to the virtual contact object in the virtual space model. It is useful for virtual reality applications because it can be used.

1 Virtual force presenter 2 Force presenter 2a Main body 2b, 2b-1, 2b-2, 2b-3, 2b-4 Force presentation 2c Flexible cover 2d, 2d-1,2d-2, 2d-3 Tactile force sense presentation 3 Control unit 4 Real-time 3D motion measurement system 5 Display device 21 Servo motor 22 Pinion gear 23 Rack 23a One end of rack 23b The other end of rack 23 24 Flat surface of the main body 25 Opening 26 Force sense Flat surface of the presentation unit 31 Force sense presenter control unit 32 Application execution unit 33 Storage unit 41 Motion capture camera 42 Signal processing unit 311 Actual position / posture detection unit 312 Relative position calculation unit 313 Force sense presentation control unit 314 Force sense presentation drive Unit 321 Virtual space model display control unit

Claims (10)

A graspable force sense presenter that presents force sense as a user interface that interacts with the virtual space model.
With a grippable body
It is provided with one or a plurality of force sense presenting parts that present a force sense by expanding and contracting from the main body part.
The force sense presenter is associated with a virtual object in a virtual space model including a virtual contact body, and is associated with the virtual object.
Each of the one or a plurality of force sense presenting units makes the contact of the virtual object with the virtual contacted body, and the finger or palm holding the main body portion in the real space so as to follow the virtual space model. movement is restricted or attitude restriction, and to allow presenting the force sense by imparting an impact to the finger or the palm, each having a Akuchuta uniaxial or multiaxial stretch linearly to one-way or two-way, The spatial position of the surface of the palm holding the force sense presenter is physically determined by the force sense presentation so that the area of the virtual contacted body is perceived by the force sense presentation that the virtual object cannot enter. The actuator for restricting the movement or posture so as to keep it at the real space position corresponding to the abutting body cannot be grasped so that the palm cannot enter the real space area corresponding to the virtual abutting body. A force sensation presenter characterized in that it is configured to extend to a position where it becomes. The force point at the tip of each of the one or more force sense presenting portions is driven and controlled so as to always stay at the real space position corresponding to the surface of the contact target in the virtual contacted body. The force sense presenter according to claim 1, wherein the force sense of contact of the virtual object with respect to the virtual contact body is presented. The maximum extension amount of each of the one or a plurality of force sense presenting portions is individually changed and controlled at the same time, so that the virtual object can be moved according to the position of the virtual object with respect to the virtual contact body. The force sense presenter according to claim 1 or 2, wherein the force sense of contact with the virtual contacted body is presented. The force sense presenter according to any one of claims 1 to 3, further comprising one or more tactile force sense presenters for perceiving the shape and hardness of a virtual object. A control means for controlling the force sense presenter so as to interact with the virtual space model is provided in the main body, and the control means is close to the virtual space model display control unit that presents the virtual space model to the display device. The force sense presenter according to any one of claims 1 to 4, wherein the force sense presenter is configured to be interlocked and controlled by wireless communication. The force sensation presenter according to any one of claims 1 to 3 and
A control means for controlling the force sense presenter to interact with the virtual space model,
It is provided with a virtual space model display control unit that presents the virtual space model to the display device.
The control means has a finger or a finger that grips the main body portion so as to make the contact of the virtual object with the virtual contacted body with respect to each of the one or a plurality of force sense presenting portions according to the virtual space model. Provided with means for controlling the movement or posture of the palm in real space, or giving an impact to the fingers or the palm.
The control means controls the actuator for restricting the movement or posture to extend to a position where the palm cannot be grasped so as to prevent the palm from entering the real space region corresponding to the virtual contact body. As a result, the spatial position on the surface of the palm that grips the force sense presenter is restricted in movement or posture so as to be physically held at the real space position corresponding to the virtual contact body, and the virtual contact is restricted. A virtual force sense presentation device characterized in that the virtual object is made to perceive that the virtual object cannot enter the contact area by force sense presentation. The force sense presenter according to claim 4 and
A control means for controlling the force sense presenter to interact with the virtual space model,
It is provided with a virtual space model display control unit that presents the virtual space model to the display device.
The control means has a finger or a finger that grips the main body portion so as to make the contact of the virtual object with the virtual contacted body with respect to each of the one or a plurality of force sense presenting portions according to the virtual space model. A means for controlling the movement or posture of the palm in real space, or for giving an impact to the fingers or the palm.
A means for controlling each of the one or more tactile force sense presentation units to expand and contract in the contraction direction from the initial position and to present a reaction force in order to perceive the shape and hardness of the virtual object.
Equipped with a,
The control means controls the actuator for restricting the movement or posture to extend to a position where the palm cannot be grasped so as to prevent the palm from entering the real space region corresponding to the virtual contact body. As a result, the spatial position on the surface of the palm that grips the force sense presenter is restricted in movement or posture so as to be physically held at the real space position corresponding to the virtual contact body, and the virtual contact is restricted. A virtual force sense presentation device characterized in that the virtual object is made to perceive that the virtual object cannot enter the contact area by force sense presentation. The force sense presenter according to claim 5 and
It is provided with a virtual space model display control unit that presents the virtual space model to the display device.
The control means has a finger or a finger that grips the main body portion so as to make the contact of the virtual object with the virtual contacted body with respect to each of the one or a plurality of force sense presenting portions according to the virtual space model. Provided with means for controlling the movement or posture of the palm in real space, or giving an impact to the fingers or the palm.
The control means controls the actuator for restricting the movement or posture to extend to a position where the palm cannot be grasped so as to prevent the palm from entering the real space region corresponding to the virtual contact body. As a result, the spatial position on the surface of the palm that grips the force sense presenter is restricted in movement or posture so as to be physically held at the real space position corresponding to the virtual contact body, and the virtual contact is restricted. A virtual force sense presentation device characterized in that the virtual object is made to perceive that the virtual object cannot enter the contact area by force sense presentation. The control means ensures that the force point at the tip of each of the one or more force sense presenting portions always stays at the real space position corresponding to the surface of the contact target in the virtual contacted body. The virtual force sense presenting device according to any one of claims 6 to 8, further comprising drive control. The control means according to any one of claims 6 to 9, wherein each maximum extension amount of each of the one or a plurality of force sense presenting units is individually changed and controlled at the same time. Virtual force presentation device.

Images