JP5188103B2 - Haptic presentation device and haptic presentation method - Google Patents

Description

  The present invention relates to a force sense presentation device and a force sense presentation method for presenting a virtual space to a user using a force sense.

  In order to apply VR technology (Virtual reality technologies) to the daily living environment, it is necessary to develop an interface that does not restrain the user, and one of them is a haptic presentation method using the pressure of fluid such as air. Proposed. In this force sense presentation method, for example, a large number of ejection ports (ejection nozzles) that eject air are arranged in a two-dimensional plane in a matrix. Then, air is selectively ejected from these ejection nozzles, and this ejected air is received by the wind receiver 82 held by the user so that the user can perceive the surface of the three-dimensional object. (For example, refer nonpatent literature 1.).

Yuriko Suzuki et al., “Untethered Force Display by Wind Pressure: Representation of Three-Dimensional Objects”, IEICE Technical Report [Image Engineering], pp. 11-29. 71-76, July 17, 2003

  By the way, in the force sense presentation method, in order for the receptor to perceive the surface of the three-dimensional object with high accuracy, it is necessary to accurately apply the blown air to the receptor. For this purpose, it is necessary to appropriately select the ejection nozzle corresponding to the position of the receptor, but the receptor is arbitrarily moved above the ejection plane by the user's operation, and the selected ejection nozzle ejects air. Since there is a time lag until it is done, it is difficult to always apply the blown air to the receptor accurately.

  On the other hand, a method of ejecting fluid from all the ejection nozzles is also conceivable in order to always apply the ejection air to the receptor accurately. However, if this is done, the total amount of air jetted per unit time becomes enormous, and a compressor commensurate with the total amount of jetting must be prepared, leading to an increase in system size and cost.

  The present invention has been made by paying attention to the above circumstances, and the object of the present invention is that even if there is a time delay from the detection of the position of the receptor to the actual ejection of the fluid from the ejection port, It is an object of the present invention to provide a force sense presentation device and a force sense presentation method capable of always and accurately applying a jetted fluid to a receptor without causing an increase in size.

In order to achieve the above object, one aspect of the present invention is directed to an ejection unit in which a plurality of ejection ports each ejecting a fluid are arranged with regularity, and to face an arrangement surface of the plurality of ejection ports. Receiving means for detecting the state of movement of the receptor that receives fluid ejected from the plurality of ejection ports, and an ejection control unit that selectively controls ejection of fluid from the plurality of ejection ports; A force sense presentation device comprising: a virtual object linked to a movement state of the receptor based on a detection result of the movement state of the receptor by the receptor detection unit by the ejection control unit . Calculate the change in state. Then, at least one of the calculated state change of the virtual object, the state of the other virtual object expected to contact the virtual object and the change thereof, and the state of the receptor by the receptor detection means Based on the detection result of the movement state, the position of the receptor is predicted after an ejection delay time set in advance from the detection time of the movement state, and the prediction result of the position of the receptor and the plurality of outlets A jet outlet from which the fluid is to be jetted is selected from the plurality of jet outlets based on the arrangement position, and the jet unit is controlled to jet the fluid from the selected jet outlet.

Therefore, even if there is a time delay from the detection of the position of the receptor until the fluid is actually ejected from the ejection port, the position of the receptor after the ejection delay time has elapsed is predicted. Based on the position prediction result and the arrangement positions of the plurality of jet outlets, the jet outlet to which the fluid is to be jetted is selected. For this reason, it becomes possible to always apply the fluid to the receptor accurately without ejecting the fluid from all of the ejection ports, thereby preventing an increase in size and cost of the system.
In addition, when there are virtual objects that are linked to the state of movement of the receptor, or when there are other virtual objects that are expected to come into contact with the virtual object, the change in the state of these virtual objects is used as a prediction parameter. In addition, it is possible to predict the position of the receptor and the amount of fluid ejection.

One aspect of the present invention is characterized by further including the following components.
The first component includes a time required for detecting the movement state of the receptor in the ejection control unit and a period from when the movement state of the receptor is detected until the ejection command is output to the ejection unit. At least one of the required time, the time required for the ejection unit to eject the fluid after the ejection command is output, and the time necessary for the fluid ejected from the ejection port to reach the receptor Is set as the delay parameter, and the value of the ejection delay time is set based on the delay parameter and the interval time until the next movement state detection time of the receptor.
Therefore, it is possible to predict the position of the receptor after an appropriate time considering both the delay parameter and the interval time.

The second component detects the processing load when the prediction unit, the selection unit, and the ejection control unit execute processing by the ejection control unit when setting the value of the ejection delay time. Then, in accordance with the detected processing load, the delay parameter value representing the time required from when the moving state of the receptor is detected until the ejection command is output to the ejection unit is variably set. is there.
In this way, even if the predicted calculation processing time changes depending on the state of the processing load, an appropriate ejection delay time can always be set accordingly.
The third component predicts the position of the receptor after a preset ejection delay time from the time when the movement state of the receptor is detected, and predicts the movement path of the receptor to the predicted position. A plurality of outlets to which fluid should be ejected from a plurality of outlets based on the prediction result of the position and the prediction result of the movement path to the predicted position and the arrangement positions of the plurality of outlets stored in advance To do.
Therefore, for example, fluid is ejected from a plurality of jet outlets including a jet outlet around the predicted position and jet outlets on the movement path of the receptor. As a result, it is possible to absorb the prediction error and increase the probability that the fluid hits the receptor.

  In short, according to the present invention, even if there is a time lag from when the position of the receptor is detected to when the fluid is actually ejected from the ejection port, the ejection air can be supplied to the receptor without increasing the size of the system. It is possible to provide a force sense presentation device and a force sense presentation method that can always be accurately applied.

Embodiments according to the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a schematic configuration diagram of a force sense presentation device according to the first embodiment of the present invention. This device comprises a jet unit 1. In the ejection unit 1, a horizontal table is placed on a support base 2, and a plurality of ejection ports (ejection nozzles) 3, 3,. As shown in FIG. 2, the arrangement configuration of the jet outlets 3, 3,... Is set in a matrix so that the positional relationship between adjacent ones forms a square.

  Each of the jet outlets 3, 3,... Is connected to the compressor 5 through a tube. The compressor 5 sends compressed air through the tube to each of the jet outlets 3, 3,. Further, an electromagnetic valve unit 4 is provided between the compressor 5 and each of the ejection ports 3, 3,. This solenoid valve unit 4 has a plurality of solenoid valves provided in a one-to-one correspondence with the respective jet outlets 3, 3,. These solenoid valves individually open and close in response to an open / close control signal generated from the control unit 6, thereby selectively sending the compressed air sent from the compressor 5 to the respective jet outlets 3, 3. Supply and squirt.

  A receiver 8 is disposed in the space above the ejection unit 1. The receiver 8 has a hook-shaped receiver 82 attached to a tip end of a handle 81 gripped by a user, and the receiver 82 plays a role of receiving air ejected from one of the jet outlets 3. . FIG. 2 shows this state, and the user perceives the ejection of the air A received by the receiver 82 while moving the receiver 82 in the horizontal direction through the handle 81, and assumes it on the ejection unit 1. The three-dimensional object to be recognized can be recognized by force sense. The outer dimension of the receiver 82 is set to be equal to or slightly larger than the arrangement interval of the jet outlets 3, 3..., So that the receiver 82 can be located at any position above the jet unit 1. However, the air A ejected from at least one of the ejection ports 3, 3,... Can be received.

  A camera 7 is disposed above the ejection unit 1. The camera 7 is for detecting the horizontal position of the receiver 82, picks up the receiver 82 from above and sends the image signal to the control unit 6. At this time, a marker M is displayed at the center of the upper surface of the receiver 82 as shown in FIG. The marker M is used by the control unit 6 to detect the position of the receiver 82 from the image signal.

By the way, the control unit 6 is composed of a personal computer, for example, and is configured as follows. FIG. 4 is a block diagram showing the functional configuration.
That is, the control unit 6 includes a central processing unit (CPU) 11, a program memory 13 and a data memory 14 are connected to the CPU 11 via a bus 12, and a camera interface (camera I / F) 15. The external interface (external I / F) 16 and the input / output interface (input / output I / F) 17 are connected to each other.

  First of all, the camera I / F 15 takes in an image signal from the camera 7 under the control of the CPU 11. The external I / F 16 outputs an open / close control signal to the solenoid valve unit 4 under the control of the CPU 11. An input device 18 and a display device 19 are connected to the input / output I / F 17. The input device includes a keyboard and a mouse, for example. The display device 19 includes a liquid crystal display. The input / output I / F 17 takes the operation information of the input device 18 as input information and passes it to the CPU 11. At the same time, display data is supplied to the display device 19 for display under the control of the CPU 11.

The data memory 14 uses a memory such as a hard disk that can be written and read as needed as a storage medium, and includes a storage area 141 for spout position information and a storage area 142 for receptor change information. The spout position information includes coordinate data representing the positions of the spouts 3, 3,... In the spout unit 1 and the correspondence between the spouts 3, 3,. Table data to represent. The receptor change information represents a change history of the movement state of the receptor 82 from the past to the present, and includes, for example, information indicating a change in the position of the receptor 82 during movement and a change in velocity or acceleration. The data memory 14 is also used to temporarily store image data captured via the camera I / F 15 and information representing the prediction result of the position of the receptor 82.

The program memory 13 stores a receptor position detection program 131, a receptor position prediction program 132a, a spout selection program 133, and a spout control program 134.
The receptor position detection program 131 causes the camera I / F 15 to receive the image data picked up by the camera 7 at a constant period, and temporarily stores the received image data in the work area in the data memory 14. Then, the CPU 11 is caused to execute processing for detecting the position of the marker M displayed at the center of the upper surface of the receiver 82 by performing image pattern recognition processing or the like on the image data.

  The receptor position prediction program 132 a is based on the current position of the receptor 82 detected by the receptor position detection program 131 and information indicating the past movement state of the receptor 82 stored in the data memory 14. Then, the CPU 11 is caused to execute a process for predicting the position of the receiver 82 after a preset ejection delay time.

  The spout selection program 133 includes the predicted position data of the receiver 82 obtained by the receiver position prediction program 132a and the arrangement position data of the spouts 3, 3,... Stored in the data memory 14. In addition, the CPU 11 is caused to execute a process of selecting at least one ejection port from which air should be ejected after the ejection delay time.

  Based on the selection result of the jet outlet selection program 133, the jet control program 134 creates a correspondence table of the jets and the solenoid valves stored in the data memory 14 for the solenoid valves corresponding to the selected jet outlet. The CPU 11 is caused to execute a process of selecting based on and outputting an open / close control signal for opening the selected solenoid valve from the external I / F 16 to the solenoid valve unit 4.

Next, a force sense presentation operation by the force sense presentation device configured as described above will be described in accordance with a control procedure of the control unit 6. FIG. 5 is a flowchart showing a control procedure and control contents by the control unit 6.
In the camera 7, an operation for capturing an image on the horizontal table including the receiver 82 from the vertically upper position of the ejection unit 1 is always performed, and the image data is input to the camera I / F 15.

  When it is time to detect the position of the receiver 82, the control unit 6 first captures the image data output from the camera 7 via the camera I / F 15 in step S41. The captured image data is temporarily stored in the data memory 14, and then image pattern recognition processing is performed to detect the position coordinates of the marker M displayed on the receiver 82 on the horizontal table. At this time, since the marker M is displayed at the center of the receiver 82, the two-dimensional coordinates of the center position of the receiver 82 can be detected by the above detection process.

  Next, in step S42, the control unit 6 determines the coordinate data of the detected current position of the receiver 82 and information indicating the past movement history of the receiver 82 stored in the data memory 14, that is, the position. Position coordinates (predicted position data) that the receiver 82 is expected to reach after a preset ejection delay time are calculated based on the change trajectory (movement direction) and the change trajectory of velocity or acceleration. Subsequently, the control unit 6 proceeds to step S43, and based on the calculated predicted position data of the receiver 82 and the arrangement position data of each of the outlets 3, 3,... Stored in the data memory 14. In addition, at least one ejection port from which air is to be ejected after the ejection delay time is selected.

The position predicting process of the receiver 82 and the jet selecting process according to the predicted position described above will be described more specifically.
Prior to the start of control, the control unit 6 is set with a region or position where force sense needs to be presented to the receptor 82 held by the user. This region and position may change in the time axis direction. Further, an ejection delay time determined in consideration of the detection cycle or interval of the position of the receiver 82 is also set. This ejection delay time is set, for example, after 100 msec.

Examples of predicting and calculating the position of the receptor 82 include assuming that the receptor 82 travels at or near uniform speed after detection of the position, and assumes that the receptor 82 travels at or near equal acceleration. There are cases where both are combined. When a prediction equation for calculating the X-coordinate values on the assumption that advances at a constant speed mentioned as embodiments, X-coordinate value detected in the time t X_ _t, X-coordinate value detected in the past t-a time from time t X_ _t against _-a, X-coordinate value X_ _{t + b} at t + b time to be predicted,
_{X_ t + b = X_ t +} (X_ t -X_ t-a) * (b / a) * C (C is a constant)
Can be calculated as

As a specific example, it is assumed that the detected speed of the receptor 82 is always constantly changing, but it is assumed that the speed of the receptor 82 detected at the most recent time will continue thereafter, and the actual speed from the time of detection is assumed. calculating delay until ejection occurs includes a coordinate value X_ _t of the detection time point, a predicted position on the assumption that the coordinate value X_ value close to a constant multiple of _t-1 detected by the detection time of the one before There is a case. In this case, always holds the two values of position X_ _t-1 at time of detecting the position X_ _t and its previous that time is detected at the time the predicted value of the two values constant multiple to calculate.
_{X_ t + b = X_ t +} (X_ t -X_ t-1) * b * C (C is a constant).

However, since actually the speed is constantly changing, which past depends whether choosing time X_ _t-a, by preparing a plurality of time points X_ _t-a, a plurality of predicted position is calculated. In addition, since the time delay from the time of detection to the actual occurrence of ejection varies depending on various conditions, a plurality of C values may be prepared and a plurality of predicted positions may be calculated. Similarly, other coordinates are also predicted and air is ejected from one or more corresponding ejection openings so that the ejection air strikes the receiver 82 assumed to be at that position.

In addition, considering the position of the receptor 82 at the time of detection, the predicted position assumes that the receptor 82 exists within a certain distance range in consideration of the case where the movement proceeds as it is and the direction in which the movement is suddenly changed. It is also conceivable to calculate the value. An example of a calculation formula for calculating the X coordinate value in this case is as follows:
_{X_ t + b = X_ t ±} b * C (C is a constant)
It becomes.

  In this case, a plurality of C values may be prepared and a plurality of predicted positions may be calculated. As a result, a plurality of outlets around the position of the receiver 82 at the time when the receiver 82 is detected are set as ejection targets. Further, when the jet outlets are arranged at a constant interval D, the constant C is set to be a constant multiple of D so that the jet outlet corresponding to the position of the receiver 82 at the time when the receiver 82 is detected and If it sets so that the jet nozzle located in the both sides may be selected unconditionally, the selection process of a jet nozzle can be simplified.

It is also possible to combine the above two methods and calculate the predicted value assuming that the receptor 82 exists within a certain distance range centered on the position where the receptor 82 is predicted to move at a constant speed. It is done. Examples of formulas are:
_{X_ t + b = X_ t +} (X_ t -X_ t-a) * (b / a) * C1 ± b * C2 (C1, C2 are constants)
It becomes.

  According to each method described above, air is simultaneously ejected from a plurality of peripheral outlets including the outlet corresponding to the predicted position of the receiver 82 within a range where the load on the compressor 5 does not become excessively large. However, there is no worry that this will cause confusion in the sense of force received by the user. This is because, in addition to jetting from the corresponding jet outlet so that the jetted air hits the position of the receiver 82 where the prediction is hit, even if air is jetted from the jet outlet located where the receiver 82 does not exist This is because it is possible to prevent the haptic presentations from being disturbed by these simultaneously ejected airs, and there is no fear of adversely affecting the haptic presentation to the user.

  In addition, when the jet direction of the air from the jet outlets 3, 3,... Can be changed to other than vertically upward, or when a plurality of jet outlet arrays in which a plurality of jet outlets are arranged are arranged orthogonally, a plurality of receiving holes are further received. When using the bodies 82, 82,..., Simultaneous ejection from the plurality of ejection ports 3, 3,. For example, with respect to an ejection from a jet outlet that is controlled so that the blown air hits the position of the receiver 82 that is out of prediction, the receiver 82 does not exist at the position where the prediction is out of order, but it is received in the ejection direction. This is a case where the container 82 is present and receives force due to the blown air.

  As a specific example, when the receptor 82 exists in a certain ejection direction and the receptor 82 exists in the predicted position closer to the ejection port, force sense presentation is not necessary, and it exists in the farther predicted position. The case is when force sense presentation is required. When there are two or more predicted positions of the receiver 82 in the ejection direction from the ejection port calculated by the prediction, the position and orientation of the receptor 82 or the ejection ports 3, 3,. Based on the arrangement, it is conceivable to control by changing the jet port so that the jets from the jet port do not interfere with each other.

  In addition, when a plurality of receptacles 82 are used and another receptacle 82 is in the ejection direction from the ejection port calculated by prediction, or when it is close to the ejection port, the ejection to the other receptacle 82 is not hit. Therefore, based on the position and orientation of the receiver 82 and the arrangement of the outlets 3, 3,..., The outlet is changed and controlled so that each other's outlet air is ejected from the outlet that does not interfere with force sense presentation. It is possible to do.

  Further, not all the values of the coordinate axes are predicted and calculated for the position coordinate values of the receiver 82 in the coordinate space indicating the position or orientation of the receptors 82, but only the values of some of the coordinate axes are predicted and calculated. A method of predicting the position of the receiver 82 by using position data at the time when the value of the remaining coordinate axes is detected is also considered. For example, the prediction calculation is performed only corresponding to the movement of the receiver 82 in the direction parallel to the horizontal table on which the jet nozzle is arranged, and the prediction calculation is performed for the movement of the receiver in the direction perpendicular to the horizontal table. This is a method that is not performed.

  As a specific example, there is a case where the jet outlets 3, 3,... Arranged on the horizontal table surface are configured to be able to eject air only in one direction such as vertically upward. For example, the jets 3, 3,... As shown in FIG. 2 are arranged in a plane upward, and the receiver 82 moves over the plane where the jets 3, 3,. , 3,... Are arranged radially outside the spherical surface.

  In the configuration shown in FIG. 2, when a force sense by ejection is required, a calculation method for predicting the position of the receptor 82 is parallel to the plane of the coordinate values of the receptor 82. The predicted calculation of only the coordinate values is performed, and the predicted position of the receiver 82 is determined by using the value at the detection time as it is for the coordinate values in the ejection direction (direction perpendicular to the plane). In this case, if there is only one receiver 82, two or more predicted positions exist in the ejection direction by performing no prediction calculation in the direction perpendicular to the arrangement surface of the ejection ports 3, 3,. There is no. Therefore, even if air is simultaneously ejected from a plurality of ejection openings corresponding to a plurality of predicted positions, the force presentation is not disturbed.

  As described above, when the prediction calculation process of the position of the receiver 82 and the selection process of the jet nozzle based on the result are completed, the control unit 6 proceeds to step S44, and the selected jet nozzle or the jet group is the force. It is determined whether or not it is a control target for haptic presentation. As a result of this determination, if the selected spout or spout group is a force sense control target, the solenoid valve corresponding to the selected spout or spout group is stored in the data memory 14. The selection is made based on the correspondence table between the outlet and the solenoid valve.

  The opening / closing control signal for opening the selected solenoid valve is determined based on the jet delay in consideration of the time delay from when the position of the receiver 82 is detected until air is actually jetted from the jet port 3. Output from the external I / F 16 to the solenoid valve unit 4 at a timing before a certain time before the time elapses. As a result, the selected electromagnetic valve is opened for a certain period, and the compressed air of the compressor 5 is supplied to the selected outlet through the electromagnetic valve and is ejected from the outlet toward the receiver 82. .

Thus, the user is presented with a sense of force via the receptor 82. It should be noted that a three-dimensional object can be expressed on a two-dimensional plane in which the jet outlets 3, 3,... Are arranged by variably setting the air jet pressure for each jet outlet.
The above control operation is repeatedly executed until the user performs an end operation and this operation is detected in step S45.

  As described above, in the first embodiment, based on the current position of the receptor 82 detected at a certain point in time and information indicating the past movement state of the receptor 82 stored in the data memory 14, The position of the receiver 82 after a preset ejection delay time is predicted. Then, based on the predicted position data and the arrangement position data of each of the outlets 3, 3,... Stored in the data memory 14, at least one jet outlet to which air is to be jetted after the jet delay time is selected. Then, an electromagnetic valve corresponding to the selected outlet is opened to eject air.

  Therefore, even if there is a time delay from when the position of the receiver 82 is detected to when the air is actually ejected from the ejection port 3, the position of the receptor 82 after the ejection delay time has elapsed is predicted. Based on the prediction result of the position of the receiver 82 and the arrangement position data of the plurality of jets 3, 3,... For this reason, it becomes possible to always apply air accurately to the receiver 82 without blowing air from all the jet outlets 3, 3,..., Thereby preventing an increase in size and cost of the system. it can.

  Further, the position of the receiver 82 is predicted, the movement path of the receiver 82 to the predicted position is predicted, the prediction result of the position of the receiver 82 and the prediction result of the movement path to the predicted position, Based on the stored arrangement position data of the plurality of nozzles, a plurality of nozzles from which a plurality of nozzles 3, 3,. If it does in this way, gas will be jetted from the several jet nozzle including the jet nozzle of the circumference | surroundings centering on an estimated position, and the jet nozzle on the movement path | route of a receptor, for example. As a result, it is possible to absorb the prediction error and increase the probability that the fluid hits the receptor.

(Second Embodiment)
In the second embodiment of the present invention, the change in the state of the virtual object linked to the receiver 82 is calculated based on the detection result of the movement state of the receiver 82. Then, the calculated state change of the virtual object linked to the receiver 82, the state of the other virtual object expected to contact the virtual object and the change thereof are added to the prediction parameter, and The position and the amount of ejection after the ejection delay time are predicted.

FIG. 5 is a block diagram showing a functional configuration of a control unit used in the force sense presentation device according to the second embodiment of the present invention. In the figure, the same parts as those in FIG.
In the program memory 13, a virtual object state calculation program 135 is newly stored in addition to the receptor position detection program 131, the receptor position prediction program 132 b, the jet outlet selection program 133, and the jet control program 134.

  The virtual object state calculation program 135 calculates the state of the virtual object in the virtual space, which changes corresponding to the movement state (for example, position and orientation) of the receptor 82 detected by the execution of the receptor position detection program 131. At the same time, the CPU 11 is caused to execute processing for displaying the virtual space including the virtual object on the display device 19 having a role as a virtual space display device based on the calculation result. For example, a virtual object corresponding to the position of the acceptor 82 is prepared in the virtual space, and the attributes of the virtual object such as the position or orientation of the corresponding virtual object in the virtual space are linked with the position or orientation of the acceptor 82. Change.

  The receptor position prediction program 132b is based on the position or orientation data of the receptor 82 detected by the execution of the receptor position detection program 131, and the position of the virtual object that changes in conjunction with the receptor 82 or Based on the shape or their change information, and further based on the position or shape of other virtual objects expected to contact the virtual object or their change information, it is determined in advance from the position detection time of the receiver 82. The CPU 11 is caused to execute processing for predicting and calculating one or more positions and orientations of the receiver 82 after a certain period of time.

  The spout selection program 133 is based on the calculated predicted position data and the arrangement positions of the spouts 3, 3,... Stored in the data memory 14, and one or more along the route to the predicted position and the predicted position. The CPU 11 is caused to execute a process of selecting the jet nozzle.

  The ejection control program 133 further performs a process for controlling the amount of air ejected from the ejection port using the detected position or orientation of the receiver 82 or the state of the virtual object, to the CPU 11. Let it run. Note that the air ejection amount can be controlled by varying the opening amount of the solenoid valve of the solenoid valve unit 4.

  Next, a force sense presenting operation by the apparatus configured as described above will be described in accordance with a control procedure of the control unit 6. FIG. 6 is a flowchart showing a control procedure and control contents by the control unit 6. In the figure, the same steps as those in FIG.

  When it is time to detect the position of the receiver 82, the control unit 6 first captures the image data output from the camera 7 via the camera I / F 15 in step S41. The captured image data is temporarily stored in the data memory 14, and then image pattern recognition processing is performed to detect the position coordinates of the marker M displayed on the receiver 82 on the horizontal table.

  Next, in step S <b> 61, the control unit 6 calculates the state of the virtual object that changes according to the movement of the receptor 82, and displays the calculation result on the display device 19. In step S62, the state of the virtual object that changes according to the movement of the receiver 82, the presence / absence of contact between the virtual object and another virtual object, and contact during a period up to a predetermined time are possible. Determine the presence or absence of sex. Subsequently, in step S63, the control unit 6 determines the position of the receiver 82 based on the detected position or orientation data of the receiver 82, or based on the position or shape of the virtual object in the virtual space or their change information. One or more positions of the receptor 82 after a predetermined time from the position detection time are predicted and calculated.

  Subsequently, in step S <b> 64, the control unit 6 selects a jet outlet corresponding to the calculated predicted position of the receiver 82 based on the jet outlet arrangement position data stored in the data memory 14. In step S65, air is ejected from the selected ejection port, and a process for controlling the ejection amount is executed.

The operations from the virtual object state calculation processing described above to the position prediction processing of the receiver 82, the ejection port selection processing according to the predicted position, and the air ejection control processing will be described more specifically.
For example, when a virtual object in the virtual space that moves in conjunction with the position of the receiver 82 contacts another virtual object in response to a change in the state of the virtual object in the virtual space system, Using the position or orientation and the information on the movement trajectory and the change in speed or acceleration of the position of the receptor 82 from the past to the present, and using the state information of the virtual object in the virtual space, the receptor at the time of detection The ejection amount of air from the ejection port according to the position or orientation of 82 is controlled.

  Further, one or more positions or orientations of the receiver 82 after a predetermined ejection delay time has elapsed from the position detection time of the receiver 82 are predicted and calculated, and a jet outlet corresponding to the calculated predicted position; A jet outlet located on the path leading to the predicted position is selected, air is jetted from these jet outlets, and the jet quantity is controlled. Thus, the user can recognize the virtual object by a force sense by receiving the ejected air with the receiver 82.

  Since the calculation of the state of the virtual object may take time, the position of the receiver 82 is detected again when the ejection of the virtual object is determined as a result of the calculation of the state of the virtual object. The position or the predicted position of the receiver 82 after the ejection delay time may be recalculated, and the ejection port may be reselected based on the position after the recalculation.

  As a specific example, it is assumed that the movement of the virtual object in the virtual space is linked to the movement of the acceptor 82 held by the user, and the virtual object in the virtual space moves in conjunction with the movement by moving the acceptor 82. When the virtual object is present in a certain area in the virtual space or at a certain position, or when it touches another virtual object in the virtual space, Define it.

  In addition, as in the first embodiment, the time after which the position of the receiver 82 is predicted is determined in advance. In this state, the position of the receptor 82 is detected, and if the position of the virtual object exists at a position where it is necessary to present the haptic sense to the user, the control unit 6 performs the haptic presentation by the ejection. The position of the receptor 82 at the time of position detection or the ejection delay time from the position detection time using the detection information such as the position of the object or the state information such as the position, orientation, and attribute of the virtual object in the virtual space The air is ejected from the ejection port corresponding to the predicted position of the receiver 82 later. In this case, the position of the receiver 82 is linked to the position of the virtual object, but the predicted calculated position of the receiver 82 is not linked to the virtual object and is used only for ejection control, so the correspondence between the virtual object and the ejection position May shift.

Further, at the time of selecting the ejection port from which the air is to be ejected, the position of the receiver 82 may be detected again, and the selection result of the ejection port may be corrected based on the detected position. . However, in this case, the virtual object and the ejection position may be further shifted.
Furthermore, air is ejected by selecting a jet outlet located on the path from the position at the time of detection to the predicted position of the receiver 82. As a method of calculating the predicted position using the state information of the position or shape of the virtual object, for example, the prediction method is changed between a case of a virtual object that moves quickly and a case of a virtual object that does not move, or a parameter for predicting speed, etc. It may be used as There are also methods for changing the speed and changing the time for which a predetermined time has been predicted.

  As described above, in the second embodiment, the change in the state of the virtual object linked to the receiver 82 is calculated based on the detection result of the movement state of the receiver 82. Then, the calculated state change of the virtual object linked to the receiver 82, the state of the other virtual object expected to contact the virtual object and the change thereof are added to the prediction parameter, and The position and the ejection amount after the ejection delay time are predicted.

  Therefore, when there are virtual objects that change in conjunction with the movement state of the receiver 82, or when there are other virtual objects that are expected to come into contact with the virtual object, the states of these virtual objects are predicted. In addition to the parameters, the position and ejection volume of the receptor can be predicted. Therefore, not only the movement state of the receiver 82 but also the state of the virtual object that is linked to the movement of the receiver 82 and the state of the virtual object that is expected to come into contact with the virtual object are considered. The outlet can be selected and air can be ejected, so that the air can be applied to the receiver 82 more accurately.

(Third embodiment)
The third embodiment of the present invention stores the prediction data of the position of the receptor 82 obtained by the prediction calculation, detects the actual position of the receptor 82 when the ejection delay time has elapsed, The detected actual position of the receiver 82 and the stored predicted position of the receiver 82 are compared to calculate the target level of prediction, and the jet outlet is selected based on the calculated target level. The result is corrected.

FIG. 7 is a block diagram showing a functional configuration of a control unit used in the force sense presentation device according to the third embodiment of the present invention. In the figure, the same parts as those in FIG. 3 or FIG.
In the program memory 13, in addition to the receptor position detection program 131, the receptor position prediction program 132c, the jet outlet selection program 133, the jet control program 134, and the virtual object state calculation program 135, a prediction correction program 136 is newly stored. Yes.

  The prediction correction program 136 determines the degree of accuracy of the predicted position with respect to the actual detection position of the receiver 82, and determines the position coordinate data predicted in the past, its calculation condition, and the number of jet outlets to which air should be ejected. This is held in the data memory 14 until the predicted timing is reached. Then, at the timing of the prediction destination, the actual position of the receiver 82 is detected, and the detected actual position coordinate data is compared with the coordinate data of the stored predicted position so that the prediction is performed. Check the degree of accuracy. Then, the CPU 11 is caused to execute a process of correcting the number of the ejection ports to be selected according to the degree of accuracy.

  Next, a force sense presenting operation by the apparatus configured as described above will be described in accordance with a control procedure of the control unit 6. FIG. 8 is a flowchart showing a control procedure and control contents by the control unit 6. In the figure, the same steps as those in FIG. 6 are denoted by the same reference numerals, and detailed description thereof is omitted.

  When the calculation of the predicted position data of the receptacle 82 is completed in step S63, and the selection process of the ejection port from which air is to be ejected is completed in step S64, the control unit 6 stores the selection result of the ejection port in step S81. The data is stored in a predetermined area in the memory 14. Note that the predicted position data of the receiver 82 may be stored in the data memory 14 instead of the selection result of the spout.

  Next, at the timing of the prediction destination, the control unit 6 captures the captured image data of the camera 7 via the camera I / F 15 in step S82, and the actual position coordinates of the receiver 82 based on this image data. Detect data. In step S83, the detected actual position coordinate data of the receiver 82 is compared with the predicted position coordinate data held in the data memory 14, thereby determining the target level of prediction. Subsequently, in accordance with the determination result of the target degree of prediction, the number of jet outlets selected is adjusted in step S84.

  For example, if the deviation between the actual position coordinate data of the receiver 82 and the predicted position coordinate data is larger than a preset threshold value, it is determined that the target degree is low, while the deviation is less than the threshold value. If there is, it is determined that the target level is high. If it is determined that the target level is high, the number of selected outlets is maintained or reduced by a certain number. On the other hand, if it is determined that the target degree is low, the number of selected outlets is increased by a certain number.

  Specifically, when the position of the receiver 82 is predicted at a certain point in time using a certain calculation formula, and an ejection port existing within an arbitrary radius range near the ejection port corresponding to the predicted position is selected. The radius can be changed with respect to the target degree of prediction. If the target degree is higher than the threshold value, the radius is decreased or maintained as it is. If the target degree is less than the threshold value, the radius is increased. In addition, the degree of target is examined using a plurality of calculation methods, and a method with a high degree of target is adopted.

  It is also conceivable to change the number of jet outlets selected by comparing not only the predicted position shift but also the speed shift. For example, although the predicted position is calculated on the assumption that the vehicle travels at an equal speed or a speed close thereto, there is a method of increasing the number of prediction candidates when the acceptor 82 has not traveled at the same speed or a speed close thereto. As a result, if the user's arm holding the receiver 82 is moving at a similar speed, such as an arc on the left and right, at an equal speed or a speed close thereto, the predicted number of the receivers 82 is small. The number of exits selected is also small. On the other hand, when the user changes the direction of movement of the arm or approaches that state, the predicted position of the receiver 82 is increased, and the number of jets selected is increased.

  In addition, when a plurality of receivers 82, 82 ′ are used and another receiver 82 ′ is present in the vicinity of the receiver 82 calculated by the prediction calculation, the air blown to the other receiver 82 ′ In order not to affect the receiver 82, the number of predicted positions of the receiver 82, and hence the number of jets selected, may be reduced.

As described above, in the third embodiment, the prediction data of the position of the receiver 82 obtained by the prediction calculation or the selection number of the jet outlet corresponding to the data is stored, and the reception at the time when the jet delay time has elapsed is stored. The actual position of the container 82 is detected, and the detected actual position of the receiver 82 is compared with the stored predicted position of the receiver 82 to calculate the predictive level. The selection result of the ejection port is corrected based on the target level.
Therefore, even if a deviation occurs between the predicted data of the position of the receiver 82 and the detection data of the actual position, the number of jet outlets selected can be adjusted in the direction in which the deviation is reduced, thereby reducing the air Thus, it is possible to apply air to the receiver 82 more accurately while suppressing unnecessary ejection.

(Fourth embodiment)
In the fourth embodiment of the present invention, the time required to detect the movement state of the receiver 82 and the opening control signal of the electromagnetic valve to the electromagnetic valve unit 4 after the movement state of the receiver 82 is detected. Required to output the time, the time required from the time when the opening control signal is output until the jets 3, 3,... Jet air, and the air jetted from the jets 3, 3,. The time required to reach the receiver 82 is defined as a delay parameter, and the ejection delay time is determined based on these delay parameters and the interval time until the next movement state detection time of the receiver 82. The value of is set.

  FIG. 9 is a block diagram showing a functional configuration of a control unit used in the force sense presentation device according to the fourth embodiment of the present invention. In the figure, the same parts as those in FIG.

The data memory 14 is provided with a delay parameter storage area 143 in addition to the jet nozzle position information storage area 141 and the receptor change information storage area 142. This storage area 143 as delay parameters, a moving state of the original on the receiver 82 to the image data capturing the image data of the camera 7 (position and orientation) time required to detect (H_ _a) , the receiver 82 moving state the time required until the output opening control signal of the solenoid valve to the electromagnetic valve unit 4 from the detection of the (H_ _b), spout from the opened control signal is output 3,3, ... is the time necessary to eject the air (H_ _c), ejection ports 3,3, time required for the air ejected from ... to reach the receiver 82 (H_ _d) is Remembered. The storage area 143 also stores an interval time until the next movement state detection time of the receiver 82, that is, a position detection cycle (P) of the receiver 82.

Receptor position forecasting program 132d is, the delay parameter stored in the storage area _{143 (H_ a), (H_} b), (H_ c), (H_ d) and the position detection period receptor 82 (P) Based on the above, the CPU 11 is caused to execute a process for setting the time to the prediction destination timing, that is, the ejection delay time, and predicting the position of the receptor 82 after the set ejection delay time.

Because of such a configuration, the following operation is performed in the force sense presentation device of the present embodiment.
Now, it is assumed that the receiver 82 proceeds at a constant speed or a speed close thereto after the position detection time. 'Predicted position X_ at _t' prediction time t after the position detection point, and the coordinate values X_ _t of receptors 82 at the position detecting point of time, the previous coordinates values detected at the detection timing X_ _t-1 of X_ _{t '=} X_ _t + close to a constant multiple _{(X_ t -X_ t-1)} * C (C is a constant)
Can be calculated as

Here, the time (H_delay) taken from the position detection time until the jet nozzle actually blows air and hits the receiver 82 is obtained by the following equation.
_{H_delay = H_ a + H_ b +} H_ c + H_ d
That is, the delay time (H_delay), each delay parameter stored in the storage area 143 of the data memory _{14 (H_ a), (H_} b), is calculated as (H_ _c), the sum of (H_ _d).

Further, since the receptor detection program 131 detects the position of the receptor at each time interval (P), the receptor position prediction program 132d actually ejects air from the position detection time point and the receiver 82 H_delay / P of the position detected at the detection time with respect to the ejection delay which is the time required to hit
_{(H_ a + H_ b + H_} c + H_ d) / P
Predict and calculate the previous position.

That calculation, X coordinate value X_ _t detected in time _t, with respect to the X-coordinate value X_ _t-1 detected in t-1 time, 'X coordinate value X_ _t at the _time' t want to predict, according to the following equation Is done.
_{X_ t '= X_ t + (} X_ t -X_ t-1)
_{* {(H_ a + H_ b} + H_ c + H_ d) / P-C} (C is a constant)
_{Incidentally, {(H_ a + H_ b} + H_ c + H_ d) / P-C} may be calculated in advance, also determine that such 5 or 6 as an integer, calculates five destination or six destination It can also be set as

Further, the operation of this embodiment will be described with specific numbers.
The length of time (H_ _a ) required for the receptor position detection program 131 to detect the position of the receiver 82 is about 30 msec. After the movement state of the receiver 82 is detected, the electromagnetic valve unit 4 is operated with an electromagnetic valve. time necessary to output an opening control signal (H_ _b) about 20 msec, the opened spout from the control signal is outputted 3,3, ... is the time required until the ejecting air (H_ _c) a spout 3,3, ... air injected is a total time of about 50msec and the time required to reach the receiver 82 (H_ _d) from further time interval for detecting the position of the receiver 82 ( When P) is about 20 msec, H_delay / P is
H_delay / P = (30 + 20 + 50) / 20 = 5
It becomes.

Therefore, the pre-reading position is set to about 5 points ahead, and the prediction calculation formula is set as follows.
_{X_ t '= X_ t + (} X_ t -X_ t-1) × 5
Substituting this result, the position of the receiver of the detection time (X_ _t), holds the position of the detected in the previous receiver 82 (X_ _t-1), the position coordinate data of both the above equation Then, the predicted position can be obtained.

As described above, in the fourth embodiment, the time (H_ _a ) necessary for detecting the movement state of the receptor 82 and the electromagnetic valve unit 4 after the movement state of the receptor 82 is detected. time necessary to output an opening control signal of the solenoid valve (H_ _b) and, spout from the opened control signal is outputted 3,3, ... is the time required until the ejecting air (H_ _c) If, spout 3,3, time required for the air ejected from ... to reach the receiver 82 (H_ _d) and, time interval until the next moving state detection time point of the receptacle 82 (P) Based on the above, the timing of the prediction destination is set. The position of the receiver 82 at the set prediction destination timing is predicted.
Therefore, a plurality of delay parameters _{(H_ a), (H_ b} ), (H_ c), (H_ d) and omission consideration, considering further position interval detection time (P), receptors after appropriate time The position of 82 can be predicted.

(Other embodiments)
Wherein in the fourth embodiment, delay parameter _{(H_ a), (H_ b} ), (H_ c), it has dealt with the case where preset as a fixed value (H_ _d). However, the present invention is not limited to this, and the CPU 11 itself detects the processing load amount of the CPU 11 when executing the receptor position prediction program 132d, the jet outlet selection program 133, and the jet control program 134, respectively. Then, in accordance with the detected processing load may be the value of the ejection delay time for setting the prediction target timing (H_ _b) is variably set.

For example, CPU 11 is counting the time necessary to output an opening control signal of the solenoid valve (H_ _b) to the solenoid valve unit 4 from the moving state of the receptor 82 by the receptor position detecting program 131 is detected To do. Then, when this measured time is longer than a threshold set in advance, a longer time only H_ _b corresponding to the difference, it changes the prediction formula using the H_ _b of this length is changed.

In the above-described embodiment, the position of the receiver 82 is detected based on the image data captured by the camera 7. However, the position detection sensor such as an acceleration sensor or a gyroscope is built in the receiver 82. The position of the receptor may be detected based on the detection signal.
Further, in the above-described embodiment, the case where the plurality of jet nozzles 3, 3,... Are arranged on a two-dimensional plane has been described as an example. However, the present invention is not limited to this. For example, a plurality of jet outlets may be regularly arranged on a three-dimensional surface such as a cube, a rectangular parallelepiped, a sphere, a hemisphere, a column, or a cone.

  Further, assuming that the virtual object moves in a three-dimensional virtual space and the receptor 82 moves in a three-dimensional direction, the prediction calculation of the position of the receptor 82 is executed according to the three-dimensional coordinates (X, Y, Z coordinates). You may make it do. In this way, it is possible to predict the three-dimensional position of the receptor 82 after the ejection delay time in consideration of the movement of the virtual object in the three-dimensional virtual space and the movement of the receptor 82 in the three-dimensional direction. it can.

  In addition, the processing pattern of the ejection pattern of the ejection unit in the ejection unit, the receptor position detection program in the ejection control unit, the receptor position prediction program, the ejection port selection program, the ejection control program, the virtual object state calculation program, and the prediction correction program The processing contents, the type of fluid, and the ejection direction thereof can be variously modified and implemented without departing from the gist of the present invention.

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, some components may be deleted from all the components shown in each embodiment. Furthermore, you may combine suitably the component covering different embodiment.

1 is a diagram illustrating an overall configuration of a force sense presentation device according to a first embodiment of the present invention. It is a figure which shows the air ejection operation | movement by the force sense presentation apparatus shown in FIG. 1, and its pressure reception operation | movement by a receptor. It is a block diagram which shows the function structure of the control unit in the force sense presentation apparatus shown in FIG. It is a flowchart which shows the procedure and content of force sense presentation control by the control unit shown in FIG. It is a block diagram which shows the function structure of the control unit in the force sense presentation apparatus concerning 2nd Embodiment of this invention. It is a flowchart which shows the procedure and content of force sense presentation control by the control unit shown in FIG. It is a block diagram which shows the function structure of the control unit in the force sense presentation apparatus concerning the 3rd Embodiment of this invention. It is a flowchart which shows the procedure and content of force sense presentation control by the control unit shown in FIG. It is a block diagram which shows the function structure of the control unit in the force sense presentation apparatus concerning 4th Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Jet unit, 2 ... Support stand, 3 ... Jet port, 4 ... Solenoid valve unit, 5 ... Compressor, 6 ... Control unit, 7 ... Camera, 8 ... Receptor, M ... Marker, A ... Jet air, 11 ... Central processing unit (CPU), 12 ... bus, 13 ... program memory, 14 ... data memory, 15 ... camera interface, 16 ... external interface, 17 ... input / output interface, 18 ... input device, 19 ... display device, 81 ... handle , 82 ... receptor, 131 ... receptor position detection program, 132a, 132b, 132c, 132d ... receptor position prediction program, 133 ... jet outlet selection program, 134 ... jet control program, 135 ... virtual object state calculation program, 136 ... Prediction correction program, 141 ... Spout position information, 142 ... Receptor change information, 1 3 ... delay parameter.

Claims (6)

An ejection unit in which a plurality of ejection ports each ejecting a fluid are arranged with regularity;
A receptor detection means for detecting a state of the movement of a receptor that receives fluid ejected from the plurality of ejection ports while moving opposite to the arrangement surfaces of the plurality of ejection ports;
An ejection control unit that selectively controls ejection of fluid from the plurality of ejection ports;
The ejection control unit includes:
Means for calculating a change in the state of the virtual object linked to the state of movement of the receptor based on the detection result of the state of movement of the receptor by the receptor detection unit;
At least one of the calculated change in the state of the virtual object, the state of the other virtual object expected to contact the virtual object and the change, and the movement of the receptor by the receptor detection means; Prediction means for predicting the position of the receptor after a preset ejection delay time from the detection time of the state of movement based on the detection result of the state;
A selection means for selecting a jet outlet from which the fluid is to be jetted out of the plurality of jet outlets based on a prediction result of the position of the receptor by the prediction means, and an arrangement position of the plurality of jet outlets;
Ejection control means for controlling the ejection unit to eject fluid from the ejection port selected by the selection means ;
The time required for the receptor detection means to detect the movement state of the receptor, the time required from the detection of the movement state of the receptor to the output of an ejection command to the ejection unit, and At least one of the time required for the ejection unit to eject the fluid after the ejection command is output and the time necessary for the fluid ejected from the ejection port to reach the receiver is used as a delay parameter. A force sense presentation device comprising: means for setting the ejection delay time based on a delay parameter and an interval time until the next movement state detection time of the receptor . The ejection control unit includes:
Means for detecting a processing load when the prediction means, the selection means and the ejection control means each execute processing;
Means for variably controlling the value of a delay parameter indicating a time required from when the moving state of the receptor is detected until the ejection command is output to the ejection unit according to the detected processing load; The force sense presentation device according to claim 1 , further comprising:   The prediction means predicts the position of the receptor after a preset ejection delay time from the detection time of the movement state of the receptor, and predicts the movement path to the predicted position,
  The selection unit is configured to select a fluid from the plurality of jets based on the prediction result of the position of the receptor by the prediction unit, the prediction result of the movement path to the prediction position, and the arrangement positions of the plurality of jets. The force sense presentation device according to claim 1 or 2, wherein a plurality of jetting outlets for jetting the jet are selected. By using a jet unit in which a plurality of jets are arranged with regularity, and by selectively ejecting fluid from the plurality of jets under the control of the control unit, the arrangement surface of the plurality of jets is arranged. In a force sense presentation method that gives a tactile sensation to a receptor that moves oppositely,
A detection process for detecting a state of movement of the receptor at an arbitrary detection timing;
Calculating a change in the state of the virtual object linked to the state of movement of the receptor based on the detection result of the state of movement of the receptor;
Based on at least one of the calculated change in the state of the virtual object, the state of the other virtual object expected to be in contact with the virtual object and the change, and the detected state of movement of the receptor A prediction process for predicting the position of the receptor after a preset ejection delay time from the time of detection of the state of movement,
A selection process of selecting a jet outlet from which the fluid is to be jetted out of the plurality of jet outlets based on a prediction result of the position of the receptor by the prediction process and an arrangement position of the plurality of jet outlets;
An ejection control process for controlling the ejection unit to eject a fluid from the ejection port selected in the selection process ;
The time required to detect the movement state of the receptor, the time necessary to output the ejection command to the ejection unit after the movement state of the receptor is detected, and the ejection command are output. At least one of the time required for the ejection unit to eject the fluid and the time required for the fluid ejected from the ejection port to reach the receiver, and the delay parameter And a step of setting the ejection delay time based on an interval time until the next movement state detection time of the receptor . Detecting a processing load of the control unit when performing the prediction process, the selection process, and the ejection control process;
In accordance with the detected processing load, a process of variably controlling a delay parameter value indicating a time required from when the moving state of the receptor is detected to when an ejection command is output to the ejection unit. The force sense presentation method according to claim 4, further comprising: The prediction process predicts the position of the receptor after a preset ejection delay time from the detection time of the movement state of the receptor, and predicts the movement path to the predicted position,
  The selection process is performed based on the prediction result of the position of the receptor in the prediction process, the prediction result of the movement path to the prediction position, and the arrangement position of the plurality of jet nozzles, and the fluid is selected from the plurality of jet nozzles. The force sense presentation method according to claim 4 or 5, wherein a plurality of jetting outlets from which jetting is to be jetted are selected.

Images