JP6192579B2 - Input device and input program - Google Patents

Description

  The present invention relates to an input device and an input program for measuring an external force applied to an object based on elastic deformation.

  In recent years, in games that run on a computer, the movement of the user's body and the object held by the user is photographed and recognized by a camera, and the virtual object in the virtual space in the computer where the game progresses based on the recognized movement It is done to operate. (For example, refer to Patent Document 1.)

  In the technique disclosed in Patent Document 1, after a human body or object is recognized, individual models are created by scanning the recognized human body or object or matching with a model prepared in advance. To do. After creating the model, the created model is used to recognize a human body or an object and to manipulate a virtual object. In the technique of Patent Document 1, a skeleton model is used to recognize the movement of the human body.

Special table 2012-521039 gazette

  However, in a baseball game, for example, when a user grasps a stick that is close to the bat and hits the ball in the virtual space by swinging, the user measures how much the user is holding the bat. I couldn't. Therefore, using the gripping force to grip the bat as a parameter, it was impossible to change the flying direction and speed of the hit ball due to the difference in the gripping force.

  In general, the external force applied to an object cannot be measured (estimated) without using a pressure sensor or the like.

  In view of the circumstances as described above, an object of the present invention is to provide an input device and an input program for estimating an external force applied to an object based on elastic deformation.

  In order to achieve the above object, an input device according to an embodiment of the present invention is capable of imaging a state in which a first object having linear elasticity and a known elastic modulus is pressed with an unknown first external force. And an amount of distortion of the first object based on the captured image, and the first external force based on the elastic modulus of the first object and the estimated amount of distortion of the first object. And an estimation unit for estimating.

  In order to achieve the above object, in the input device according to an aspect of the present invention, the estimation unit may be configured to estimate the distortion amount from a difference between an image in a non-pressed state and an image in a pressed state.

  In order to achieve the above object, in the input device according to an aspect of the present invention, the imaging unit uses the first object to convert the first object having a linear elasticity and an unknown elastic modulus to the first object. The state of pressing with an external force is imaged, and the estimation unit estimates a distortion amount of the second object based on the captured image, and the first external force and the estimated distortion amount of the second object. Based on the above, the elastic modulus of the second object may be estimated.

  In order to achieve the above object, in the input device according to an aspect of the present invention, the imaging unit presses the second object whose elastic modulus is estimated by the third object with an unknown second external force. The state may be captured, and the estimation unit may estimate the second external force based on the estimated amount of distortion of the second object and the estimated elasticity of the second object.

  In order to achieve the above object, in the input device according to an aspect of the present invention, the imaging unit captures an image of a state in which the second object whose elastic modulus is estimated is placed on a third object, and The estimation unit may be configured to estimate the weight of the second object based on the estimated amount of distortion of the second object and the estimated elasticity of the second object.

  In order to achieve the above object, an input program according to an embodiment of the present invention is based on an image obtained by photographing a state in which a first object having linear elasticity and a known elastic modulus is pressed with an unknown external force. A computer is caused to function as an estimation unit that estimates a strain amount of the first object and estimates the external force based on an elastic modulus of the first object and the estimated strain amount.

  As described above, according to the present invention, an external force applied to an object can be estimated based on elastic deformation.

It is a block diagram of the input device 1 concerning this embodiment. It is a block diagram of the software which implement | achieves the functional block used by this embodiment. It is a figure which shows a deformation | transformation state in case the object O1 which the finger | toe F pinches is a rigid body. It is a figure which shows a deformation | transformation state when the object O2 which the finger | toe F pinches is very soft compared with a finger. Softness of the object O 3 sandwiching the finger F and the finger F is a diagram showing a deformation state where comparable. It is an activity diagram for demonstrating the flow of the calibration process in the input device 1 concerning this embodiment. It is an activity diagram explaining the flow of processing for estimating an unknown elastic modulus when an object with an unknown elastic modulus is grasped using a finger whose elastic modulus is estimated by calibration. It is an activity figure explaining the flow of the processing which estimates the weight of the object which estimated the elasticity modulus. It is an activity figure explaining the flow of processing which executes an application program using the estimated elastic modulus and weight. It is a figure which shows the example which applied the input device 1 of this embodiment to the baseball batting game.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Overview and prerequisites]
First, an outline and preconditions of the present embodiment will be described.

  In this embodiment, a state where a finger or the like grabbed an object is imaged with a camera or the like, the amount of distortion or elastic modulus of the finger is estimated based on the captured image, and the magnitude of the external force applied to the finger from the estimation result This is an estimate.

  Here, in the present embodiment, when imaging a state in which a finger or the like grabbed and deformed with a camera or the like, the finger or the object is below the elastic limit, that is, the stress (external force) and the amount of distortion are proportional to the elastic constant. It is assumed that the linear elasticity is in proportion.

  The camera used in the present embodiment is premised on that it is always arranged at an optimal position and can appropriately capture a state where a finger or an object sandwiched between fingers is deformed.

  The outline and the preconditions of the present embodiment have been described above.

[Constitution]
Next, the configuration of the input device according to the present embodiment will be described. FIG. 1 is a configuration diagram of an input device 1 according to the present embodiment.

  The input device 1 includes a camera 10 (imaging unit) and an information processing device 30.

  As the camera 10, a general camera may be used as long as it can capture an image of a deformed state where a finger or the like grabs an object.

  The information processing apparatus 30 can use a general computer such as a PC (Personal Computer). The information processing apparatus 30 includes a processor 31, a memory 32, an input / output unit 33, and the like.

  The information processing apparatus 30 may be connected to a display unit 20 for presenting information processing results to the user. The display unit 20 and the camera 10 are connected to the input / output unit 33 of the information processing apparatus 30.

  Each function block used in the present embodiment is realized by the processor 31 executing a program stored in the memory 32 or the like. FIG. 2 shows the configuration of software that implements the functional blocks used in this embodiment. FIG. 2 is a configuration diagram of software that implements functional blocks used in the present embodiment.

  The software that implements the functional blocks used in the present embodiment includes a main program 31a, a recognition engine 31b (estimator), and an application program 31c.

  Details of the main program 31a and the recognition engine 31b will be described later. The application program 31c is, for example, a game set in a virtual space as disclosed in Patent Document 1.

  The configuration of the input device 1 according to the present embodiment has been described above.

[Relationship between finger and object deformation]
Next, a state in which a finger and an object pressed by the finger that are imaged by the camera 10 of the input device 1 according to the present embodiment are elastically deformed by the pressing will be described.

  FIG. 3 is a diagram illustrating a deformed state when the object O1 between the fingers F is a rigid body. Since the object O1 is very hard, it does not deform even when pressed by a finger. Conversely, the finger is greatly deformed. The pressing force can be estimated from the deformation amount of the finger F.

  FIG. 4 is a diagram showing a deformed state when the object O2 held by the finger F is very soft compared to the finger. The object O2 is very soft and is greatly deformed. Conversely, the finger is hardly deformed. The pressing force can be estimated from the deformation amount of the object O2.

FIG. 5 is a diagram illustrating a deformed state when the softness of the object O 3 between the finger F and the finger F is approximately the same. Both the object O3 and the finger F are deformed. The pressing force can be estimated from the deformation amount of the finger F and the deformation amount of the object O3.

  As described above, the state in which the finger and the object pressed by the finger that are imaged by the camera 10 of the input device 1 according to the present embodiment are elastically deformed by the pressing has been described.

[Calibration process flow]
Next, the flow of calibration processing in the input device 1 according to the present embodiment will be described. FIG. 6 is an activity diagram for explaining the flow of calibration processing in the input apparatus 1 according to the present embodiment. In the calibration process, the pressing force is obtained by imaging a state in which an object having a known elastic modulus is grasped by a hand, and the elastic modulus (rigidity) of the hand is estimated from the pressing force and the amount of deformation of the hand. Is.

  First, the main program 31a notifies the user that measurement of the elastic modulus (rigidity) of the hand is started (step S1).

  Next, the main program 31a instructs the user to prepare a calibration object (first object) whose elastic modulus is known (step S2).

  Next, the camera 10 captures an image of a hand (finger, second object) and an object (a force has not yet been applied) (step S3).

  Next, the recognition engine 31b recognizes the imaged hand (finger) and the object (step S4).

  Next, the main program 31a instructs the user to press the object with a finger (step S5).

  Next, the camera 10 captures an image of a finger and an object that are deformed by pressing (step S6).

  Next, the recognition engine 31b recognizes the deformation amount (distortion amount) of the finger and the object (step S7).

  Next, the recognition engine 31b estimates the pressing force (first external force) from the recognized amount of distortion of the object and the known elastic modulus, and estimates the elastic modulus of the finger from the estimated force and the amount of distortion of the finger ( Step S8). The deformation amount (distortion amount) can be obtained from the difference between the image of the object before applying the external force and the image of the object deformed by applying the external force.

  Next, the main program 31a adds the estimated elastic modulus as an attribute to the hand (finger) model used in the application program 31c (step S9).

  Next, the main program 31a notifies the user of the end of the measurement of the elastic modulus (rigidity) of the hand (step S10).

  The flow of the calibration process in the input device 1 according to the present embodiment has been described above.

[Flow of processing to estimate unknown elastic modulus]
Next, a flow of processing for estimating an unknown elastic modulus when an object with an unknown elastic modulus is grasped using a finger whose elastic modulus is estimated by calibration will be described. FIG. 7 is an activity diagram for explaining the flow of processing for estimating an unknown elastic modulus when an object whose elastic modulus is unknown is grasped using a finger whose elastic modulus is estimated by calibration.

First, the main program 31a informs the user that measurement of the elastic modulus (rigidity) of an object with an unknown elastic modulus ( third object ) is started (step S21).

  Next, the main program 31a instructs the user to prepare an object whose elastic modulus is to be obtained (step S22).

  Next, the camera 10 images a hand (finger) and an object (a force has not been applied yet) (step S23).

  Next, the recognition engine 31b recognizes the imaged hand (finger) and the object (step S24).

  Next, the main program 31a instructs the user to press the object with a finger (step S25).

  Next, the camera 10 captures an image of a finger and an object that are deformed by pressing (step S26).

  Next, the recognition engine 31b recognizes the deformation amount (distortion amount) of the finger and the object (step S27).

  Next, the recognition engine 31b estimates the pressing force (second external force) from the recognized finger distortion amount and the estimated elastic modulus, and estimates the elastic modulus of the object from the estimated force and the object distortion amount. (Step S28).

  Next, the main program 31a adds the estimated elastic modulus as an attribute to the object model used in the application program 31c (step S29).

  Next, the main program 31a notifies the user of the end of the measurement of the elastic modulus (rigidity) of the object (step S30).

  The flow of processing for estimating an unknown elastic modulus when an object with an unknown elastic modulus is grasped using a finger whose elastic modulus is estimated by calibration has been described above.

[Flow of processing to estimate unknown weight]
Next, the flow of processing for estimating the weight of an object whose elastic modulus has been estimated by the above processing will be described. FIG. 8 is an activity diagram for explaining the flow of processing for estimating the weight of an object whose elastic modulus has been estimated.

  First, the main program 31a informs the user that measurement of the weight of the object whose elastic modulus has been estimated is started (step S41).

Next, the main program 31a instructs the user to place an object (a third object ) whose weight is to be obtained on a hand or the like (a second object) (step S42).

  Next, the camera 10 images an object that is deformed by its own weight (step S43).

  Next, the recognition engine 31b recognizes the deformation amount (distortion amount) of the object (step S44). As the image of the object before deformation used for the difference for obtaining the amount of distortion, the image captured in step S23 can be used.

  Next, the recognition engine 31b estimates the weight of the object from the amount of distortion of the recognized object and the estimated elastic modulus (step S45).

  Next, the main program 31a adds the estimated weight as an attribute to the object model used in the application program 31c (step S46).

  Next, the main program 31a notifies the user of the end of the measurement of the weight of the object (step S47).

  The processing flow for estimating the weight of the object whose elastic modulus has been estimated has been described above.

[Flow of application program execution]
Next, the flow of processing for executing an application program using the elastic modulus and weight estimated above will be described. FIG. 9 is an activity diagram for explaining the flow of processing for executing an application program using the estimated elastic modulus and weight.

  First, the main program 31a executes an application program 31c such as a game using a model applied with the elastic modulus and weight of a finger or an object estimated by the above-described calibration processing or the like (step S51).

  Next, the main program 31a determines whether or not the execution of the application program 31c has ended (step S52).

  When the execution has not been completed (N in step S52), the camera 10 images the deformation state of the finger or object (step S53), and the recognition engine 31b presses the finger or object according to the imaged deformation amount. (Step S54), and the application program 31c is executed using the estimated force.

  In this case, the application program 31c is executed by estimating the elastic modulus and weight. However, the application program is not limited to this, and the application program is executed using the general elastic modulus of the finger without estimating the elastic modulus and weight. May be executed.

The process flow for executing the application program using the estimated elastic modulus and weight has been described above.

[Application Example 1 of Input Device 1]
Next, the example which applied the input device 1 of this embodiment to the baseball batting game is demonstrated. FIG. 10 is a diagram illustrating an example in which the input device 1 of the present embodiment is applied to a baseball batting game.

  Conventionally, the direction in which the hit ball flies is calculated from the weight, speed, and rotation of the ball and bat.

  However, in this example, the camera 10 images the hand holding the bat, and the amount of deformation of the hand when the ball hits the bat can also be imaged.

  In this example, the angle of the bat at the moment the ball hits the bat can be calculated based on how much force the bat is gripped on, and more accurately the direction of the hit ball can be calculated. I can do it.

  Thus, even when an object that does not include a pressure sensor is used for the bat, the force used for pressing can be estimated and the direction in which the hit ball flies can be calculated more accurately.

  The example in which the input device 1 of the present embodiment is applied to a baseball batting game has been described above.

[Application Example 2 of Input Device 1]
Next, an example in which the input device 1 of the present embodiment is applied to a keyboard that is an input device of a computer will be described.

  Normally, each key of the keyboard can only detect either a pressed state or a non-pressed state. For example, if the lowercase “a” is entered by pressing the a key, the “Shift” key must also be pressed to enter the uppercase “A”.

  However, by applying the input device 1 of this embodiment to a keyboard and estimating the pressure to press the key from the amount of finger deformation when the key is pressed, for example, when the key “a” is lightly pressed, the lowercase letter “a” ”Is input, and when pressed with a strong force, a capital“ A ”can be input.

  The example in which the input device 1 of the present embodiment is applied to a keyboard that is an input device of a computer has been described above.

[Additional notes]
In addition, the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the scope of the present technology.

DESCRIPTION OF SYMBOLS 1 ... Input device 10 ... Camera 20 ... Display part 30 ... Information processing apparatus 31 ... Processor 31a ... Main program 31b ... Recognition engine 31c ... Application program 32 ... Memory 33 ... Input / output part F ... Fingers O1, O2, O3 ... Object

Claims (6)

An imaging unit capable of imaging a state in which a first object having linear elasticity and a known elastic modulus is pressed by a second object having linear elasticity and an unknown elastic modulus with an unknown first external force; ,
Estimating a distortion amount of the first object and a distortion amount of the second object based on the captured image;
Estimating the first external force based on the elastic modulus of the first object and the estimated amount of strain of the first object ;
An input device comprising: an estimation unit configured to estimate an elastic modulus of the second object based on the estimated first external force and the estimated amount of distortion of the second object . The input device according to claim 1,
The imaging unit
Imaging the state in which the first object is not pressed by the second object,
The estimation unit includes
From the difference between the image in the state where the first object is not pressed by the second object and the image in the state where the first object is pressed by the second object, the distortion amount of the first object and An input device for estimating a distortion amount of the second object . The input device according to claim 2,
The imaging unit
Imaging a state in which a third object having a linear elasticity and an unknown elastic modulus is not pressed by the second object whose elastic modulus is estimated;
Imaging the state in which the third object is pressed with an unknown second external force by the second object having the estimated elastic modulus,
The estimation unit includes
From the difference between the image in a state where the third object is not pressed by the second object and the image in the state where the third object is pressed by the second object, the distortion amount of the second object is calculated. Estimate
An input device that estimates the second external force based on the estimated amount of distortion of the second object and the estimated elastic modulus of the second object . The input device according to claim 3 ,
The estimation unit includes
From the difference between the image in a state where the third object is not pressed by the second object and the image in the state where the third object is pressed by the second object, the distortion amount of the third object is calculated. Further estimate,
An input device that estimates an elastic modulus of the third object based on the estimated second external force and the estimated amount of distortion of the third object . The input device according to claim 4 ,
The imaging unit
Imaging a state in which the third object whose elastic modulus is estimated is placed on the second object,
The estimation unit includes
Input apparatus for estimating the weight of the third object on the basis of the amount of distortion and the third object modulus of which is the estimate of the third object described above estimation. Based on an image obtained by imaging a state in which a first object having linear elasticity and a known elastic modulus is pressed by a second object having linear elasticity and an unknown elastic modulus with an unknown first external force , Estimating a distortion amount of the first object and a distortion amount of the second object ;
Estimating the first external force based on the elastic modulus of the first object and the estimated amount of strain of the first object ;
An input program for causing a computer to function as an estimation unit that estimates an elastic modulus of the second object based on the estimated first external force and the estimated amount of distortion of the second object .

Images