JPH0816861B2 - 3D graphic tactile feedback device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Outline] The present invention relates to a three-dimensional graphic tactile feedback device, which is intended for a cursor or the like operated by an input device when directly operating a three-dimensional graphic displayed on a display screen. It is a three-dimensional graphic tactile feedback device that can directly feed back as a reaction of the operator's hand when touching a three-dimensional figure.

[Field of Industrial Application] The present invention relates to a three-dimensional graphic tactile feedback device.

In recent years, three-dimensional graphic technology has been widely used for CAD (Computer Aided Design), software design, and the like. In that case, the figure is displayed three-dimensionally on a display device such as a CRT display to display the shape of the object in three dimensions, making it easy to understand the shape of the object and facilitating complicated design work in various technical fields. Is being done.

An input device that directly operates a three-dimensional graphic (called an object) has also come to be used, and a processing targeting a specific graphic can be performed by operating a cursor or the like that indicates a three-dimensional position. .

[Prior Art] Conventionally, when a graphic on a display device is processed, various processes are performed by instructing a specific position with an input means such as a keyboard, a mouse, and a light pen. With the means, pointing of two-dimensional coordinates is mainly performed, and it is difficult to perform an operation for processing three-dimensional position data and an object having a depth of a three-dimensional figure.

On the other hand, an input device (3) which has a three-dimensional movement of an object recently drawn in a screen with a depth such as a three-dimensional screen.
Dimensional input device) can be used to capture and operate. An example of such an input device is a device called a data glove (trade name). This device has an optical fiber folded at the tip attached to each finger of the glove, the angle of bending of the finger is detected by measuring the amount of light passing through the fiber, and the output of the magnetic generator attached to the glove is fixed. A magnetic sensor is used to detect a signal corresponding to the three-dimensional position of the hand and the twist angle, and a data signal is generated (for example, Nikkei Electronics 1988, November 14).
Sun, p. 106).

In such a three-dimensional input device, an indicator such as a cursor is three-dimensionally operated on a screen of a three-dimensional graphic to indicate (or touch) a specific object of the three-dimensional figure (target. Operate to grasp objects with input device),
Processing can be performed on the object.

[Problems to be Solved by the Invention] In the above-described conventional three-dimensional input device, even if a cursor or the like indicating a three-dimensional position on the CRT display contacts an object, a three-dimensional image is displayed on a two-dimensional plane display screen. Since it is displayed in, it is not possible to clearly distinguish whether the cursor simply overlaps the object on the screen or whether it is in agreement (the cursor and the object are in contact) on the three-dimensional coordinates (three-dimensional position). .

In this case, for example, it is possible to provide feedback to the operator by detecting that the three-dimensional positions of the object and the cursor are coincident with each other by detecting a conventionally known figure calculation process and displaying a color change of the contacted object.
It is not possible to give a touch feeling to the operator who operates the cursor.

SUMMARY OF THE INVENTION It is an object of the present invention to realize a three-dimensional graphic touch feeling feedback device capable of directly feeding back the touch feeling to an operator's hand when touching an object in a system for operating a three-dimensional graphic display screen. .

[Means for Solving the Problems] FIG. 1 is a basic configuration diagram of the present invention.

In FIG. 1, reference numeral 10 is an operation for inputting a spatial position and an operation instruction on a screen on which a three-dimensional figure is displayed, and for outputting a signal indicating a position of a cursor or the like to the processing device or a signal instructing a processing content. An input control unit 101 is an operation unit that is operated by an operator to generate a signal indicating a three-dimensional position and a processing instruction, 11 is a processing device that processes graphic data and controls each unit, 12 is a display unit, and 13 is a display unit. A storage unit that stores image data for graphic display and object data including object hardness (softness),
Reference numeral 14 denotes a touch feeling feedback control unit, and 15 denotes a main body unit.

The present invention operates the operating means to move the cursor,
When an object is touched on the display screen, the touch feeling is fed back to the operator's hand.

[Operation] On the display unit 12, an object having a three-dimensional depth or an image representing information having multidimensional elements is displayed as a three-dimensional figure. Specific figure (object) in this display screen
The operation means 101 is operated in order to perform processing on the.
When a three-dimensional operation is performed by the operation means 101, a three-dimensional position signal corresponding to the movement is input to the operation input control unit 10, converted into coordinate data and input to the cursor control unit 111.

The output of the cursor control unit 111 is given to the display unit 12,
The cursor position on the screen (the position where the three-dimensional position is converted into the two-dimensional coordinate on the screen) moves. The operation means 10
If you want to control (1) to capture (or grasp) a specific object on the screen, move the cursor to the position where it touches the target object, and issue the capture instruction to the operating means 10.
When input from 1, the operation input control unit 10 gives the instruction to the processing device 11.

The cursor control unit 111 of the processing device 11 controls the movement of the cursor in response to the input, and the coordinate data thereof is detected by the coincidence detection unit 11
Output to 3. On the other hand, the display figure generating section 112 generates a screen to be output to the display section 12, and outputs the data (shape, coordinates) of each object figure to the coincidence detecting section 113. The match detection unit 113 is driven when a capture instruction is input from the operation input control unit 10 to perform a match detection operation, and the cursor control unit 113
It is determined whether or not the coordinate signal from 111 and the object graphic from the display graphic generating unit 112 match at a three-dimensional position. When a match is detected, object data representing the attribute of the object is retrieved from the storage unit 13 and output to the touch feeling feedback control unit 14 via the match detection unit 113.

As the object data, data representing "hardness" (or softness) of the object represented by the object is used.

As a result, the contact feeling feedback control unit 14 controls the main body unit 15 with the data indicating that the two match. The main body 15 is made of a shape memory alloy and is used by being attached to the operator's hand. When the contact feeling feedback control unit 14 is driven by the "hardness" data, a plurality of shape memory alloys are used to respond to it. The main body 15 of the configured feedback device is driven (driven to return to the memory shape).

At that time, the contact feeling feedback control unit 14 is supplied with a control signal corresponding to the "hardness" data of the object to the main body unit 15, and the shape memory alloy is deformed in response to the control signal and exerts an action on the operator's hand. As a result, the operator can obtain a feeling of hardness of the object at the same time as a feeling of touching the object.

In the above description, when the match detection unit 113 detects a match, the data representing the “hardness” of the object is output. However, the contact feeling feedback control unit 14 is simply driven by the match output to operate the main body unit 15. It can also be configured.

When the object is pointed (touched) by the cursor, processing such as update, deletion, rotation, movement, etc. can be performed on the object.

[Embodiment] FIG. 2 is a system configuration diagram of an embodiment of the present invention, FIG. 3 is an embodiment configuration diagram of a feedback control unit, FIG. 4 is an embodiment configuration diagram of a main body unit, and FIG. It is a processing flow figure.

In FIG. 2, reference numeral 20 is a disk device in which an object shape information file including data of “hardness” of an object represented by a graphic is stored, 21 is a processing device (CPU and memory), 2
2 is a CRT display device, 23 is a touch feeling feedback device, 24 is an operation input control device, and 25 is an operation input device.

The operation input device 25 and the operation input control device 24 correspond to the operation means 101 and the operation input control unit 10 in FIG. 1, and are for inputting three-dimensional coordinate positions and operation instructions by the movement of the operator's hand. . As a specific example, the data glove (product name) described above can be used. It is also possible to use other means for inputting the three-dimensional coordinates (for example, by inputting depth data in addition to the coordinates on the two-dimensional plane with a mouse).

In this embodiment, a hand-shaped cursor shown in the CRT display device 22 of FIG. 2 is moved on the display screen according to the operation of the operation input device 25, and the operator's hand is moved when the cursor touches an object. Etc. to the touch feeling feedback device, and the strength of the action is set to any one of a plurality of stages depending on the object.

In the processing device 21, a program and data for graphic creation processing are stored in the disk device 20, graphic data processing is performed in the memory, and events (three-dimensional objects, information having multidimensional elements, etc.) ) Performs a specified process on a graphic having various forms in which the three-dimensional space is expressed, and creates a three-dimensional graphic.

The data of the three-dimensional space that has been subjected to the three-dimensional processing is displayed on the CRT display device 22 as the display data of the two-dimensional screen. This C
In the example of the display screen of the RT display device 22, when the two columnar object figures 221 and 222 are displayed, the operation input device 25 is operated to move the cursor 223 corresponding to the movement thereof, and the cursor 223 is moved. The state where the object 221 is touched is displayed. At this time, when the processing device detects that a contact has been made, data representing the "hardness" of the object is extracted from the object shape information file stored in the disk device 20, and the contact feeling feedback device is provided.
It is input to the control unit 231 of 23, and the main body unit 232 is driven so as to give a touch feeling to the operator.

In FIG. 2, data is transmitted and received between the operation input control device 24 and the processing device 21, and between the contact feeling feedback device 23 and the processing device in the form of packet data by a known RS-232C interface (not shown).

FIG. 3 shows an embodiment configuration of the control unit 231 of the contact feeling feedback device 23, and FIG. 4 shows an embodiment configuration of the main body unit.

In FIG. 3, reference numeral 30 denotes a serial-parallel conversion circuit, which receives packets transmitted in series from the processing device 21 through an interface (for example, a well-known RS232C interface). The packet contains data representing the hardness of the object. In the example of the figure, the hardness data represented by 4-bit "1110" (in Fig. 3, the right end is displayed first) is input. Has been done. These 4 bits are used to express “hardness” in 16 steps (2 ⁴ = 16). In the case of “1110”, if “0001” is the lowest level (soft), it is 14 Equivalent to the second hardness.

The serial data is converted into parallel data, and the bit signals are output to the control lines l _{1 to} l ₄ . Each control line l _{1 to} l ₄ is connected to a relay R _{1 to} R ₄ , respectively, and each relay is driven when “1” is supplied to the corresponding control line to close the contact. Third
In the state shown in the figure, “1” is supplied to the control lines l ₁ , l ₂ , and l ₃ and “0” is supplied to the control line l ₄ , so that the relays R ₁ , R ₂ , and R ₃ are driven, and the relay R ₄ is not driven. Therefore, the contact points r ₁ , r
₂ and r ₃ are closed, and the voltage of the power supply line B is supplied to the output lines A ₁ , A ₂ and A ₃ . This output line is supplied to the main body shown in FIG.

 Next, the construction of the embodiment of the main body shown in FIG. 4 will be described.

In FIG. 4, reference numerals 41 to 44 are main bodies made of four congruent shape memory alloys. However, this number may be a number other than four, and the number can be freely increased if the hardness level is increased.

Each of the body parts is made of a thin alloy and is thermally insulated from each other. The alloy is attached on the upper side of the human hand (back of the hand), for example, on gloves, and flexibly follows the movement of the hand at room temperature (the relay switch is off).

A known alloy is used as the shape memory alloy, and there is no particular shape at room temperature, and the shape memory alloy can flexibly respond to the movement of the human hand. Then, at a certain temperature determined by the alloy, the memorized shape (initial shape) is shaped into the shape shown in Fig. 4B (shape with slightly open hand).

In addition, the shape memory alloy has 4 of 41 to 44 as shown in A.
It is constructed by using two lines (or plates) (42 to 44 are only the thumb portion), and is driven according to the signals supplied to the output lines A _{1 to} A ₄ from the control unit (FIG. 3). When driven, heat is generated by the electric current, and when it reaches a certain temperature, the memory shape (Fig. 4
The action to return to B.) works. When the number of alloys driven by the drive signal from the control unit (returning to the memory shape) increases, the returning force increases and the reaction on the human hand shown in FIG. 4C increases.

This action on the operator is shown in FIG. 4C. When the shape memory alloy is opened (see FIG. 4B.), It reacts with the human hand (the fingers of the hand are normally inward). While facing, the action in the opposite direction) works. That is, when trying to grab an object by bending the fingers of the hand inward, the control device is driven by the contact information, and the shape of the main body portion is deformed to the opposite side. The strength of the reaction at this time differs depending on how many of the plurality of alloys are driven, and the reaction force becomes one of a plurality of stages accordingly. The force of this reaction is perceived by the operator as the hardness of the object.

As shown in FIG. 3, when the control unit drives the lines A ₁ , A ₂ , and A ₃ by the data of “1110”, the corresponding three shape memory alloy lines shown in FIG. At the same time, the respective initial states are returned to, so that a reaction corresponding to the sum of the operations occurs.

 FIG. 5 is a processing flow chart of the embodiment.

This processing is executed by the processing apparatus shown in FIG. 2, and it is first determined whether or not the cursor holds the object n (graphic to be processed) on the CRT (three-dimensional positions match) (step 50). . If it is determined that they match, the hardness value data regarding the object n is read from the object shape information file (disk device 20 in FIG. 2) and output to the RS232C interface described above (step 51). This hardness value data is developed (parallelized) in the above-mentioned feedback control section (Fig. 3) (step 52), and the relay is turned ON only in the part where the relay input of the control section is "1" (step 53), and the relay is relayed. The main part is activated when is turned on.

In the above description of the embodiments, the shape memory alloy is used as the material of the main body and the hardness is sensed by using the four wires, but other similar elements can be used. It is also clear that a number of lines other than four can be used. In the simplest form, the main body can be composed of only one line as long as the user simply feels the contact. Further, although the shape memory alloy wire is used in the above-mentioned embodiment, it is also possible to use another shape of alloy such as a thin plate.

The application of the present invention can be considered in various fields. CAD for example
When virtually constructing a structure on a CRT display in, you can get a feeling of actual operation of grasping and carrying the components of the structure, and you can feed back the hardness of the components, so that you can Accurate construction is possible with the lower part on the soft part and the upper part on the soft part.

Moreover, in the example of software design, in so-called visual programming (Visual Programming),
It can be used when constructing the entire software by combining the program modules expressed in graphic form. In that case, for example, it is possible to represent the amount of the source code of the module in a graphic form, the function of the module in color, and the reliability of the module in hardness. According to this method, more information about the software design can be obtained at the same time.

[Effects of the Invention] According to the present invention, when a cursor comes into contact with an object on the screen of the display device, the contact feeling can be fed back to the operator's hand. Further, by setting the hardness of the object, it is possible to give the operator a feeling of hardness / softness. Further, it is possible to give a feeling of reality to the operation on the screen of the display device and improve the operability of the three-dimensional graphic.

[Brief description of drawings]

FIG. 1 is a basic configuration diagram of the present invention, FIG. 2 is a system configuration diagram of an embodiment of the present invention, FIG. 3 is an embodiment configuration diagram of a feedback control unit, and FIG. 4 is an embodiment configuration diagram of a main body unit, FIG. 5 is a processing flow chart of the embodiment. In FIG. 1, 10: operation input control unit 101: operating means 11: processing device 12: display unit 13: storage unit 14: touch feeling feedback control unit 15: main body unit

Claims (1)

[Claims] 1. A contact notification method of a three-dimensional graphic in which a plurality of events are displayed as a three-dimensional graphic on a display screen of a display unit, and the three-dimensional position of the hand attached to the operator's hand is changed according to the movement of the hand. An operation means (101) for generating a signal representing the movement of the hand and a signal representing the movement of the hand indicating the processing content, and a signal representing the three-dimensional position of the hand from the operation means, and generating corresponding three-dimensional coordinate data. At the same time, an operation input control section (10) for generating information indicating an operation instruction from a signal for instructing the processing content, and a cursor at a position on the display screen corresponding to the three-dimensional coordinate data generated by the operation input control section. When it is detected that the spatial position of the cursor corresponding to the movement of the hand wearing the operating means and the spatial position of the three-dimensional figure on the display screen to be selected coincide with each other, the three-dimensional figure is displayed. A processing device (11) for extracting data representing the hardness of an object and generating it as output data, a contact feeling feedback control section (14) for receiving the output data and generating a corresponding control signal, the contact feeling feedback control A body portion (15) made of a shape memory alloy that deforms into a memorized shape by heat generated in response to a control signal from the section and acts on the hand of an operator who operates the operating means, The three-dimensional graphic tactile feedback device, wherein the main body is integrally formed with the operating means that is attached to an operator's hand and operated by the movement of the hand.