JP7209310B2 - Input device and image display system - Google Patents

Description

The present invention relates to input devices and image display systems.

Conventionally, 3D stereoscopic televisions, head-mounted displays, and the like have been known as devices that output a virtual three-dimensional structure in a three-dimensional virtual space. Also, a virtual three-dimensional structure to be displayed in a three-dimensional virtual space on a 3D stereoscopic television or head-mounted display can be created by, for example, CAD (computer-aided design). As a data input device for processing the created virtual three-dimensional structure, a known mouse, keyboard, controller, etc. as disclosed in Patent Document 1 are generally used.

Japanese Patent Publication No. 2008-541268

However, existing input devices such as mice, keyboards, and controllers do not allow for delicate and intuitive input.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an input device and an image display system that allow delicate and intuitive input.

Such objects are achieved by the present invention described below.

(1) a first member having a first position reference portion and a second position reference portion spaced apart from each other;
a second member having a third position reference portion and a fourth position reference portion spaced apart from each other;
An input device, wherein information of a virtual plane determined by the first position reference portion, the second position reference portion, the third position reference portion, and the fourth position reference portion is generated.

(2) The input device according to (1) above, wherein the information includes at least one of the shape of the virtual surface, the inclination of the virtual surface, and the position of the virtual surface.

(3) an elastic layer defining a first surface to which an external force is input and a second surface opposite to the first surface;
a marker disposed on the elastic layer and displaced as the elastic layer deforms;
The input device according to (1) or (2) above, further comprising a detection unit positioned on the second surface side of the elastic layer and detecting deformation of the elastic layer based on displacement of the marker.

(4) The input device according to (3) above, wherein the markers include a first marker and a second marker that are displaced in the thickness direction of the elastic layer.

(5) The input device according to any one of (1) to (4) above, wherein the first member and the second member are separated.

(6) The input device according to any one of (1) to (4) above, wherein the first member and the second member are rotatably connected.

(7) The input device according to (6) above, wherein the first member and the second member can be moved toward and away from each other.

(8) The input device according to (6) or (7) above, wherein the first member and the second member can be combined and separated.

(9) The input according to any one of (1) to (8) above, wherein one of the first member and the second member is held in the user's right hand and the other is held in the user's left hand. Device.

(10) a substrate;
a first position reference portion and a second position reference portion provided on the base body and separated from each other;
a direction detection unit provided on the base for detecting a predetermined direction,
An input device, wherein information of a virtual plane defined by a virtual straight line defined by the first position reference portion and the second position reference portion and the predetermined direction is generated.

(11) The input device according to (10) above, wherein the predetermined direction is a vertical direction.

(12) the virtual processing tool has a contact portion that contacts the virtual object;
The input device according to (10) or (11), wherein the contact portion is defined by the virtual straight line.

(13) The input device according to (12) above, which has an elastic input portion that deforms the contact portion.

(14) the elastic input portion includes an elastic layer defining a first surface to which an external force is input and a second surface opposite to the first surface;
a marker disposed on the elastic layer and displaced as the elastic layer deforms;
The input device according to (13) above, further comprising: a detection unit positioned on the second surface side of the elastic layer and detecting deformation of the elastic layer based on displacement of the marker.

(15) The input according to any one of (1) to (14) above, wherein the virtual object generated by virtual reality or augmented reality can be processed by pressing the virtual surface against the surface of the virtual object. Device.

(16) The input device according to (15) above, wherein a virtual machining tool corresponding to the virtual surface is displayed in virtual reality or augmented reality.

(17) The input device according to (16) above, further comprising a notification unit that notifies of contact of the virtual machining tool with the virtual object.

(18) The input device according to (17) above, wherein the notification unit has a vibration unit.

(19) The input device according to any one of (1) to (18) above;
and an image display device for displaying an image.

According to the present invention, it is possible to generate a signal according to information on a virtual plane determined by the first position reference portion, the second position reference portion, the third position reference portion, and the fourth position reference portion. For example, one hand holds the first member, the other hand holds the second member, and the first and second members are twisted to deform the shape of the virtual surface. By moving the first member and the second member while maintaining the relative positional relationship between them, delicate and intuitive input can be performed.

Further, according to the present invention, the input signal is generated according to the information of the virtual plane determined by the virtual straight line determined by the first position reference portion and the second position reference portion and the predetermined direction detected by the direction detection portion. It has become so. Therefore, for example, by holding the base and changing the posture of the base or moving the base, delicate and intuitive input can be performed.

1 is a configuration diagram of an image display system according to a first embodiment of the present invention; FIG. 2 is a diagram showing an example of a virtual three-dimensional structure displayed on the image display device shown in FIG. 1; FIG. 1 is a perspective view of an input device; FIG. It is a figure which shows an example of the virtual surface formed by an input device. It is a figure which shows an example of the virtual surface formed by an input device. It is a figure which shows an example of the virtual surface formed by an input device. It is a figure which shows the processing method of a virtual three-dimensional structure. It is a figure which shows the processing method of a virtual three-dimensional structure. It is a figure which shows another processing method of a virtual three-dimensional structure. It is a figure for demonstrating the operation|movement of an alerting|reporting part. It is a figure for demonstrating the operation|movement of an alerting|reporting part. It is a figure for demonstrating the operation|movement of an alerting|reporting part. It is a figure which shows the modification of an alerting|reporting part. FIG. 10 is a plan view showing an input device according to a second embodiment of the invention; FIG. 15 is a cross-sectional view of an elastic input portion included in the input device shown in FIG. 14; FIG. 16 is a cross-sectional view showing an input method to the elastic input portion shown in FIG. 15; 16A and 16B are diagrams for explaining the function of the elastic input unit shown in FIG. 15; 16A and 16B are diagrams for explaining the function of the elastic input unit shown in FIG. 15; It is a cross-sectional view showing a modification of the elastic input portion. It is a cross-sectional view showing a modification of the elastic input portion. It is a cross-sectional view showing a modification of the elastic input portion. It is a top view which shows the input device of 3rd Embodiment of this invention. It is a top view which shows the input device of 4th Embodiment of this invention. It is a perspective view which shows the input device of 5th Embodiment of this invention. 25 is a side view of the input device shown in FIG. 24; FIG. 25 is a diagram showing an example of how to use the input device shown in FIG. 24; FIG. It is a cross-sectional view showing an input device according to a sixth embodiment of the present invention. FIG. 14 is a cross-sectional view showing an input device according to a seventh embodiment of the invention; FIG. 21 is a cross-sectional view showing an input device according to an eighth embodiment of the present invention; It is a cross-sectional view showing an input device according to a ninth embodiment of the present invention. FIG. 21 is a cross-sectional view showing an input device according to a tenth embodiment of the present invention; It is a cross-sectional view showing an input device according to an eleventh embodiment of the present invention. FIG. 21 is a cross-sectional view showing an input device according to a twelfth embodiment of the present invention; FIG. 22 is a cross-sectional view showing an input device according to a thirteenth embodiment of the present invention; FIG. 20 is a cross-sectional view showing an input device according to a fourteenth embodiment of the present invention; FIG. 20 is a cross-sectional view showing an input device according to a fourteenth embodiment of the present invention; FIG. 20 is a cross-sectional view showing an input device according to a fourteenth embodiment of the present invention; FIG. 20 is a plan view showing an input device according to a fifteenth embodiment of the present invention; It is a figure which shows an example of a virtual three-dimensional structure. It is a figure which shows the processing method of a virtual three-dimensional structure. It is a figure for demonstrating the operation|movement of an alerting|reporting part. It is a figure for demonstrating the operation|movement of an alerting|reporting part. It is a figure for demonstrating the operation|movement of an alerting|reporting part. FIG. 20 is a plan view showing an input device according to a sixteenth embodiment of the present invention; 45 is a side view of the input device shown in FIG. 44; FIG. 45 is a cross-sectional view showing an elastic input portion included in the input device shown in FIG. 44; FIG. It is a figure explaining the function of an elastic input part. It is a figure explaining the function of an elastic input part.

DETAILED DESCRIPTION OF THE INVENTION An input device and an image display system according to the present invention will now be described in detail based on preferred embodiments shown in the accompanying drawings.

<First embodiment>
First, an image display system according to the first embodiment of the present invention will be described.

FIG. 1 is a configuration diagram of an image display system according to the first embodiment of the present invention. FIG. 2 is a diagram showing an example of a virtual three-dimensional structure displayed on the image display device shown in FIG. FIG. 3 is a perspective view of the input device. 4 to 6 are diagrams each showing an example of a virtual surface formed by the input device. 7 and 8 are diagrams respectively showing a method of processing a virtual three-dimensional structure. FIG. 9 is a diagram showing another processing method for a virtual three-dimensional structure. 10 to 12 are diagrams for explaining the operation of the notification unit. FIG. 13 is a diagram showing a modification of the notification unit.

The image display system 100 shown in FIG. 1 has an input device 200 , an image display device 300 and a terminal 400 .

[Image display device]
The image display device 300 is a VR (virtual reality) head-mounted display fixed to the user's head, and a known device can be used. Also, the image display device 300 preferably has a position tracking function. Thereby, the position of the user in the space can be recognized, and the image of the image display device 300 can be changed (followed) according to the movement of the user. Therefore, the image display system 100 is more realistic.

As the image display device 300, a head-mounted display for AR (Augmented Reality) may be used. Further, the image display device 300 is not limited to a head-mounted display as long as it can display a virtual three-dimensional structure (virtual object), and may be a display installed around the user. Alternatively, a screen and a projector may be used.

[Terminal]
The terminal 400 is not particularly limited, and includes, for example, a personal computer, a tablet terminal, a work station, a game machine, a smart phone, and the like. Further, the terminal 400 uses a system such as CAD (computer-aided design), for example, to display a virtual three-dimensional structure X and a virtual three-dimensional structure for processing the virtual three-dimensional structure X on a screen 310 as shown in FIG. A processing tool 900 is displayed. Also, the terminal 400 moves the virtual processing tool 900 within the screen 310 based on the input signal from the input device 200 , and changes the shape of the virtual three-dimensional structure X according to the movement of the virtual processing tool 900 . Thereby, the user can confirm how the virtual three-dimensional structure X is processed by the virtual processing tool 900 .

[Input device]
The input device 200 is held and used by a user. As shown in FIG. 3, the input device 200 has a first member 210 and a second member 220 which are paired and separated from each other. One of the first member 210 and the second member 220 is held in the user's right hand, and the other is held in the user's left hand. In FIG. 3, the first member 210 is held with the right hand, and the second member 220 is held with the left hand.

Further, the first member 210 is provided with a first position reference portion 231 and a second position reference portion 232 which are set apart from each other, and the second member 220 is provided with a third position reference portion which is set apart from each other. A reference portion 233 and a fourth position reference portion 234 are provided. In the input device 200, information of the virtual plane F determined by the first position reference portion 231, the second position reference portion 232, the third position reference portion 233, and the fourth position reference portion 234 (hereinafter also referred to as "virtual plane information"). ) is generated. The virtual plane F defined by the first position reference portion 231, the second position reference portion 232, the third position reference portion 233, and the fourth position reference portion 234 is, for example, the first position reference portion 231, the second position reference portion 232, the third position reference portion 233 and the fourth position reference portion 234; It means a substantially rectangular imaginary plane F having corners of .

The image display system 100 has a generator 500 that generates virtual plane information. Such a generator 500 is built in the first member 210 or the second member 220, or built in the terminal 400 or the image display device 300, for example. Note that the generation unit 500 is built in the terminal 400 in this embodiment.

The virtual surface information generated by the generation unit 500 preferably includes at least one of the shape of the virtual surface F, the inclination of the virtual surface F, and the position of the virtual surface F, and includes all of them. is more preferable. Since such information is included in the virtual surface information, it is possible to process the virtual three-dimensional structure X as described later with higher accuracy.

The configuration of the first position reference unit 231 to the fourth position reference unit 234 is not particularly limited as long as each of them can detect the position in the real space. can be done. For example, each of the first position reference section 231 to the fourth position reference section 234 can be configured to include an infrared emitting section for emitting infrared rays. In this case, infrared rays emitted from each of the first position reference section 231 to the fourth position reference section 234 are detected by a plurality of infrared cameras arranged in the space, so that each of the first position reference sections 231 in the space to detect the position of the fourth position reference portion 234; Further, the first position reference part 231 to the fourth position reference part 234 are points (markers), respectively. The positions of the position reference portion 231 to the fourth position reference portion 234 may be detected.

Further, as the configuration of the first position reference unit 231 to the fourth position reference unit 234, for example, a device for detecting a position in space such as a GPS (Globle Positioning System) or a device for detecting a high altitude in a space such as an air pressure sensor. A device for detecting height (altitude), a triaxial acceleration sensor (a sensor capable of detecting acceleration in each of the mutually orthogonal X, Y, and Z axes) and a triaxial angular velocity sensor (mutually orthogonal A device for detecting tilt, such as a 6-axis motion sensor that combines sensors capable of detecting angular velocities around each of the X, Y, and Z axes, may be used, or two or more of these may be used. may be used in combination.

Here, the first member 210 has a substantially rectangular plate shape, and has a first facing surface 211 on its side surface that faces the second member 220 during use. Similarly, the second member 220 has a plate-like shape and has a second facing surface 221 on its side surface that faces the first member 210 during use. That is, the input device 200 is used with the first facing surface 211 of the first member 210 and the second facing surface 221 of the second member 220 facing each other. Therefore, the first member 210 and the second member 220 are arranged in a plane, and the user can easily imagine the virtual plane F. FIG. However, the shape of each of the first member 210 and the second member 220 is not particularly limited, and for example, does not have to be plate-like.

Further, when the first opposing surface 211 and the second opposing surface 221 are arranged to face each other, the direction in which the first and second opposing surfaces 211 and 221 are arranged is the direction in which the first opposing surface 211 of the first member 210 is arranged. A gripping portion 212 for gripping the first member 210 is provided at the end opposite to the second member 220, and the second member 220 is gripped at the end opposite to the second facing surface 221 of the second member 220. A gripper 222 is provided for holding. The gripping portion 212 is formed (shaped) so as to grip the first member 210 from the side opposite to the first facing surface 211 , and the gripping portion 222 grips the second member 220 opposite to the second facing surface 221 . It is formed (shaped) to be gripped from the side. Therefore, when the first member 210 is held with the right hand and the second member 220 is held with the left hand, the first opposing surface 211 and the second opposing surface 221 naturally face each other. With such a configuration, the operability of the input device 200 is improved.

Also, the first and second position reference portions 231 and 232 provided on the first member 210 are arranged along the first opposing surface 211 . That is, the line segment connecting the first and second position reference portions 231 and 232 is substantially parallel to the first opposing surface 211 . In particular, in this embodiment, the line segment connecting the first and second position reference portions 231 and 232 is substantially perpendicular to the thickness direction of the first member 210 . Similarly, the third and fourth position reference portions 233 and 234 provided on the second member 220 are arranged along the second facing surface 221 . That is, the line segment connecting the third and fourth position reference portions 233 and 234 is substantially parallel to the second opposing surface 221 . In particular, in this embodiment, the line segment connecting the third and fourth position reference portions 233 and 234 is substantially orthogonal to the thickness direction of the second member 220 . By arranging the first position reference portion 231 to the fourth position reference portion 234 in this manner, the user can easily imagine the virtual plane F. FIG.

Next, an example of how to use the image display system 100 (how to operate the input device 200) will be described. A method of designing an automobile using the image display system 100 will be described below, but the method of using the image display system 100 (the input method of the input device 200) is not particularly limited.

As shown in FIG. 2, a screen 310 displays a virtual three-dimensional structure X to be processed and a virtual processing tool 900 (for example, a “spatula”) corresponding to a virtual plane F for processing the virtual three-dimensional structure X. tool) is displayed. The virtual three-dimensional structure X is likened to industrial clay, and its position in the virtual space is fixed. Therefore, by moving the user, the virtual three-dimensional structure X can be viewed from the front, from the rear, or from the side. On the other hand, the virtual processing tool 900 has a plate-like main body 910 and a blade portion 920 (contact portion) provided on the main body 910 and brought into contact with the virtual three-dimensional structure X. FIG. Such a virtual processing tool 900 corresponds to the virtual plane F (displayed on the virtual plane F). ing. Therefore, the user can associate the input device 200 held by the user with the blade portion 920, thereby enabling more intuitive input (operation). In addition, by displaying the virtual processing tool 900 on the screen 310, the virtual three-dimensional structure X can be processed in a state (feeling) closer to the real space.

For example, as shown in FIG. 4, when the first member 210 and the second member 220 are positioned on the same plane, the virtual plane F becomes a rectangular flat plane, and the corresponding virtual machining tool 900 is also rectangular. In terms of shape, the blade portion 920 is linear. Note that this state is hereinafter also referred to as a “reference state”.

Further, as shown in FIG. 5, when the second member 220 is rotated backward from the reference state, the virtual plane F becomes a twisted curved plane. Therefore, the virtual machining tool 900 has a twisted shape from the reference state. Further, when the first member 210 and the second member 220 are separated from each other, the virtual plane F extends. Therefore, the virtual machining tool 900 has a shape extending from the reference state. Further, as shown in FIG. 6, when the first member 210 and the second member 220 are spaced apart in an arc shape (fan shape), the virtual plane F becomes a flat surface curved in an arc shape. Therefore, the virtual machining tool 900 has a shape curved in an arc from the reference state. By operating the first and second members 210 and 220 in this manner, the shape of the virtual processing tool 900 can be freely changed.

Such a virtual processing tool 900 is generated by the generation unit 500 that acquires the position information of each of the first position reference unit 231, the second position reference unit 232, the third position reference unit 233, and the fourth position reference unit 234. , is displayed on the screen 310 of the image display device 300 and is displaced within the screen 310 according to the displacement of the input device 200 .

In such a configuration, the virtual three-dimensional structure X can be processed by the blade portion 920 by bringing the blade portion 920 into contact with the virtual three-dimensional structure X. Specifically, for example, as shown in FIG. 7, when the cutting edge 920 of the virtual processing tool 900 is slid while being pressed against the corner of the virtual three-dimensional structure X, the corner can be scraped off. Also, at this time, the corners of the virtual three-dimensional structure X can be rounded by curving the blade portion 920 . Further, for example, as shown in FIG. 8, when the end 921 of the blade portion 920 is slid while being pressed against the surface of the virtual three-dimensional structure X, grooves and steps can be formed on the surface. Thus, by operating the virtual processing tool 900 using the input device 200 and processing the virtual three-dimensional structure X, an automobile can be designed. In particular, according to the input device 200, since the first and second members 210 and 220 are operated with both hands, the processing of the virtual three-dimensional structure becomes closer to the actual operation. Therefore, delicate and intuitive input can be performed, and the virtual three-dimensional structure X can be processed more precisely.

As another specific example, for example, the screen 310 displays a virtual machining tool 900 projecting downward from the virtual plane F and perpendicular to the virtual plane F, as shown in FIG. ing. Then, the virtual three-dimensional structure X may be processed by bringing this virtual processing tool 900 into contact with the virtual three-dimensional structure X. In this embodiment, the virtual machining tool 900 is arranged so as to protrude from the central portion of the virtual plane F, but the arrangement of the virtual machining tool 900 is not particularly limited. may be placed in the department.

Here, as shown in FIG. 3, the input device 200 is provided with an input section 290 having at least one button. The input unit 290 is not particularly limited. The configuration may include a button for rotating, a button for fixing the shape of the virtual plane F, and the like. By having such an input unit 290, processing of the virtual three-dimensional structure X can be performed more easily. Although the arrangement of the buttons of the input unit 290 is not particularly limited, it is preferable that the buttons be arranged at positions where the user can easily operate them while holding the first member 210 and the second member 220 . In this embodiment, three are arranged near the grip portion 212 of the first member 210 and three near the grip portion 222 of the second member 220, respectively, and can be operated with the thumb of each hand.

In addition, as shown in FIG. 3, the input device 200 is provided with a notification unit 250 . The notification unit 250 has a first notification unit 251 arranged on the first member 210 and a second notification unit 252 arranged on the second member 220. The first notification unit 251 and the second notification unit 252 The contact state of the virtual processing tool 900 with respect to the virtual three-dimensional structure X is notified by . Here, the "contact state" includes, for example, the contact strength and contact angle of the virtual processing tool 900 with respect to the virtual three-dimensional structure X. FIG. By having such a notification unit 250, the user can easily recognize the contact state of the virtual processing tool 900 with respect to the virtual three-dimensional structure X, and can process the virtual three-dimensional structure X as desired.

Here, the first notification section 251 has a vibration section 2511 and the second notification section 252 has a vibration section 2521 . These vibrating units 2511 and 2521 can each be composed of a vibrator built in, for example, a mobile phone (including a smart phone). By configuring the first notification unit 251 and the second notification unit 252 in this manner, the contact state of the virtual processing tool 900 with respect to the virtual three-dimensional structure X can be notified to the user more reliably.

In particular, in this embodiment, two vibrating portions 2511 are arranged so as to correspond to the first position reference portion 231 and the second position reference portion 232 . That is, one vibrating portion 2511 is arranged near the first position reference portion 231 , and the other vibrating portion 2511 is arranged near the second position reference portion 232 . Similarly, two vibrating portions 2521 are arranged so as to correspond to the third position reference portion 233 and the fourth position reference portion 234 . That is, one vibrating portion 2521 is arranged near the third position reference portion 233 , and the other vibrating portion 2521 is arranged near the fourth position reference portion 234 .

For example, as shown in FIG. 10, when the virtual processing tool 900 is pressed against the virtual three-dimensional structure X, the stronger the contact strength, the greater the vibration of the notification unit 250 as a whole. The contact strength can be obtained based on the deviation between the position of the input device 200 corresponding to the state in which the virtual processing tool 900 contacts the virtual three-dimensional structure X and the actual position of the input device 200 . That is, if the virtual processing tool 900 contacts the virtual three-dimensional structure X when the input device 200 is in the state indicated by the solid line in FIG. It can be said that the greater the amount of deviation in the contact direction (pressing direction), the greater the contact strength.

Further, for example, as shown in FIG. 11, when the virtual processing tool 900 is pressed against the virtual three-dimensional structure X, if the contact strength on the first member 210 side and the contact strength on the second member 220 side are equal, , the vibration intensity of the first notification unit 251 and the vibration intensity of the second notification unit 252 are made equal. Further, when the contact strength on the first member 210 side is greater than the contact strength on the second member 220 side, the vibration strength of the first notification unit 251 is made larger than the vibration strength of the second notification unit 252, and the first As the difference between the contact strength on the member 210 side and the contact strength on the second member 220 side increases, the difference between the vibration strength of the first notification unit 251 and the vibration strength of the second notification unit 252 increases. Conversely, when the contact strength on the first member 210 side is smaller than the contact strength on the second member 220 side, the vibration strength of the first notification unit 251 is made smaller than the vibration strength of the second notification unit 252, and the vibration strength of the second notification unit 252 is reduced. As the difference between the contact strength on the first member 210 side and the contact strength on the second member 220 side increases, the difference between the vibration strength of the first notification unit 251 and the vibration strength of the second notification unit 252 increases.

Further, for example, as shown in FIG. 12, when the virtual processing tool 900 is pressed against the corner of the virtual three-dimensional structure X, the blade portion 920 is evenly in contact with the corner of the virtual three-dimensional structure X, that is, When the angle θ1 between the blade portion 920 and the plane X1 is equal to the angle θ2 between the blade portion 920 and the plane X2, the vibration intensity of the first notification portion 251 and the vibration intensity of the second notification portion 252 are made equal. do. On the other hand, when the blade portion 920 is inclined toward the plane X1, that is, when θ1<θ2, the vibration intensity of the first notification portion 251 is made larger than the vibration intensity of the second notification portion 252, As the difference between θ1 and θ2 increases, the difference between the vibration intensity of the first notification portion 251 and the vibration intensity of the second notification portion 252 increases. Conversely, when the blade portion 920 is inclined toward the plane X2, that is, when θ1>θ2, the vibration intensity of the first notification portion 251 is made smaller than the vibration intensity of the second notification portion 252, and θ1 and As the difference from θ2 increases, the difference between the vibration intensity of the first notification unit 251 and the vibration intensity of the second notification unit 252 increases.

In this way, by changing the vibration intensity of the entire notification unit 250 according to the contact state, or by making a difference between the vibration intensity of the first notification unit 251 and the second notification unit 252, the user can sense the virtual three-dimensional structure. It becomes easier to recognize the contact state of the virtual processing tool 900 with respect to X, and the virtual three-dimensional structure X can be processed as desired.

Note that the number and arrangement of the vibrating portions 2511 and 2521 are not particularly limited. For example, vibrating portions 2511 and 2521 may be provided in an arrangement as shown in FIG.

<Second embodiment>
Next, an image display system according to a second embodiment of the invention will be described.

FIG. 14 is a plan view showing the input device according to the second embodiment of the invention. 15 is a cross-sectional view of an elastic input portion included in the input device shown in FIG. 14. FIG. 16 is a cross-sectional view showing a method of inputting to the elastic input portion shown in FIG. 15. FIG. 17 and 18 are diagrams for explaining the function of the elastic input section shown in FIG. 15, respectively. 19 to 21 are cross-sectional views showing modifications of the elastic input portion.

The image display system according to this embodiment is the same as the image display system of the first embodiment described above, except that the configuration of the input device is different.

In the following description, the image display system of the second embodiment will be described with a focus on the differences from the above-described embodiments, and the description of the same matters will be omitted. Moreover, in FIGS. 14 to 20, the same reference numerals are given to the same configurations as in the above-described embodiment.

As shown in FIG. 14 , the input device 200 of this embodiment has an elastic input section 600 having a different configuration from the input section 290 , apart from the input section 290 . Such an elastic input portion 600 may be provided on only one of the first member 210 and the second member 220, or may be provided on both. Note that, in this embodiment, the elastic input portion 600 is provided only on the first member 210 .

As shown in FIG. 15, the elastic input unit 600 includes a hollow housing 610, an elastic layer 620 arranged outside the housing 610 and arranged on the outer peripheral surface of the housing 610, and an elastic layer 620. a detection unit 630 that is arranged inside the housing 610 and detects deformation of the elastic layer 620; a light source 640 that illuminates the elastic layer 620; and a generation unit 650 .

The housing 610 supports the elastic layer 620 from inside. Further, the housing 610 has a hollow hemispherical shape and has an internal space S inside thereof. An elastic layer 620 is arranged on the outer peripheral surface of the housing 610, and a detection section 630 and a light source 640 are housed in the internal space S. Note that the shape of the housing 610 is not particularly limited, and can be appropriately set according to the application of the input device 200 .

Further, the housing 610 is made of a hard member, specifically, a member having such hardness that it is not substantially deformed when pressed by a finger. Further, the housing 610 has optical transparency. In this embodiment, the housing 610 is made of a resin material and is substantially colorless and transparent. Note that the housing 610 may be omitted if, for example, the elastic layer 620 can maintain a substantially constant shape without being supported by the housing 610 .

The elastic layer 620 functions as an input section operated by the user. For example, input to the elastic input unit 600 can be performed by pressing, pinching, or stretching the elastic layer 620 . Each of these motions is a motion that a person performs on a daily basis without being particularly conscious of it. In addition, the elastic layer 620 can give the user who touches the elastic layer 620 a sense of "touching something" by repulsion. Therefore, the user can intuitively input to the elastic layer 620 . In addition, since the user can freely determine the strength, direction, speed, etc., of input to the elastic layer 620 with delicate touch (sense), more detailed input is possible.

The elastic layer 620 is a layer defining an outer peripheral surface 620a (first surface) to which an external force is input and an inner peripheral surface 620b (second surface) opposite to the outer peripheral surface 620a. Such an elastic layer 620 has elasticity and can be elastically deformed. Specifically, the elastic layer 620 has such soft elasticity as to be elastically deformed when pushed, pulled, or pinched by the user.

In addition, the elastic layer 620 is located on the outer peripheral surface side of the housing 610 and arranged to cover the housing 610 . Therefore, it can be said that the elastic layer 620 is supported by the housing 610 from the inner peripheral surface 620b side. In addition, the elastic layer 620 has a shape that follows the outer shape of the housing 610, that is, has a hemispherical shape. In this way, by forming the elastic layer 620 into a hemispherical shape, that is, having a convexly curved portion, input to the elastic layer 620 can be performed more naturally. In addition, since the elastic layer 620 can be easily pulled and pinched, more delicate input can be performed accordingly.

A marker M that is displaced by deformation of the elastic layer 620 is arranged on the elastic layer 620 . This marker M is a detection target that can be detected by the detection unit 630 . The marker M has a first marker 621 , a second marker 622 and a third marker 623 that are shifted in the thickness direction of the elastic layer 620 . Also, the first marker 621, the second marker 622 and the third marker 623 are each dot-shaped, that is, point-shaped. Also, the first marker 621 , the second marker 622 and the third marker 623 have different distances from the housing 610 .

These first marker 621 , second marker 622 and third marker 623 are each displaced as the elastic layer 620 is deformed. Therefore, the deformation of the elastic layer 620 can be detected based on the displacements of the first marker 621, the second marker 622 and the third marker 623, and the input to the elastic layer 620 can be detected from the deformation of the elastic layer 620. can.

In addition, the elastic layer 620 is arranged on the outer periphery of the housing 610 and has a first marker 621 on the first elastic layer 624 (first marker arrangement layer), and is arranged on the first elastic layer 624 and has a second marker 622 on the first elastic layer 624 . a second elastic layer 625 (second marker placement layer), a third elastic layer 626 (third marker placement layer) disposed on the second elastic layer 625 and having third markers 623, and a third elastic layer and a protective layer 627 disposed over 626 .

The first elastic layer 624 has optical transparency and elasticity (resilience). A plurality of first markers 621 are spaced apart from each other on the surface of the first elastic layer 624 . In addition, the second elastic layer 625 has optical transparency and elasticity (restoring force). A plurality of second markers 622 are spaced apart from each other on the surface of the second elastic layer 625 . In addition, the third elastic layer 626 has optical transparency and elasticity (restoring force). A plurality of third markers 623 are spaced apart from each other on the surface of the third elastic layer 626 .

In this embodiment, the first, second and third elastic layers 624, 625 and 626 are each substantially colorless and transparent. However, at least one of the first, second, and third elastic layers 624, 625, and 626 may be colored and transparent, for example, as long as it has optical transparency.

The material constituting the first, second, and third elastic layers 624, 625, and 626 is not particularly limited, and examples thereof include polyurethane elastomer, styrene thermoplastic elastomer, olefin thermoplastic elastomer, and vinyl chloride thermoplastic Various thermoplastic elastomers such as elastomers, ester thermoplastic elastomers, amide thermoplastic elastomers, silicone thermoplastic elastomers, fluorine thermoplastic elastomers, acrylic rubbers, silicone rubbers, butadiene rubbers, styrene rubbers, etc. Rubber materials and the like can be mentioned, and one or more of these can be used in combination (for example, as a laminate of two or more layers).

Also, the elastic moduli (Young's moduli) of the first, second, and third elastic layers 624, 625, and 626 are not particularly limited, and may be equal to or different from each other. When the elastic moduli of the first, second, and third elastic layers 624, 625, and 626 are different, for example, the Young's modulus of the first elastic layer 624 is set to E1, and the Young's modulus of the second elastic layer 625 is set to E2. , where E3 is the Young's modulus of the third elastic layer 626, it may be designed to satisfy the relationship of E1<E2<E3, or conversely, to satisfy the relationship of E1>E2>E3. You may

For example, when designed to satisfy the relationship E1>E2>E3, an input of a weak force F1 substantially deforms only the softest third elastic layer 626, and an input of a force F2 larger than the force F1 causes deformation. , only the second and third elastic layers 625, 626 are substantially deformed, and upon input of a force F3 greater than the force F2, the first, second and third elastic layers 624, 625, 626 are deformed. can be realized. In this case, the input intensity can be roughly detected by detecting which elastic layer's marker M has been displaced.

Also, the first, second, and third markers 621 , 622 , 623 are different from each other in at least one of shape and color, and are identifiable by the detection unit 630 . In this embodiment, the first, second, and third markers 621, 622, and 623 have different colors. For example, the first marker 621 is red, the second marker 622 is green, and the third marker 623 is blue. It has become. If the shapes of the first, second, and third markers 621, 622, and 623 are different from each other instead of the colors, for example, the first marker 621 is circular, the second marker 622 is triangular, and the third marker 623 is It can be rectangular. Of course, both the colors and shapes may be made different, or the first, second, and third markers 621, 622, and 623 may be made distinguishable by the detection unit 630 by a method other than making the colors and shapes different.

Also, the first, second, and third markers 621, 622, and 623 are preferably arranged so as not to overlap each other when viewed from the camera 631 of the detection unit 630 in the natural state. That is, in the image captured by the camera 631, the second marker 622 is not hidden by the first marker 621 in front of it, and the third marker 623 is hidden by the first and second markers 621 and 622 in front of it. It is preferably arranged so as not to Accordingly, the displacement of the first, second and third markers 621 , 622 and 623 at the time of input to the elastic layer 620 can be captured more accurately by the camera 631 . Therefore, the signal generator 650 can generate a more accurate input signal. The "natural state" refers to, for example, a state in which the user does not touch the elastic layer 620 (in other words, a state in which no external force other than gravity is applied).

Note that the configurations of the first, second, and third markers 621, 622, and 623 are not particularly limited as long as they can be detected by the detection unit 630, respectively. For example, the first, second, and third markers 621, 622, and 623 are not limited to dot shapes, ie, point shapes, and may be linear shapes, surface shapes, three-dimensional shapes, and the like. Also, for example, the first, second and third markers 621, 622 and 623 may be attached to the surfaces of the first, second and third elastic layers 624, 625 and 626, respectively. Alternatively, the surfaces of the first, second, and third elastic layers 624, 625, and 626 may be printed using ink or the like. Further, for example, a configuration in which the marker M is arranged on a spherical sheet body that covers the elastic layer 620 may be used. Also, for example, the first, second and third markers 621, 622 and 623 may be embedded in the first, second and third elastic layers 624, 625 and 626, respectively.

The protective layer 627 mainly functions to protect the third elastic layer 626 . Also, the protective layer 627 has optical transparency and elasticity, like the first, second, and third elastic layers 624 , 625 , and 626 . In this embodiment, the protective layer 627 is colorless and transparent. However, the protective layer 627 may be colored and transparent. The constituent material of such a protective layer 627 is not particularly limited, but for example, materials similar to the constituent materials of the first, second, and third elastic layers 624, 625, and 626 described above can be used.

In the elastic input unit 600 as described above, input is performed by pressing, pinching, or stretching the elastic layer 620 . Since these actions are actions that humans perform in their daily lives, intuitive input is possible. Further, since the strength, direction, speed, etc. of input to the elastic layer 620 can be performed with a delicate touch (feeling), delicate input is possible. In this embodiment, the elastic input section 600 can be operated with the user's thumb. Using the thumb facilitates input to the elastic input unit 600 and enables more delicate input.

The detection unit 630 three-dimensionally detects deformation of the elastic layer 620 by stereo photography. Thus, by using the stereophotography method, the deformation of the elastic layer 620 can be detected relatively easily and accurately.

The detection unit 630 has a plurality of cameras 631 arranged inside the internal space S. As shown in FIG. Each camera 631 is arranged to face the elastic layer 620 and is capable of imaging the elastic layer 620 . Also, each part of the elastic layer 620 can be imaged by at least two cameras 631, so that each part of the elastic layer 620 can be recognized as a three-dimensional image, that is, as a stereo image. . The camera 631 is not particularly limited, and for example, a CCD camera, a CMOS camera, or the like can be used.

In this embodiment, two cameras 631 are used to perform three-dimensional image recognition of each part of the elastic layer 620. However, the present invention is not limited to this. A plurality of images may be acquired in a time division manner at 631 and three-dimensional image recognition of each portion of the elastic layer 620 may be performed using these images. With such a configuration, it is possible to reduce the size and cost of the imaging unit.

The detection unit 630 also has a processing unit 632 that performs three-dimensional image recognition of the elastic layer 620 based on image information from each camera 631 . The processing unit 632 includes, for example, a CPU 632a that controls each unit such as each camera 631, a memory 632b, a storage unit 632c such as a flash memory, etc., and is configured to execute a predetermined program (code). Note that the program may be stored in a storage medium, or may be downloaded from an external server, for example.

Here, as described above, input to the elastic input section 600 is performed by pushing, pinching, stretching, or pulling the elastic layer 620 . The elastic layer 620 is deformed according to this input, and the first, second and third markers 621, 622 and 623 are displaced along with the deformation. To give a specific example, for example, as shown in FIG. 16, when a certain portion of the elastic layer 620 is pushed and deformed, the first, second, and third markers 621 positioned immediately below and around it are deformed. , 622 and 623 are displaced according to the force received. The processing unit 632 detects such displacements of the first, second, and third markers 621, 622, and 623 by three-dimensional image recognition, and based on the detection results, determines the input position, input direction, input intensity, input Input information (contact input information) including velocity, input acceleration, etc. is obtained.

Since the first, second and third markers 621 , 622 and 623 are arranged on the elastic layer 620 in this way, the detection unit 630 can detect the first, second and third markers 621 , 622 and 623 Deformation of the elastic layer 620 can be detected more accurately based on the displacement. In particular, in this embodiment, the first, second, and third markers 621, 622, and 623 are shifted in the thickness direction of the elastic layer 620 and arranged. Therefore, the detection unit 630 can detect deformation at different positions in the thickness direction of the elastic layer 620, and can detect deformation of the elastic layer 620 in more detail.

Here, an example of a method for acquiring input information by the processing unit 632 will be briefly described. The storage unit 632c of the processing unit 632 stores the three-dimensional coordinates of each camera 631 in advance. Then, the processing unit 632 obtains images at the same time using the two cameras 631 used to detect the displacement of the predetermined marker M, and obtains the two-dimensional coordinates of the predetermined marker M in both images. Next, the processing unit 632 acquires the three-dimensional coordinates of the predetermined marker M based on the deviation of the two-dimensional coordinates of the predetermined marker M between the two images and the three-dimensional coordinates of each camera 631, and stores them in the storage unit 632c. . The processing unit 632 continuously performs this operation for each frame. Then, the processing unit 632 compares the previously acquired three-dimensional coordinates of the predetermined marker M with the three-dimensional coordinates of the predetermined marker M newly acquired this time, and calculates the displacement of the predetermined marker M that occurred between them. To detect. The processing unit 632 can obtain input information by performing such work for all the markers M. FIG. Note that the frame rate of each camera 631 is not particularly limited, and may be, for example, 15 frames/second, 30 frames/second, 60 frames/second, or the like.

Although the method of acquiring input information by the processing unit 632 has been described above, the method of acquiring input information is not limited to this. For example, if the above-described operation is performed for all markers M, the amount of processing becomes enormous, and depending on the performance of the processing unit 632, the processing may not be able to keep up. Also, depending on the application, it may be sufficient if rough input information is obtained. Therefore, in such a case, for example, the processing unit 632 may perform the above-described operation only for some markers M selected in advance.

Further, the processing unit 632 stores, for example, an image of each part of the elastic layer 620 in a natural state as a reference image, and compares the reference image with the image obtained as described above in real time to obtain a first image. , second and third markers 621 , 622 , 623 may obtain the above input information.

Note that the processing method in the processing unit 632 is not limited to the above-described method. By specifying the displacement of the three markers 621, 622, 623, the above input signals can also be generated.

The signal generator 650 receives the input information (contact input information) acquired by the detector 630 and generates an input signal (contact input signal) based on the received input information. Note that such a signal generator 650 may be included in the detector 630 .

In this embodiment, the processing unit 632 and the signal generation unit 650 are arranged in the input device 200, but are not limited to this. , may have it on the cloud.

Here, in this embodiment, each layer 624, 625, 626, 627 of the elastic layer 620 is colorless and transparent. Therefore, the camera 631 can image the external world of the elastic input section 600 via the elastic layer 620 . Therefore, the processing unit 632 can capture the movement of the user's finger, and input without touching the elastic layer 620 or input with the markers 621, 622, and 623 touched lightly to the extent that the markers 621, 622, and 623 are not displaced. It becomes possible. Specifically, for example, when the thumb is moved to stroke the elastic layer 620, the processing unit 632 can recognize the movement of the thumb and generate an input signal from input information based on the movement. In addition to the contact-type input as described above, such non-contact-type input becomes possible, thereby increasing the variety of inputs to the elastic input unit 600 and improving operability and convenience.

For example, the input signal based on the non-contact input is generated when the markers 621, 622, 623 overlapping the image-recognized fingers are not displaced, and the markers 621, 622, 623, 622, 623 Preferably, when 623 is displaced, it will only generate an input signal based on a touch-type input. This can reduce the generation of erroneous input signals. In addition, contact input and non-contact input may be performed at the same time. That is, for example, it may be possible to perform contact-type input with one finger while performing non-contact-type input with another finger.

Note that non-contact input is not essential and may not be possible. In this case, the protective layer 627 does not need to be optically transparent, so it does not have to be optically transparent. If the protective layer 627 does not transmit light, the light from the light source 640 does not leak out of the elastic input part 600, and the user does not feel dazzled.

A light source 640 can illuminate the elastic layer 620 from inside the housing 610 . As shown in FIG. 15, the light source 640 has at least one light emitting section 641 arranged within the internal space S. As shown in FIG. The light emitting unit 641 is not particularly limited, and for example, an LED can be used. Further, the light emitting section 641 may emit visible light, NIR light (near infrared light), or ultraviolet light. When the light emitting unit 641 that emits visible light is used, each camera 631 is configured to support visible light. , and when a device that emits ultraviolet light is used as the light emitting unit 641, each camera 631 may be configured to support ultraviolet light. In particular, since NIR light and ultraviolet light are invisible to the human eye, even if they leak outside through the elastic layer 620, the user will not feel dazzled. In particular, when each camera 631 emits ultraviolet light, the marker M may be a phosphor. Thereby, the marker M can be imaged more clearly by the camera 631 .

Note that the light source 640 may be omitted, for example, when the elastic layer 620 can be kept sufficiently bright so that the image can be recognized by light from the outside. Further, a luminance sensor may be arranged in the internal space S, and driving of the light emitting unit 641 may be controlled based on the brightness of the internal space S detected by the luminance sensor. As a result, for example, the interior space S can be maintained at a substantially constant brightness, and image recognition of the elastic layer 620 (marker M) by the detection unit 630 can be performed more stably.

The elastic input unit 600 has been described above. Next, an example of how to use the input device 200 having such an elastic input unit 600 will be described. 17 and 18 show a virtual processing tool 900 displayed on the screen 310 of the image display device 300. FIG. As shown in FIG. 17, in a normal state (a state in which no input is made from the elastic input section 600), the blade section 920 of the virtual processing tool 900 is linear. From this state, for example, as shown in FIG. 18, when the elastic input part 600 is pressed, the blade part 920 of the virtual processing tool 900 is deformed based on the input signal generated by the pressing. By bringing the blade portion 920 into contact with the virtual three-dimensional structure X in a state of being deformed into a desired shape, the virtual three-dimensional structure X can be processed following the shape of the blade portion 920 . Since the shape of the blade portion 920 is configured to be deformable by input to the elastic input portion 600 in this manner, the virtual three-dimensional structure X can be processed more delicately. In particular, when the elastic input portion 600 is pushed, the blade portion 920 is deformed so as to be recessed (see FIG. 18), and when the elastic input portion 600 is pinched, the blade portion 920 is deformed so as to protrude. By linking (corresponding to) the deformation of the elastic layer 620 and the deformation of the blade portion 920, more intuitive and delicate input becomes possible.

Such a second embodiment can also exhibit the same effect as the first embodiment described above.

Note that the configuration of the elastic input portion 600 is not limited to the configuration described above. For example, in the present embodiment, the elastic layer 620 has a first elastic layer 624, a second elastic layer 625, and a third elastic layer 626. 624 is provided, and for example, at least one of the second elastic layer 625 and the third elastic layer 626 may be omitted.

Also, for example, the elastic layer 620 may have a variable elastic force. The structure of the elastic layer 620 in this case is not particularly limited. A first marker 621 arranged, a second covering covering the first covering and having a second airtight space between the first covering and a second marker 622 arranged on the surface of the second covering. a third covering portion covering the second covering portion and having a third airtight space between itself and the second covering portion; a third marker 623 disposed on the surface of the third covering portion; a cover, and a protective covering portion having a fourth airtight space between the cover and the third covering portion; By supplying a gas such as air or rare gas to the space, the pressure of each space can be adjusted. Further, in this case, for example, a substantially non-stretchable string arranged in the first airtight space connects the housing 610 and the first covering part at several points, and the housing 610 It is preferable to suppress unintentional displacement of the first covering portion. The same is true for the second, third and protective coverings.

Further, in this embodiment, the marker M has the first, second, and third markers 621, 622, and 623 with different distances from the housing 610. For example, at least one of the first, second, and third markers 621, 622, 623 may be omitted.

In this embodiment, the first marker 621 is placed on the first elastic layer 624, the second marker 622 is placed on the second elastic layer 625, and the third marker 623 is placed on the third elastic layer 626. However, for example, as shown in FIG. 19, a plurality of markers M may be arranged regularly or irregularly within one elastic layer 620 so that the distances from the housing 610 are different.

Also, the elastic input section 600 may have a reference marker that is not displaced by deformation of the elastic layer 620 and that can be detected by the detection section 630 . Specifically, for example, as shown in FIG. 20, the elastic input portion 600 may have a reference marker 628 that does not substantially displace when the elastic layer 620 is pushed or pulled. In the configuration of FIG. 20, the reference marker 628 is arranged between the first elastic layer 624 and the housing 610 and on the outer peripheral surface of the housing 610. , and may be arranged on the inner peripheral surface of the housing 610, for example.

The reference marker 628 has a constant relative positional relationship with each camera 631 and functions as a reference when the detection unit 630 detects displacements of the first, second and third markers 621 , 622 and 623 . According to such a configuration, for example, the detection unit 630 detects the displacement of the first, second, and third markers 621, 622, and 623 with respect to the reference marker 628, thereby obtaining more accurate input information. can be done. Note that the reference marker 628 is, for example, the first, second, and third markers 621, 622 so that the detection unit 630 can distinguish (distinguish) from the first, second, and third markers 621, 622, and 623, respectively. , 623 in at least one of shape and color.

In addition, although the elastic layer 620 has the protective layer 627 in this embodiment, the protective layer 627 may be omitted as shown in FIG. In this case, the third marker 623 is positioned on the surface of the elastic layer 620 . That is, the elastic layer 620 has a third marker 623 as an exposed marker exposed on the surface of the elastic layer 620 . Since the surface of the elastic layer 620 is a place that directly receives an input, placing the third marker 623 at such a position enables more delicate input.

Also, when the third marker 623 is exposed on the surface of the elastic layer 620 , the third marker 623 preferably has a portion protruding from the surface of the elastic layer 620 . This makes it easier for the finger pressing the elastic layer 620 to get caught on the third marker 623 , and in particular, improves the responsiveness to input such as sliding the finger on the surface of the elastic layer 620 . Also, in this case, the Young's modulus of the third marker 623 is preferably larger than that of the third elastic layer 626 . That is, it is preferable that the third marker 623 be harder than the third elastic layer 626 . This makes it easier for the finger to get caught on the third marker 623, making the above-described effect more remarkable.

Further, in the present embodiment, the housing 610 is made of a hard member, but may be made of a soft member that bends at the time of input, for example. That is, the housing 610 may deform together with the elastic layer 620 when an input is applied to the elastic layer 620 . Thereby, the amount of displacement of each of the first, second and third markers 621, 622 and 623 can be increased. Therefore, the detection unit 630 can obtain more accurate input information.

Also, the input device 200 may have a tactile sense providing unit that provides a tactile sense to the finger performing input on the elastic layer 620 . This makes the feeling of input more realistic and enables more intuitive input. The structure of the tactile sensation providing unit is not particularly limited, but for example, a plurality of speakers are arranged in the housing 610, and ultrasonic waves (sound waves) are emitted from at least one of the speakers toward the input point of the elastic layer 620. may be configured to provide a tactile sensation to the finger performing input on elastic layer 620 by outputting . In particular, by outputting ultrasonic waves from a plurality of speakers and forming a focal point of the sound at an input point of the elastic layer 620, it is possible to more effectively provide a tactile sensation to the finger.

The input device 200 also has an image projection unit that projects image light from the inside of the housing 610 toward the inner peripheral surface of the housing 610 to display an image (video) visible from the outside of the elastic layer 620. You may have The image is not particularly limited, but includes, for example, an image that assists and guides input to the elastic layer 620 . This improves the operability of the elastic input portion 600 and facilitates input to the elastic layer 620 . Although the image projection unit is not particularly limited, for example, a liquid crystal projector, an optical scanning projector, or the like can be used.

<Third Embodiment>
Next, an image display system according to a third embodiment of the invention will be described.

FIG. 22 is a plan view showing the input device of the third embodiment of the invention.

The image display system according to this embodiment is the same as the image display system of the first embodiment described above, except that the configuration of the input device is different.

In the following description, the image display system of the third embodiment will be described with a focus on the differences from the above-described embodiments, and the description of the same matters will be omitted. Also, in FIG. 22, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

As shown in FIG. 22, in the input device 200 of the present embodiment, the first member 210 and the second member 220 are rotatably connected with the first facing surface 211 and the second facing surface 221 facing each other. are doing. Further, in the input device 200 of the present embodiment, the first member 210 and the second member 220 can approach and separate in the direction in which the first opposing surface 211 and the second opposing surface 221 are arranged. With such a configuration, for example, compared to the configuration in which the first member 210 and the second member 220 are separated as in the first embodiment, the first member 210 and the second member 220 are separated. The positional relationship can be easily grasped, and the processing of the virtual three-dimensional structure X is facilitated accordingly.

Such a third embodiment can also exhibit the same effect as the first embodiment described above.

<Fourth Embodiment>
Next, an image display system according to a fourth embodiment of the invention will be described.

FIG. 23 is a plan view showing an input device according to a fourth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system of the first embodiment described above, except that the configuration of the input device is different.

In the following description, the image display system of the fourth embodiment will be described with a focus on the differences from the above-described embodiments, and the description of the same items will be omitted. In addition, in FIG. 23, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

In the input device 200 of this embodiment, the first member 210 and the second member 220 can be combined and separated. As shown in FIG. 23, when the first member 210 and the second member 220 are combined, the first opposing surface 211 and the second opposing surface 221 face each other, and the imaginary plane F is flat. It's becoming According to such a configuration, by separating the first member 210 and the second member 220, it is possible to change the shape of the virtual plane F as described in the above-described first embodiment. By combining the member 210 and the second member 220, the shape of the virtual plane F can be fixed. Therefore, the input device 200 can cope with both the case where the virtual plane F is to be changed and the case where the virtual plane F is to be fixed, and the operability is improved.

The structure for uniting and separating the first member 210 and the second member 220 is not particularly limited. In addition, in this embodiment, a concave-convex fitting is used, and the protrusions 281 provided on the first and second opposing surfaces 211 and 221 and the protrusions 281 provided on the first and second opposing surfaces 211 and 221 and a concave portion 282 that can be fitted with.

Such a fourth embodiment can also exhibit the same effect as the first embodiment described above.

<Fifth Embodiment>
Next, an image display system according to a fifth embodiment of the invention will be described.

FIG. 24 is a perspective view showing an input device according to a fifth embodiment of the invention. 25 is a side view of the input device shown in FIG. 24. FIG. 26 is a diagram showing an example of how to use the input device shown in FIG. 24. FIG.

The image display system according to this embodiment is the same as the image display system of the first embodiment described above, except that the configuration of the input device is different.

In the following description, the image display system of the fifth embodiment will be described with a focus on differences from the above-described embodiments, and descriptions of the same items will be omitted. Also, in FIGS. 24 and 25, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

As shown in FIGS. 24 and 25, in the input device 200 of this embodiment, the first member 210 and the second member 220 are shaped to be grasped by one hand. An elastic input portion 600 is provided on each of the first member 210 and the second member 220 . Moreover, in the present embodiment, the elastic input portion 600 has a substantially spherical shape. Further, as shown in FIG. 25, a first position reference portion 231 and a second position reference portion 232 are arranged in the first member 210, and a notification portion 250 (first notification portion 251) is arranged. It is Although not shown, a third position reference portion 233 and a fourth position reference portion 234 are arranged in the second member 220 in the same manner as the first member 210, and a notification portion 250 (second A notification unit 252) is arranged.

Such a fifth embodiment can also exhibit the same effect as the first embodiment described above.

26, the input device 200 of the present embodiment is configured such that the first member 210 and the second member 220 are aligned (for example, the elastic input portions 600 are in contact with each other). It can be used by pulling 200 toward you. This movement is similar to how a spatula is moved, allowing for more intuitive input.

<Sixth embodiment>
Next, an image display system according to a sixth embodiment of the invention will be described.

FIG. 27 is a cross-sectional view showing an input device according to a sixth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the second embodiment described above, except that the configuration of the elastic input section of the input device is different.

In the following description, the image display system of the sixth embodiment will be described, focusing on differences from the second embodiment described above, and descriptions of the same items will be omitted. In addition, in FIG. 27, the same reference numerals are given to the same configurations as in the above-described embodiment.

As shown in FIG. 27, in the input device 200 of the present embodiment, the elastic layer 620 includes a light transmission layer 620A arranged on the outer peripheral surface of the housing 610 and a light reflection layer laminated on the surface of the light transmission layer 620A. 620B and . The light-transmitting layer 620A has light-transmitting properties, and in particular, is substantially colorless and transparent in this embodiment. On the other hand, the light reflecting layer 620B has a light reflectivity of reflecting light L, which will be described later. In addition, the light transmission layer 620A deforms together with the light reflection layer 620B when the input to the elastic input section 600 is performed, and has a function of allowing the shape change of the inner surface 620B' of the light reflection layer 620B.

The detection unit 810 three-dimensionally detects deformation of the elastic layer 620 by an optical probe method. By using the optical probe method, the deformation of the elastic layer 620 can be detected relatively easily and accurately. The detection unit 810 will be described below.

As shown in FIG. 27, the detection unit 810 detects a light source 811 that emits light L toward the inner surface 620B′ of the light reflecting layer 620B and the one-dimensional position of the spot light on the light receiving surface. It comprises an optical system 813 having a semiconductor position sensitive device 812 (PSD) consisting of a photodiode capable of transmitting light. Note that the light source 811 is not particularly limited, and for example, an LD (laser diode) can be used.

Light L emitted from the light source 811 is narrowed by the lens system 814 to form a light spot LS on the inner surface 620B' of the light reflecting layer 620B. This light spot LS is imaged on the surface of the semiconductor position detecting element 812 by the lens system 815 . In such a configuration, the amount of relative displacement Z1 between the optical system 813 and the light spot LS is observed as the amount of movement A of the image on the surface of the semiconductor position sensitive element 812. FIG. That is, the relative displacement amount Z1 can be obtained from the movement amount A, and the coordinate values of the portion of the inner surface 620B' where the light spot LS is formed can be obtained.

According to the detection unit 810 having such a configuration, for example, the coordinate values of each portion of the inner surface 620B′ in the natural state are stored as reference coordinate values, and the reference coordinate values and the coordinate values of each portion of the inner surface 620B′ are stored. , in real time, the deformation of the inner surface 620B' can be detected. Furthermore, the detection unit 810 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

As described above, since the detector 810 detects the deformation of the elastic layer 620 based on the deformation of the inner surface 620B', the light reflecting layer 620B is preferably thin. For example, the light reflecting layer 620B is preferably thinner than the light transmitting layer 620A. This allows the inner surface 620B' to be positioned closer to the surface of the elastic layer 620 (the surface on which the input is made), thereby allowing the inner surface 620B' to deform more accurately and to a greater extent in response to the input. Therefore, detection accuracy of the detection unit 810 is improved. For this reason, it can be said that the light reflecting layer 620B functions as the marker M in this embodiment.

Such a sixth embodiment can also exhibit the same effect as the above-described second embodiment. Note that the detection unit 810 may obtain the coordinate values of each portion of the inner surface 620B′ using one optical system 813, or may obtain the coordinate values of each portion of the inner surface 620B′ using a plurality of optical systems 813. good.

<Seventh embodiment>
Next, an image display system according to a seventh embodiment of the invention will be described.

FIG. 28 is a cross-sectional view showing an input device according to a seventh embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the sixth embodiment described above, except that the configuration of the detection unit of the input device is different.

In the following description, the image display system of the seventh embodiment will be described with a focus on differences from the above-described sixth embodiment, and the description of the same items will be omitted. In addition, in FIG. 28, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

The detection unit 820 of this embodiment three-dimensionally detects deformation of the elastic layer 620 by an optical probe method. As shown in FIG. 28, the detection unit 820 includes a light source 821 (for example, an LD) that emits light L toward the inner surface 620B′ of the light reflecting layer 620B, a lens system 822 that collects the light L, and a light source 821. It comprises an optical system 826 having a beam splitter 823 arranged between the lens system 822 , a detector 824 (photodiode) and a motor 825 for moving the lens system 822 .

Light L emitted from the light source 821 is collected by the lens system 822 to form a light spot LS on the inner surface 620B' of the light reflecting layer 620B. The light L reflected by the inner surface 620B' passes through the lens system 822, is reflected by the beam splitter 823, and forms an image. A detector 824 is arranged at the imaging point. Also, the lens system 822 is moved in the optical axis direction by a motor 825 so that the image forming point is always positioned on the detector 824 . Based on the amount of movement of the lens system 822 at this time, the coordinate values of the portion of the inner surface 620B' where the light spot LS is formed can be obtained.

According to the detection unit 820 having such a configuration, for example, the coordinate values of each portion of the inner surface 620B′ in the natural state are stored as reference coordinate values, and the reference coordinate values and the coordinate values of each portion of the inner surface 620B′ are stored. , in real time, the deformation of the inner surface 620B' can be detected. Furthermore, the detection unit 820 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

Such a seventh embodiment can also exhibit the same effect as the above-described sixth embodiment. Note that the detection unit 820 may obtain the coordinate values of each portion of the inner surface 620B′ using one optical system 826, or may obtain the coordinate values of each portion of the inner surface 620B′ using a plurality of optical systems 826. good.

<Eighth embodiment>
Next, an image display system according to an eighth embodiment of the invention will be described.

FIG. 29 is a cross-sectional view showing an input device according to an eighth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the sixth embodiment described above, except that the configuration of the detection unit of the input device is different.

In the following description, the image display system of the eighth embodiment will be described with a focus on differences from the above-described sixth embodiment, and the description of the same items will be omitted. Also, in FIG. 29, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

The detection unit 830 of the present embodiment three-dimensionally detects deformation of the elastic layer 620 by an optical probe method. As shown in FIG. 29, the detection unit 830 includes a light source 831 (for example, an LD) that emits light L toward the inner surface 620B′ of the light reflecting layer 620B, a lens system 832 that expands the light L into parallel light, and a lens A lens system 833 that collects the light L that has passed through the system 832, a polarizing beam splitter 834 positioned between the lens systems 832 and 833, and a λ/4 plate positioned between the polarizing beam splitter 834 and the lens system 833. 835, a wavefront splitting mirror 836 that splits the light L reflected by the inner surface 620B′, a first detector 837 (photodiode) that receives one light L split by the wavefront splitting mirror 836, and the other light L. an optical system 839 having a second detector 838 (photodiode) that

In such a configuration, when the inner surface 620B' is displaced from the focal position of the lens system 833, the reflected light flux changes and a difference in the light amount of the first and second detectors 837 and 838 is produced. Therefore, based on this difference, the coordinate values of the portion of the inner surface 620B' irradiated with the light L can be obtained.

According to the detection unit 830 having such a configuration, for example, the coordinate values of each portion of the inner surface 620B′ in the natural state are stored as reference coordinate values, and the reference coordinate values and the coordinate values of each portion of the inner surface 620B′ are stored. , in real time, the deformation of the inner surface 620B' can be detected. Furthermore, the detection unit 830 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

Such a seventh embodiment can also exhibit the same effect as the above-described sixth embodiment. Note that the detection unit 830 may obtain the coordinate values of each portion of the inner surface 620B′ using one optical system 839, or may obtain the coordinate values of each portion of the inner surface 620B′ using a plurality of optical systems 839. good.

<Ninth Embodiment>
Next, an image display system according to a ninth embodiment of the invention will be described.

FIG. 30 is a cross-sectional view showing an input device according to a ninth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the sixth embodiment described above, except that the configuration of the detection unit of the input device is different.

In the following description, the image display system of the ninth embodiment will be described with a focus on the differences from the sixth embodiment described above, and the description of the same items will be omitted. Also, in FIG. 30, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

The detection unit 840 of the present embodiment three-dimensionally detects deformation of the elastic layer 620 by a cross-sectional measurement method, particularly a light section method. Specifically, the inner surface 620B' is irradiated with a slit-shaped light L, and the deformation of the elastic layer 620 is detected based on the shape of the light L reflected on the inner surface 620B'. By using such a cross-sectional measurement method (optical section method), deformation of the elastic layer 620 can be detected relatively easily and accurately.

As shown in FIG. 30, the detection unit 840 includes a light source 841 that emits light L toward the inner surface 620B′ of the light reflecting layer 620B, a slit light forming unit 842 that forms the light L into a slit, An optical system having an imaging element 844 which is provided at an off-axis position and captures the slit-shaped light L reflected on the inner surface 620B′, and a lens system 845 positioned between the inner surface 620B′ and the imaging element 844. 846 is provided.

With such a configuration, the cross-sectional shape of the portion of the inner surface 620B' where the light L is captured can be obtained based on the image captured by the imaging device 844, that is, the shape of the light L reflected on the inner surface 620B'. Therefore, for example, the shape of the inner surface 620B' can be detected by scanning the entire inner surface 620B' with the slit-shaped light L and obtaining the cross-sectional shape of each portion of the inner surface 620B'.

According to the detection unit 840 having such a configuration, for example, the shape of the inner surface 620B′ in the natural state is stored as a reference shape, and the reference shape and the shape of the inner surface 620B′ are compared in real time to detect the inner surface 620B′. A deformation of 620B' can be detected. Further, the detection unit 840 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

Such a ninth embodiment can also exhibit the same effect as the above-described sixth embodiment. Note that the detection unit 840 may detect the shape of the entire inner surface 620B′ using one optical system 846, or may detect the shape of the entire inner surface 620B′ using a plurality of optical systems 846. good.

<Tenth Embodiment>
Next, an image display system according to a tenth embodiment of the invention will be described.

FIG. 31 is a cross-sectional view showing an input device according to the tenth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the sixth embodiment described above, except that the configuration of the detection unit of the input device is different.

In the following description, the image display system of the tenth embodiment will be described with a focus on differences from the sixth embodiment described above, and descriptions of the same items will be omitted. Also, in FIG. 31, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

The detection unit 850 of the present embodiment three-dimensionally detects deformation of the elastic layer 620 by a contour measurement method, particularly a moire topography method using moire fringes. Specifically, a moiré fringe image of the inner surface 620B' is acquired, and deformation of the elastic layer 620 is detected based on this moiré fringe image. By using such a contour line measurement method, the deformation of the elastic layer 620 can be detected relatively easily and accurately. The detection unit 850 will be briefly described below.

As shown in FIG. 31, the detection unit 850 includes a light source 851 that emits light L, an imaging device 852, and a grating 853 provided between the light source 851 and the imaging device 852 and the inner surface 620B′. A system 854 is provided.

In such a configuration, the portion where the light L from the light source 851 that has passed through the grating 853 and the portion that can be seen by the imaging device 852 through the grating 853 intersects is the portion that is imaged by the imaging device 852. A substantially contoured surface is formed on the surface, and moire fringes corresponding to the contoured surface appear in the image acquired by the imaging device 852 . If this image is a moiré fringe image, the shape of the inner surface 620B' can be obtained based on the moiré fringe image.

According to the detection unit 850 having such a configuration, for example, the shape of the inner surface 620B′ in the natural state is stored as a reference shape, and this reference shape is compared with the shape of the inner surface 620B′ obtained in real time. By doing so, the deformation of the inner surface 620B' can be detected. Further, the detection unit 850 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

Such a tenth embodiment can also exhibit the same effect as the sixth embodiment described above. Note that the detection unit 850 may detect the shape of the entire inner surface 620B′ using one optical system 854, or may detect the shape of the entire inner surface 620B′ using a plurality of optical systems 854. good.

<Eleventh Embodiment>
Next, an image display system according to the eleventh embodiment of the present invention will be described.

FIG. 32 is a cross-sectional view showing an input device according to the eleventh embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the sixth embodiment described above, except that the configuration of the detection unit of the input device is different.

In the following description, the image display system of the eleventh embodiment will be described with a focus on the differences from the sixth embodiment described above, and the description of the same items will be omitted. Moreover, in FIG. 32, the same reference numerals are given to the same configurations as in the above-described embodiment.

The detection unit 860 of this embodiment detects deformation of the elastic layer 620 by a pattern projection method. By using the pattern projection method, the deformation of the elastic layer 620 can be detected relatively easily and accurately. As shown in FIG. 32, the detection unit 860 includes an image projection unit 861 that projects a reference pattern made up of light L onto the inner surface 620B′, and an image projection unit 861 that projects a reference pattern onto the inner surface 620B′ from a position shifted from the optical axis of the image projection unit 861. and an optical system 863 having an imaging element 862 for imaging the reference pattern. With such a configuration, it is possible to detect the shape of the inner surface 620B' on which the reference pattern is projected, based on the shape of the reference pattern captured in the image acquired by the imaging element 862. FIG. Therefore, for example, the shape of the inner surface 620B' can be detected by projecting the reference pattern over the entire inner surface 620B'. The reference pattern to be projected onto the inner surface 620B' is not particularly limited, and for example, a lattice pattern in which parallel straight lines are spaced apart from each other can be used.

According to the detection unit 860 having such a configuration, for example, the shape of the inner surface 620B′ in the natural state is stored as a reference shape, and this reference shape is compared with the shape of the inner surface 620B′ obtained in real time. By doing so, the deformation of the inner surface 620B' can be detected. Further, the detection unit 860 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

Such an eleventh embodiment can also exhibit the same effect as the sixth embodiment described above. Note that the detection unit 860 may detect the shape of the entire inner surface 620B′ using one optical system 863, or may detect the shape of the entire inner surface 620B′ using a plurality of optical systems 863. good.

<Twelfth Embodiment>
Next, an image display system according to a twelfth embodiment of the invention will be described.

FIG. 33 is a cross-sectional view showing an input device according to a twelfth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the sixth embodiment described above, except that the configuration of the detection unit of the input device is different.

In the following description, the image display system of the twelfth embodiment will be described with a focus on the differences from the sixth embodiment described above, and the description of the same items will be omitted. In addition, in FIG. 33, the same reference numerals are assigned to the same configurations as in the above-described embodiment.

The detection unit 870 of this embodiment detects deformation of the elastic layer 620 by a phase shift method. By using the phase shift method, the deformation of the elastic layer 620 can be detected relatively easily and accurately. As shown in FIG. 33, the detection unit 870 captures an image projection unit 871 that projects the reference pattern onto the inner surface 620B′ and the reference pattern projected onto the inner surface 620B′ from a position shifted from the optical axis of the image projection unit 871. and an optical system 873 having an imaging device 872 that

In such a configuration, for example, as a reference pattern, a fringe pattern representing a sine wave with brightness values is projected onto the inner surface 620B′, and the image sensor 872 captures an image of the reference pattern projected on the inner surface 620B′. The reference pattern is shifted by π/2 and projected four times, and each time the image is captured by the imaging device 872 . From the four images thus obtained, the shape of the inner surface 620B' where the reference pattern is projected can be detected. Note that the reference pattern, the method of shifting the reference pattern, and the like are not particularly limited.

According to the detection unit 870 having such a configuration, for example, the shape of the inner surface 620B′ in the natural state is stored as a reference shape, and this reference shape is compared with the shape of the inner surface 620B′ obtained in real time. By doing so, the deformation of the inner surface 620B' can be detected. Furthermore, the detection unit 870 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620B'.

Such a twelfth embodiment can also exhibit the same effect as the sixth embodiment described above. Note that the detection unit 870 may detect the shape of the entire inner surface 620B′ using one optical system 873, or may detect the shape of the entire inner surface 620B′ using a plurality of optical systems 873. good.

Note that, from the sixth embodiment described above to the present embodiment, the method of detecting the change in shape of the inner surface 620B′ by an optical method and thereby detecting the input to the elastic input section 600 has been described. ' is not limited to the sixth embodiment to the present embodiment. That is, any method may be used as long as the shape of the inner surface 620B' can be detected. For example, if it is a so-called "point measurement type" method, in addition to the optical probe method described in the sixth, seventh, and eighth embodiments, an ultrasonic method using an ultrasonic probe and a magnetic method are used. A magnetic method or the like can be used. In addition, if it is a so-called "surface measurement type" method, the silhouette method, the optical envelope method, the cross-sectional measurement method including the light section method described in the ninth embodiment, the interference fringe method, the holography method, the above-described method, etc. A contour measurement method including the moire topography method of the tenth embodiment can be used.

<Thirteenth Embodiment>
Next, an image display system according to a thirteenth embodiment of the present invention will be described.

FIG. 34 is a sectional view showing an input device according to a thirteenth embodiment of the invention.

The image display system according to this embodiment is the same as the image display system according to the second embodiment described above, except that the configuration of the elastic input section of the input device is different.

In the following description, the image display system of the thirteenth embodiment will be described with a focus on the differences from the second embodiment described above, and the description of the same items will be omitted. In addition, in FIG. 34, the same reference numerals are given to the same configurations as in the above-described embodiment.

As shown in FIG. 34, the elastic layer 620 has a light transmission layer 620A arranged on the outer peripheral surface of the housing 610 and an image display layer 620C laminated on the surface of the light transmission layer 620A. The light-transmitting layer 620A has light-transmitting properties, and in particular, is substantially colorless and transparent in this embodiment. On the other hand, the image display layer 620C has light reflectivity, and is a layer on which an image is displayed by light from an image projection section 890, which will be described later. The light transmission layer 620A has a function of deforming together with the image display layer 620C at the time of input to the elastic input section 600 and allowing the shape change of the inner surface 620C' of the image display layer 620C.

The input device 200 of this embodiment also has an image projection unit 890 arranged inside the housing 610 . Although the image projection unit 890 is not particularly limited, for example, a liquid crystal projector, an optical scanning projector, or the like can be used.

A predetermined image is displayed on the inner surface 620C' of the image display layer 620C by the light from the image projection unit 890. FIG. In particular, in this embodiment, the image projection unit 890 displays the marker M on the inner surface 620C' of the image display layer 620C. That is, in the input device 200 of the present embodiment, the markers M are arranged on the elastic layer 620 by displaying the markers M on the inner surface 620C'. Accordingly, since the pattern of the marker M can be changed according to the purpose, the input device 200 can exhibit excellent convenience.

Since such markers M are displaced along with the deformation of the inner surface 620C', the detector 630 can detect the deformation of the inner surface 620C' by detecting the displacement of the markers M. Furthermore, the detection unit 630 can detect deformation of the elastic layer 620, that is, input to the elastic layer 620, based on the deformation of the inner surface 620C'.

Such a thirteenth embodiment can also exhibit the same effect as the above-described second embodiment.

<14th Embodiment>
Next, an image display system according to a fourteenth embodiment of the invention will be described.

35 to 37 are cross-sectional views showing the input device according to the fourteenth embodiment of the present invention.

The image display system according to this embodiment is the same as the image display system according to the second embodiment described above, except that the configuration of the elastic input section of the input device is different.

In the following description, the image display system of the 14th embodiment will be described with a focus on the differences from the second embodiment described above, and the description of the same items will be omitted. Further, in FIGS. 35 to 37, the same reference numerals are given to the same configurations as in the above-described embodiment.

As shown in FIG. 35, in the input device 200 of the present embodiment, the first marker 621 is arranged between the first elastic layer 624 and the second elastic layer 625 and has a film shape (sheet shape). . The second marker 622 is arranged between the second elastic layer 625 and the third elastic layer 626 and has a film shape (sheet shape). The third marker 623 is arranged between the third elastic layer 626 and the protective layer 627 and has a film shape (sheet shape). These first, second, and third markers 621 , 622 , 623 deform as the elastic layer 620 deforms.

Also, the first marker 621, the second marker 622, and the third marker 623 each reflect light of a specific wavelength and transmit light of other wavelengths. Also, the first marker 621, the second marker 622, and the third marker 623 have different wavelengths of reflected light. As such first marker 621, second marker 622 and third marker 623, for example, optical filters such as dichroic filters can be used.

Further, the input device 200 of this embodiment has the detection section 810 used in the sixth embodiment described above. 35 to 37, the detection unit 810 includes a first detection unit 810A that detects deformation of the first marker 621, a second detection unit 810B that detects deformation of the second marker 622, and a second detection unit 810B that detects deformation of the second marker 622. and a third detection unit 810C that detects deformation of the 3-marker 623 .

As shown in FIG. 35, in the first detector 810A, the light source 811 emits light L1 having a wavelength reflected by the first marker 621. As shown in FIG. On the other hand, the semiconductor position detection element 812 is provided with a bandpass filter (not shown) that transmits the light L1 and blocks the transmission of the light L2 and L3, which will be described later. Therefore, the first detection unit 810A can detect deformation of the first marker 621 using the light L1. For example, the first detection unit 810A stores the coordinate values of each part of the first marker 621 in the natural state as reference coordinate values, and detects the reference coordinate values and the coordinate values of each part of the first marker 621 in real time. , the deformation of the first marker 621 can be detected.

Further, as shown in FIG. 36, in the second detection section 810B, light L2 having a wavelength reflected by the second marker 622 is emitted from the light source 811. As shown in FIG. Note that this light L2 passes through the first marker 621 . On the other hand, the semiconductor position detecting element 812 is provided with a band-pass filter (not shown) that transmits the light L2 and blocks the transmission of the light L1 and L3. Therefore, the second detector 810B can detect deformation of the second marker 622 using the light L2. For example, the second detection unit 810B stores the coordinate values of each part of the second marker 622 in the natural state as reference coordinate values, and detects the reference coordinate values and the coordinate values of each part of the second marker 622 in real time. , the deformation of the second marker 622 can be detected.

Further, as shown in FIG. 37, in the third detector 810C, the light source 811 emits light L3 having a wavelength reflected by the third marker 623. As shown in FIG. Note that this light L3 passes through the first marker 621 and the second marker 622 . On the other hand, the semiconductor position detecting element 812 is provided with a bandpass filter (not shown) that transmits the light L3 and blocks the transmission of the light L1 and L2. Therefore, the third detector 810C can detect deformation of the third marker 623 using the light L3. For example, the third detection unit 810C stores the coordinate values of each part of the third marker 623 in the natural state as reference coordinate values, and detects the reference coordinate values and the coordinate values of each part of the third marker 623 in real time. , the deformation of the third marker 623 can be detected.

According to the detection unit 810 having such a configuration, the deformation of the first marker 621 detected by the first detection unit 810A, the deformation of the second marker 622 detected by the second detection unit 810B, and the deformation of the second marker 622 detected by the third detection unit 810C Deformation of the elastic layer 620 can be detected based on the detected deformation of the third marker 623 .

Such a 14th embodiment can also exhibit the same effect as the above-described second embodiment. Although the present embodiment includes three detection units 810A, 810B, and 810C, the configuration is not limited to this, and one detection unit 810 may be provided. In this case, for example, the light source 811 is configured to cyclically switch and emit the lights L1, L2, and L3, and the deformation of the first, second, and third markers 621, 622, and 623 is detected in a time division manner. You should do it.

Also, instead of the detection unit 810, for example, the detection units 820, 830, 840, 850, 860, and 870 described in the seventh to twelfth embodiments may be used.

<Fifteenth embodiment>
Next, an image display system according to a fifteenth embodiment of the present invention will be described.

FIG. 38 is a plan view showing an input device according to the fifteenth embodiment of the invention. FIG. 39 is a diagram showing an example of a virtual three-dimensional structure. FIG. 40 is a diagram showing a method of processing a virtual three-dimensional structure. 41 to 43 are diagrams for explaining the operation of the notification unit.

The image display system according to this embodiment is the same as the image display system of the first embodiment described above, except that the configuration of the input device is different.

The input device 700 is held and used by the user, as shown in FIG. The input device 700 has an elongated plate-shaped base 710 . The base 710 is, for example, grasped with both hands at both ends. The shape of the base 710 is not particularly limited, and may be, for example, a rod shape. Moreover, although not shown, the base 710 is preferably provided with a fall prevention portion, such as a strap through which the arm can be passed, which can prevent the base 710 from falling.

Further, the input device 700 includes a first position reference portion 720 and a second position reference portion 730 provided on the base 710 and separated from each other, and a direction detection portion 740 provided on the base 710 for detecting the predetermined direction Z. have. Then, information (hereinafter also referred to as “virtual surface information”) of the virtual plane F determined by the virtual straight line Lv determined by the first position reference portion 720 and the second position reference portion 730 and the predetermined direction Z is generated. It's like Note that the virtual straight line Lv can be defined as a virtual straight line connecting the first position reference portion 720 and the second position reference portion 730, and can also be defined as a virtual straight line connecting the first position reference portion 720 and the second position reference portion 730. It can be defined as almost parallel imaginary straight lines. Also, the virtual plane F can be defined as a virtual plane including, for example, the virtual straight line Lv defined as described above and an axis that intersects with the virtual straight line Lv and extends along the predetermined direction Z.

In particular, in this embodiment, the predetermined direction Z is the vertical direction. Therefore, the virtual plane F is composed of a plane parallel to the vertical direction, in other words, a plane including the vertical direction. By setting the virtual plane F in such a direction, as will be described later, the posture of the virtual machining tool 900 displayed on the screen 310 based on the virtual plane F becomes natural. It becomes easier to process the virtual three-dimensional structure X. Note that the predetermined direction Z is not limited to the vertical direction, and may be, for example, a horizontal direction or a direction inclined with respect to both the vertical direction and the horizontal direction.

The image display system 100 has a generator 500 that generates virtual plane information. The generation unit 500 is built in the base 710, or built in the terminal 400 or the image display device 300, for example. Note that the generation unit 500 is built in the terminal 400 in this embodiment. The virtual surface information generated by the generation unit 500 includes the tilt of the virtual surface F and the position of the virtual surface F. FIG.

The configurations of the first position reference unit 720 and the second position reference unit 730 are not particularly limited as long as their positions in the physical space can be detected. , a device that detects the height (altitude) in space such as an air pressure sensor, and a three-axis acceleration sensor (acceleration in each of the X, Y, and Z axes that are perpendicular to each other) sensor) and a 6-axis motion sensor that combines a 3-axis angular velocity sensor (a sensor that can detect angular velocity around each of the mutually orthogonal X, Y, and Z axes). or a point (marker) captured by each camera in space recognition technology using a plurality of cameras, or a combination of two or more.

Also, the first position reference portion 720 and the second position reference portion 730 are provided at both ends of the elongated base 710 . Therefore, the line segment connecting the first position reference portion 720 and the second position reference portion 730 (that is, the virtual straight line Lv) substantially coincides with the longitudinal direction (extending direction) of the base 710 . As will be described later, the virtual processing tool 900 displayed on the screen 310 based on the virtual plane F has a blade portion (contact portion) 920 that comes into contact with the virtual three-dimensional structure X and cuts the virtual three-dimensional structure X. , and the inclination of the blade portion 920 corresponds to the inclination of the virtual straight line Lv. Therefore, by aligning the virtual straight line Lv substantially with the longitudinal direction (extending direction) of the base 710 , the inclination of the base 710 in the longitudinal direction corresponds to the inclination of the blade portion 920 . Therefore, it becomes easier for the user to imagine the inclination of the blade portion 920 , and the user can more naturally process the virtual three-dimensional structure X in the screen 310 .

The direction detection unit 740 is not particularly limited as long as it can detect a predetermined direction Z (vertical direction in this embodiment). and Z-axis), or a triaxial angular velocity sensor (a sensor capable of detecting angular velocity around each of the mutually orthogonal X-, Y-, and Z-axes). ), or an axial motion sensor combining an acceleration sensor and an angular velocity sensor.

Next, an example of how to use the image display system 100 will be described. Although a method of designing an automobile using the image display system 100 will be described below, the method of using the image display system 100 is not particularly limited.

As shown in FIG. 39, a screen 310 displays a virtual three-dimensional structure X to be processed and a virtual processing tool 900 for processing the virtual three-dimensional structure X corresponding to the virtual plane F. . A virtual processing tool 900 is, for example, a tool such as a spatula. The virtual processing tool 900 has a plate-like main body 910 and a blade portion 920 (contact portion) provided on the main body 910 and brought into contact with the virtual three-dimensional structure X. FIG. A plate surface (principal surface) 911 of the main body 910 is parallel to the virtual plane F, and the position and inclination of the blade portion 920 correspond to the position and inclination of the virtual straight line Lv (extending direction of the base 710). Therefore, the user can associate the input device 700 held by the user with the blade portion 920, thereby enabling more intuitive input (operation). In addition, by displaying the virtual processing tool 900 on the screen 310, the virtual three-dimensional structure X can be processed in a state (feeling) closer to the real space.

The virtual three-dimensional structure X is likened to industrial clay, and its position in the virtual space is fixed. Therefore, by moving the user, the virtual three-dimensional structure X can be viewed from the front, from the rear, or from the side.

By bringing the blade portion 920 into contact with the virtual three-dimensional structure X, the virtual three-dimensional structure X can be processed by the blade portion 920 . Specifically, for example, as shown in FIG. 40, by operating the input device 700 and sliding the cutting edge 920 of the virtual processing tool 900 while pressing against the corner of the virtual three-dimensional structure X, the corner is scraped off. be able to. Thus, by operating the virtual processing tool 900 using the input device 700 and processing the virtual three-dimensional structure X, an automobile can be designed.

Here, as shown in FIG. 38, the input device 700 is provided with an input section 790 having at least one button. The input unit 790 is not particularly limited, and may include, for example, a cancel button for canceling processing of the virtual three-dimensional structure X, a change button for changing the type of the virtual processing tool 900, and the like. . By having such an input unit 790, the virtual three-dimensional structure X can be processed more easily. Although the arrangement of the buttons of the input unit 790 is not particularly limited, it is preferable that the buttons be arranged in places where the user can easily operate them while holding the base body 710 . In this embodiment, it is provided at a position where it is easy to operate with the thumb of each hand.

Further, as shown in FIG. 38, the input device 700 is provided with a notification section 750 . The notification section 750 has a first notification section 751 arranged at one end of the base 710 and a second notification section 752 arranged at the other end. The notification unit 750 is configured to notify the contact state of the blade part 920 with respect to the virtual three-dimensional structure X by the first notification unit 751 and the second notification unit 752 . The "contact state" includes, for example, the contact strength and contact angle of the blade part 920 with respect to the virtual three-dimensional structure X. The notification unit 750 makes it easier for the user to recognize the contact state of the blade part 920 with respect to the virtual three-dimensional structure X, so that the virtual three-dimensional structure X can be processed as desired.

Also, the first notification section 751 and the second notification section 752 each have a vibration section capable of vibrating. As these vibrating units, for example, a small vibrator built in a mobile phone (including a smart phone) can be used. Accordingly, the notification unit 750 can notify the user of the contact state by vibration. Therefore, it is possible to more reliably notify the user of the contact state. However, the configuration of the notification unit 750 is not particularly limited as long as it can notify the user of the contact state. good too.

For example, as shown in FIG. 41, when the blade portion 920 is pressed against the virtual three-dimensional structure X, the stronger the contact strength, the greater the vibration of the entire notification portion 750 . The contact strength can be obtained based on the deviation between the position of the input device 700 corresponding to the state in which the blade portion 920 contacts the virtual three-dimensional structure X and the actual position of the input device 700 . That is, if the blade portion 920 contacts the virtual three-dimensional structure X when the input device 700 is in the state indicated by the solid line in FIG. It can be determined that the greater the amount of deviation in the contact direction (pressing direction), the greater the contact strength.

Further, as shown in FIG. 42, when the blade portion 920 is pressed against the virtual three-dimensional structure X, the contact strength of the end portion on the first notification portion 751 side and the contact strength of the end portion on the second notification portion 752 side are are equal, the vibration intensity of the first notification unit 751 and the vibration intensity of the second notification unit 752 are equal.

Further, when the contact strength of the end portion on the first notification portion 751 side is greater than the contact strength of the end portion on the second notification portion 752 side, the vibration strength of the first notification portion 751 is greater than that of the second notification portion 752 . The difference between the vibration intensity of the first notification unit 751 and the vibration intensity of the second notification unit 752 increases as the difference between these contact strengths increases.

Conversely, when the contact strength of the end portion on the first notification portion 751 side is smaller than the contact strength of the end portion on the second notification portion 752 side, the vibration strength of the first notification portion 751 is greater than that of the second notification portion 752 . , and the greater the difference between these contact strengths, the greater the difference between the vibration strength of the first notification unit 751 and the vibration strength of the second notification unit 752 .

Further, for example, as shown in FIG. 43, when the blade portion 920 is pressed against the corner of the virtual three-dimensional structure X, when the blade portion 920 is evenly in contact with the corner of the virtual three-dimensional structure X, that is, when the blade When the angle θ1 between the portion 920 and the surface X1 is equal to the angle θ2 between the blade portion 920 and the surface X2, the vibration intensity of the first notification portion 751 and the vibration intensity of the second notification portion 752 are equal. .

Further, when the blade portion 920 is inclined toward the plane X1 side, that is, when θ1<θ2, the vibration intensity of the first notification portion 751 becomes larger than the vibration intensity of the second notification portion 752, and θ1 and θ2 , the difference between the vibration intensity of the first notification unit 751 and the vibration intensity of the second notification unit 752 increases.

Conversely, when the blade portion 920 is inclined toward the plane X2, that is, when θ1>θ2, the vibration intensity of the first notification portion 751 becomes smaller than the vibration intensity of the second notification portion 752. As the difference from θ2 increases, the difference between the vibration intensity of the first notification portion 751 and the vibration intensity of the second notification portion 752 increases.

In this manner, by changing the vibration intensity of the entire notification unit 750 or by varying the vibration intensity of the first notification unit 751 and the second notification unit 752 depending on the contact state, the user can sense the virtual three-dimensional structure. It becomes easy to recognize the contact state of the blade part 920 with respect to X, and the virtual three-dimensional structure X can be processed as desired.

Such a fifteenth embodiment can also exhibit the same effect as the first embodiment described above.

<Sixteenth Embodiment>
Next, an image display system according to the sixteenth embodiment of the present invention will be described.

FIG. 44 is a plan view showing an input device according to the sixteenth embodiment of the present invention; 45 is a side view of the input device shown in FIG. 44. FIG. 46 is a cross-sectional view showing an elastic input portion of the input device shown in FIG. 44. FIG. 47 and 48 are diagrams for explaining the function of the elastic input section.

The image display system according to this embodiment is the same as the image display system of the first embodiment described above, except that the configuration of the input device is different.

In the following description, the image display system of the 16th embodiment will be described with a focus on the differences from the above-described embodiments, and the description of the same matters will be omitted. Moreover, in FIGS. 44 to 48, the same reference numerals are given to the same configurations as in the above-described embodiment.

As shown in FIGS. 44 and 45, an input device 700 of this embodiment has an elastic input section 600 that deforms a blade section 920 of a virtual processing tool 900 in addition to the configuration of the fifteenth embodiment described above. there is The elastic input portion 600 is provided at the lower end portion of the base 710 and arranged to extend along the longitudinal direction (virtual straight line Lv) of the base 710 . In the real world, the elastic input section 600 is likened to the blade section 920 . According to such a configuration, the user can associate the input device 700 and the virtual processing tool 900 more intuitively, and the input device 700 can perform delicate and intuitive input.

As shown in FIG. 46 , the elastic input unit 600 includes a hollow housing 610 , an elastic layer 620 arranged outside the housing 610 , and an elastic layer 620 arranged on the outer peripheral surface of the housing 610 . a detection unit 630 that is arranged inside the housing 610 and detects deformation of the elastic layer 620; a light source 640 that illuminates the elastic layer 620; and a generation unit 650 . Note that the elastic input section 600 has the same configuration as that of the above-described second embodiment except that the elastic layer 620 has a different shape, so the description thereof will be omitted.

Next, how to use the elastic input section 600 will be described. For example, as shown in FIG. 47, when part of the elastic input part 600 is pushed, the blade part 920 of the virtual processing tool 900 is recessed and deformed according to the pushed position and displacement amount. Also, for example, as shown in FIG. 48, when a portion of the elastic input unit 600 is pinched (pulled), the blade portion 920 of the virtual processing tool 900 protrudes and deforms according to the pinched portion and the amount of displacement. . In this manner, the shape of the blade portion 920 can be freely deformed by input from the elastic input portion 600 . Further, by processing the virtual three-dimensional structure X with the blade portion 920 in a deformed state, more diverse processing can be performed.

Such a sixteenth embodiment can also exhibit the same effect as the first embodiment described above.

Although the input device and the image display system of the present invention have been described above based on the illustrated embodiments, the present invention is not limited to this, and the configuration of each section may be any configuration having similar functions. can be replaced with Also, other optional components may be added to the present invention. Further, each embodiment may be combined as appropriate.

DESCRIPTION OF SYMBOLS 100... Image display system 200... Input device 210... 1st member 211... 1st opposing surface 212... Gripping part 220... 2nd member 221... 2nd opposing surface 222... Gripping part 231... 3rd 1 position reference part 232...second position reference part 233...third position reference part 234...fourth position reference part 250...informing part 251...first informing part 252...second informing part 281... Projection 282 Concave portion 290 Input unit 300 Image display device 310 Screen 400 Terminal 500 Generation unit 600 Elastic input unit 610 Housing 620 Elastic layer 620A Light transmission Layer 620B... Light reflecting layer 620B'... Inner surface 620C... Image display layer 620C'... Inner surface 620a... Outer peripheral surface 620b... Inner peripheral surface 621... First marker 622... Second marker 623... Third 3 markers 624 first elastic layer 625 second elastic layer 626 third elastic layer 627 protective layer 628 reference marker 630 detector 631 camera 632 processor 632a CPU 632b Memory 632c Storage unit 640 Light source 641 Light emitting unit 650 Signal generation unit 700 Input device 710 Substrate 720 First position reference unit 730 Second position reference unit , 740... direction detection unit, 750... notification unit, 751... first notification unit, 752... second notification unit, 790... input unit, 810... detection unit, 810A... first detection unit, 810B... second detection unit, 810C... Third detector, 811... Light source, 812... Semiconductor position detector, 813... Optical system, 814... Lens system, 815... Lens system, 820... Detector, 821... Light source, 822... Lens system, 823... Beam Splitter 824 Detector 825 Motor 826 Optical system 830 Detector 831 Light source 832 Lens system 833 Lens system 834 Polarization beam splitter 835 λ/4 plate 836 Wavefront Split mirror 837 First detector 838 Second detector 839 Optical system 840 Detector 841 Light source 842 Slit light forming unit 844 Imaging device 845 Lens system 846 Optical system , 850... Detector, 851... Light source, 852... Imaging device, 853... Grating, 854... Optical system, 860... Detecting unit, 861... Image projecting unit, 862... Imaging device, 863... Optical system, 870... Detecting unit, 871... Image projection unit 872... Imaging device 873... Optical system 890... Image projection unit 900... Virtual processing tool , 910...main body, 911...plate surface, 920...blade part, 921...end, 2511...vibration part, 2521...vibration part, A...movement amount, F...virtual plane, L...light, L1...light, L2...light , L3...light, LS...light spot, Lv...virtual straight line, M...marker, S...internal space, X...virtual three-dimensional structure, X1...plane, X2...plane, Z1...relative displacement amount, θ1...angle, θ2...Angle

Claims (7)

a substrate;
a first position reference portion and a second position reference portion provided on the base body and separated from each other;
a direction detection unit provided on the base for detecting a predetermined direction;
an elastic input unit provided on the base and deforming a contact portion of the virtual machining tool with the virtual object;
a virtual straight line connecting the first position reference portion and the second position reference portion intersects the predetermined direction, and information of a virtual plane defined by the temporary straight line and the predetermined direction is generated;
The contact portion extends along the imaginary straight line,
The elastic input portion is provided on a base body and defines a first surface to which an external force is input and a second surface opposite to the first surface; an elastic layer;
a marker disposed on the elastic layer and displaced as the elastic layer deforms;
an input device, comprising: a detector arranged in the base, positioned on the second surface side of the elastic layer, and detecting deformation of the elastic layer based on displacement of the marker. 2. The input device according to claim 1, wherein said predetermined direction is a vertical direction. 3. The input device according to claim 1, wherein the virtual object generated by virtual reality or augmented reality can be processed by pressing the virtual surface against the surface of the virtual object. 4. The input device according to claim 3 , wherein a virtual processing tool corresponding to said virtual surface is displayed in virtual reality or augmented reality. 5. The input device according to claim 4 , further comprising a notification unit for notifying that said virtual machining tool has come into contact with said virtual object. 6. The input device according to claim 5 , wherein the notification section has a vibrating section. an input device according to any one of claims 1 to 6 ;
and an image display device for displaying an image.

Images