JP3443174B2 - Eye pointer - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye pointer for detecting the direction of a line of sight from an eyeball.

[0002]

2. Description of the Related Art Conventionally, attempts have been made to detect the line-of-sight direction of a human and utilize this line-of-sight direction for machine operation. That is, if the line-of-sight direction is used, a short-circuited intention can be transmitted to the machine rather than a key input by hand. An example thereof is application to an autofocus mechanism of a camera. Further, with regard to computer software, the input operation has been simplified, and more visual methods have been adopted.

Conventionally, input to a computer is performed by a keyboard, a mouse, a pen, etc., all of which are input by a worker's hand. Instead of inputting by this hand, the input is made by line-of-sight detection. Attempted to do.

For example, as shown in FIG. 4, a display
A CCD camera 13 is attached to an eye 12 of an operator who visually observes 11.
Light is emitted from the light emitting diode 14 provided at the tip of the
The reflected light is captured by the CCD camera 13, the image captured by the CCD camera 13 is sent to the computer 15, and the computer 15 detects the visual position of the operator. That is, the computer 15 obtains the center of gravity of the black eye of the eyeball 12 and the reflection virtual image point of the light emitting diode 14 in the input image, and calculates the position of the gazing point on the display 11 of the operator. In this calculation, the operator's line of sight is calibrated in advance and non-linear processing is performed.

However, in the method shown in FIG. 4, the CCD
Since the position of the eyes with respect to the camera 13 is used as a reference, the operator cannot freely move the face, which makes practical application difficult.

In order to solve such a problem, an apparatus which can be operated even if an operator freely moves his / her face is disclosed in, for example, Japanese Patent Application Laid-Open No. 4-281582. This device, called an eye pointer, is attached to the face of the worker, detects the direction of the worker's line of sight, and emits a light beam in the same direction as this line of sight from the face side. It will illuminate the gazing point of the line of sight. In this case, the gaze point of the worker is detected by directly detecting the irradiation point of the light beam by a sensor or detecting the reflected light of the light beam by a CCD camera, and the position information is transmitted to a computer. ing.
Since this eye pointer detects the gaze point regardless of the position of the face of the worker as described above, the worker can freely move the face.

The device disclosed in Japanese Patent Laid-Open No. 4-281582 will be described with reference to FIG. As shown in FIG. 5, the rotation center of the operator's eyeball 12 is O, the center of curvature of the cornea 22 of this eyeball 12 is P, and the angle transmission with the point O1 as the center of rotation is transmitted in front of this eyeball 12. An arm 23 and an angle transmission arm 24 about the point O2 as a center of rotation are provided, and the angle transmission arm 23 and the tip portion Q of the angle transmission arm 24 are jointed to each other. Also, one of these angle transmission arms
23 is configured so that it can expand and contract in the length direction, and point O1
Is transmitted as a rotational movement about the point O2 of the other angle transmission arm 24 at a certain fixed ratio.

Further, a light source 26 for detecting the line of sight 25 is integrally provided at the left rotation end of one angle transmission arm 23. The light source 26 irradiates the eyeball 12 side with the line-of-sight detection light 27, the irradiation surface of which is the angle transmission arm 23.
Located on the vertical line of. Also, the other angle transmission arm 24
A light source 28 for line-of-sight display is integrally provided on the right end of the rotation end. The light source 28 irradiates the gazing point of the line of sight 25 with the line-of-sight light 29, which is a light beam, and the irradiation surface is located on the vertical line of the angle transmission arm 24.

Considering now that the line-of-sight detection light 27 irradiates the center of curvature P of the cornea 22 of the eyeball, the center of rotation O of the eyeball 12, the center of curvature P of the cornea 22 and the center of rotation of the angle transmission arm 23 are considered. A triangle ΔO2 O1 Q formed by the triangle ΔOO1 P formed by the point O1, the point O1 of the center of rotation, the tip Q of the connecting point of the angle transmission arms 23 and 24, and the point O2 of the center of rotation of the other angle transmission arm 24. The position of the tip Q of the connecting point is determined so that and are similar to each other. With this setting, the turning angle O1 O2 Q of the line-of-sight light 29 becomes equal to the rotation angle O1 OP of the eyeball 12, and the gaze point of the operator can be illuminated by the line-of-sight light 29.

In order to perform such an operation, the eyeball 12
Rotation of the eye is detected, and the angle transmission arms 23 and 24 are moved along with the eyeball 12.
Must be driven so that the direction of the line-of-sight light 29 follows the line of sight 25. Further, in order to detect the rotation of the eyeball 12,
A light position sensor (not shown) receives the reflected light of the line-of-sight detection light 27 from the cornea 22, detects whether the line-of-sight detection light 27 is incident on the center P of curvature of the cornea 22 from the light receiving position, and enters the center P of curvature. If not, the servo system is driven by the output of the optical position sensor so that the servo system is incident on the center of curvature P of the cornea 22.

Further, FIG. 6 shows a concrete example of the configuration based on the above-mentioned principle. At the left end of one angle transmission arm 23, a line-of-sight detection module 31 is integrally provided. The line-of-sight detection module 31 is supported by a rotary stage 32 that is rotatable in two axial directions and has a point O1 of the center of rotation shown in FIG. 5, and the line-of-sight detection module 31 has a housing shape. A light source 26 for detecting the line of sight is provided at a portion corresponding to the point O1 of the center of rotation.

Further, an image forming lens 34 is provided on the emission side of the light source 26.
And a half mirror 35 is provided. This half mirror 35 forms an angle of 45 ° with the optical axis,
The emitted light from 26 is transmitted as the line-of-sight detection light 27, and the reflected light from the eyeball 12 is branched downward. An optical position sensor 37 is provided on the side of the split light by the half mirror 35 via an imaging lens 36. Further, the optical position sensor 37 detects that the line-of-sight detection light 27 is the center of curvature P of the cornea 22 of the eyeball 12.
When the eye-gaze detection light 27 is not incident on the center of curvature P of the cornea 22, the signal corresponding to the deviation from the reference position is output.

This deviation signal is output to the servo circuit 39, and a linear motor for driving the angle transmission arm 23 in each of the two axial directions so that the deviation signal becomes zero.
Control 40.

A line-of-sight display module 42 is integrally provided at the right end of the other angle transmission arm 24. The line-of-sight display module 42 is supported by a rotary stage 43 that can rotate in two axial directions and has a point O2 of the center of rotation shown in FIG. The line-of-sight display module 42 has a cylindrical shape, and a light source 28 for line-of-sight display is provided at a portion corresponding to the point O2 of the center of rotation. Further, a lens 44 is provided on the emission side of the light source 28, and the light from the light source 28 is emitted as the line-of-sight light 29.

The line-of-sight detection module 31 and the line-of-sight display module 42 have rotary stages 32 and 43, respectively, and the extendable and retractable angle transmission arm 23 and the fixed-length angle transmission arm 24 are jointed to each other. Moreover, the angle transmitting arm 23 and the angle transmitting arm 24 are adapted to rotate in opposite directions. Therefore, the linear motor
By the servo drive by 40, the eye-gaze detection light 27 is always in the cornea
By controlling to irradiate the center of curvature P of 22,
The line-of-sight light 29 can be scanned in the same direction as the line-of-sight 25.

However, in the apparatus shown in FIG. 6, the line-of-sight detection module 31 and the line-of-sight display module 42 are formed as independent housings, respectively, and the angle transmission arms 23 and 24 are extendable and bendable. Since the transmission is performed, the device structure becomes complicated and the machine is likely to rattle. On the other hand, in consideration of the accuracy of line-of-sight expression and the servo system for line-of-sight detection, severe restrictions are required on the accuracy of rattling of the machine. In other words, it is desirable to eliminate rattling of the machine because a slight amount of rattling of the machine causes a large decrease in accuracy. However, the configuration shown in FIG. 6 causes rattling of the machine, resulting in high accuracy. Difficult to maintain.

The image forming lens 34 for the light source 26 for detecting the line of sight is, as described with reference to FIG.
Since it is provided inside 31, the image forming magnification on the eyeball 12 becomes large, and when the light source 26 is a finite area light source such as a light emitting diode, the light condensing property is deteriorated and the reflection detection light at the cornea 22 is generated. It spreads. For this reason, the light focusing property of the optical position sensor 37 also deteriorates, and it becomes difficult to configure the servo circuit 39 that responds to the visual axis detection output of the optical position sensor 37.

[0018]

As described above, in the conventional device, the rattling of the machine has a great influence on the accuracy and the light-collecting property is deteriorated, so that the line-of-sight detection becomes unclear, and the servo circuit has a structure. It has problems such as difficulty.

It is an object of the present invention to provide a high-precision eye pointer which has almost no influence of rattling of a machine and enables clear line-of-sight detection with good light-collecting properties.

[0020]

An eye pointer according to claim 1 projects detection light onto an eyeball through a predetermined optical path, receives reflected light from the eyeball through the predetermined optical path, A single housing having a line-of-sight detection unit for detecting the rotation direction and the amount of rotation of the eyeball at one end, and provided at the other end with a line-of-sight display unit for projecting a light beam as the line-of-sight light, A rotary stage that fixes the housing so as to be rotatable about two axes, a drive mechanism that drives the single housing according to the output of the line-of-sight detection unit, and a predetermined optical path between the line-of-sight detection unit and the eyeball. The image forming lens is installed independently of the single housing.

[0021] 請 Motomeko 2 Eye pointers described, in the eye pointer claim 1 Symbol placement, visual axis detection unit includes a light source for projecting a detection light, a half mirror for splitting the reflected light from the eyeball, the An imaging lens provided with an optical position sensor that outputs a deviation signal between the branched light and the reference point, and an imaging lens installed in a predetermined optical path forms an image of the light source on the center of curvature of the cornea of the eyeball. The light reflected by the cornea surface is imaged on the optical position sensor via the half mirror.

[0022]

The eye pointer according to claim 1 projects the line-of-sight detection light from the line-of-sight detection unit provided at one end of the single housing to the eyeball through a predetermined optical path, and reflects light from the eyeball. Is received by the line-of-sight detection unit via a predetermined optical path to detect the rotation direction and rotation amount of the eyeball. And
The drive mechanism is controlled based on the detected rotation direction and rotation amount, and the single housing fixed to the rotation stage capable of biaxial rotation is interlocked with the rotation operation of the eyeball. A line-of-sight display unit is provided at the other end of the single housing, and the light beam projected from the line-of-sight display unit illuminates the gazing point by the eyeball as the line-of-sight display light. Also, in the predetermined optical path,
Since the imaging lens is installed independently of the single housing, it is possible to set an appropriate imaging magnification and achieve a good visual axis detection function.

[0023] Eye pointer of 請 Motomeko second aspect, claim
In 1 Symbol mounting eye pointer, the light from the light source in the sight line detecting unit is emitted through the half mirror, is focused by an imaging lens in the optical path to the cornea center of curvature of the eye.
Further, the light reflected on the corneal surface is imaged on the optical position sensor by the imaging lens via the half mirror, and a visual axis detection signal corresponding to the rotation of the eyeball is obtained.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the eye pointer of the present invention will be described below with reference to the drawings.

In FIG. 1, reference numeral 51 denotes a single housing, which has a line-of-sight detecting section 52 at one end and a line-of-sight displaying section 53 at the other end. Further, the line-of-sight detection unit 52 projects the line-of-sight detection light 58 onto the cornea 22 of the eyeball 12 via a predetermined optical path 57 formed by a mirror 55 and a half mirror 56. In addition, the reflected light from the corneal surface of the eyeball 12 is passed through a predetermined optical path
It has a function of detecting the rotation direction and the rotation amount of the eyeball 12 by receiving light via 57. Furthermore, the line-of-sight display unit 53 projects in the line-of-sight direction as line-of-sight output light 62, which is a light beam, through an optical path 61 composed of mirrors 59 and 60.

The single housing 51 is biaxial (X-Y
Direction) and is fixed to a rotary stage 63, which is driven in each of XY directions by a drive mechanism 64 including a linear motor and a drive lever. In addition,
Only the drive mechanism 64 for one direction is shown, but X-
A drive mechanism 64 is also provided in each of the Y directions. These drive mechanisms 64 are controlled by a servo circuit 65 that responds to a detection signal which is a current signal of the line-of-sight detection unit 52.

Further, 66 is an imaging lens, and this imaging lens
66 is a predetermined optical path formed between the line-of-sight detection unit 52 and the eyeball 12.
It is installed in the housing 57 independently of the single housing 51.

As shown in FIG. 2, the line-of-sight detection unit 52 has a light source 67 for projecting the line-of-sight detection light 58, and a half mirror 68 is provided on the emission side thereof. The half mirror 68 forms an angle of 45 ° with the optical axis, and the light source 67
From the eyeball 12 and branches the reflected light from the eyeball 12 downward. Further, the imaging lens 66 in the predetermined optical path 57 shown in FIG. 1 causes the image of the light source 67 to pass through the eyeball 12 as shown in FIG.
An image is formed on the center of curvature P of the cornea 22 and reflected light from the surface of the cornea 22 is formed on the optical position sensor 70 via the half mirror 68.

As shown in FIG. 2, the line-of-sight expressing portion 53 has a light source 72 for the line-of-sight expressing light 62 and a collimator lens 73.

Here, the light branched by the half mirror 68 is incident on the optical position sensor 70, and a deviation signal between the incident light and the reference point is output as a detection signal. This detection signal is given to the servo circuit 65, and controls the drive mechanism 64 so that the reflected light from the eyeball 12 converges on the central position which is the reference point of the optical position sensor 70. In this case, the optical position sensor 70
The positional relationship between the light source 67 and the light source 67 for line-of-sight detection is
The virtual image position through the half mirror 68 at the center of is set so as to match the position of the light source 67.

FIG. 3 also illustrates the geometrical relationships of the device shown in FIG. The imaging magnification by the imaging lens 66 is
The distance from the center of rotation O of the eyeball 12 to the center of curvature P of the cornea 22 and the point O of the center of rotation of the single housing 51 by the rotating stage 63.
The ratio is 3 to the distance from the rotation center of the light source 67 for detecting the line of sight to the point Q1. As is clear from the figure, the line-of-sight light 62 is the actual line of sight of the operator on the screen 75.
Illuminates 25 gazing points.

Next, the operation of the above embodiment will be described.

The light from the light source 67 for detecting the line of sight provided in the single housing 51 passes through the half mirror 68 and then the rear mirror 55.
After being reflected by, the image is formed by the image forming lens 66, and reflected by the half mirror 56, it is converged on the center of curvature P of the cornea 22 of the eyeball 12.
Then, due to the projection of the line-of-sight detection light 58, reflection occurs on the surface of the cornea 22, and the vertically reflected light again returns to the predetermined optical path 57 and enters the line-of-sight detection unit 52 in the single housing 51. In this case, the return light is imaged by the imaging lens 66, reflected by the half mirror 68 shown in FIG.
Converge on top. The optical position sensor 70 outputs a detection signal based on the deviation between the incident point of the return light and the central position which is the reference point. This detection signal relates to the deviation direction and the deviation amount.

Further, the servo circuit 65 controls the drive mechanism 64 based on this detection signal, and rotationally drives the single casing 51 so that the return light comes to the central position of the optical position sensor 70. By this operation, the line-of-sight detection light 58 is always reflected vertically on the surface of the cornea 22. Further, the distance from the rotation center O3 of the single housing 51 to the point Q1 of the rotation center of the light source 67 for detecting the line of sight is
Since the distance between the center O of the eyeball 12 and the center of curvature P of the cornea 22 is set equal to the product of the reciprocal of the imaging magnification, the eyeball
The same amount of rotation as twelve rotations can be provided in a single housing 51.

Here, a line-of-sight display part 53 is provided on the other end side of the single housing 51, that is, on the side opposite to the line-of-sight detection part 52, and the line-of-sight display light 62 from the light source 72 thereof is reflected by the mirror 59. , 60, and the single casing 51 is rotated to follow the gaze point of the eyeball 12 for scanning. Note that FIG. 3 shows that the line-of-sight light 62 follows the gazing point of the line-of-sight 25 looking at the screen 75.

As a means for irradiating the computer display with this line-of-sight light 62 and transmitting the point-of-regard information to the computer, the display surface is captured by a CCD camera, and the center of light irradiation is input as position data. Good. Alternatively, a dedicated optical position sensor may be installed directly on the display surface.

Here, the line-of-sight detection unit 52 and the line-of-sight display unit 53 are collectively provided in the single housing 51, and as in the conventional case,
Since the angle transmission arm does not intervene between them, the structure is simplified, and the rattling of the machine hardly occurs, and high accuracy can be obtained.

Further, since the image forming lens 66 is provided independently of the single housing 51, the image forming magnification is set to a proper value as compared with the conventional one in which the image forming lens 66 is provided in the line-of-sight detection module. The value can be set, the light-collecting property on the optical position sensor 70 is improved, and high eye-gaze detection accuracy can be obtained.

The apparatus described above is a head mounted display (Head m display) which is used by being attached to a human head.
It can also be used as a line-of-sight detection module of the device. That is, the viewpoint of the eyeball 12 can be detected by detecting the rotation of the single housing 51.
In this case, since the line-of-sight detection only needs to know the relative position with respect to the head, it is not necessary to project the line-of-sight light 62 to the outside of the head mounted display, and a mechanism therefor can be omitted.

As a method for detecting the relative position of the line of sight in this head-mounted display, the above-mentioned single housing is used.
The position detection may be performed by using the two-dimensional optical sensor that is arranged opposite to the line-of-sight display side while using 51 as it is. Further, as another method, the line-of-sight display portion 53 provided on the other end side of the single housing 51 is omitted, and instead, the rotation angle of the single housing 51 is detected by using a displacement sensor. May be.

[0041]

According to the eye pointer of the first aspect, the rotation direction and the rotation amount of the eyeball are detected, and the drive mechanism is controlled based on the detected rotation direction and the rotation amount.
A single housing fixed to a rotating stage that can rotate about two axes is interlocked with the rotational movement of the eyeball, and the light beam projected from the line-of-sight display part of the single housing irradiates the gazing point by the eyeball as the line-of-sight light. However, since the imaging lens is installed in the predetermined optical path independently of the single housing, it is possible to set an appropriate imaging magnification and achieve a good line-of-sight detection function. Since the structure is simplified and the rattling of the machine is almost eliminated, the deterioration of accuracy due to the rattling of the machine can be prevented, and the imaging magnification for the eye-gaze detection light can be set appropriately, so that the eye-gaze detection can be enhanced. Can be done with precision.

[0042] According to the eye-pointer of 請 Motomeko 2, wherein,
In addition to the claim 1 Symbol mounting eye pointer, the light from the light source in the sight line detecting unit is emitted through the half mirror is imaged center of curvature of the cornea of the eye by an imaging lens of the predetermined light path. Further, the reflected light on the corneal surface is imaged on the optical position sensor through the half mirror by the imaging lens,
A line-of-sight detection signal corresponding to the rotation of the eyeball can be obtained.

[Brief description of drawings]

FIG. 1 is a perspective view showing the overall configuration of an embodiment of an eye pointer of the present invention.

FIG. 2 is a configuration diagram showing a structure inside a single housing which is a main part of the eye pointer.

FIG. 3 is a diagram showing a geometrical relationship of the above eye pointer.

FIG. 4 is an explanatory diagram showing a conventional example.

FIG. 5 is a principle view showing another conventional example.

FIG. 6 is an explanatory diagram showing a specific example of the same.

[Explanation of symbols]

12 eyeballs 22 cornea 51 single housing 52 Line-of-sight detection unit 53 Line of sight 57 predetermined optical path 58 Detection light 62 Line-of-sight light that is a light beam 63 rotation stage 64 drive mechanism 66 Imaging lens 67 Light source 68 half mirror 70 Optical position sensor

─────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-6-125875 (JP, A) JP-A-63-194634 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) A61B 3/00-3/16 G02B 7/28

Claims (2)

(57) [Claims] 1. A line-of-sight detection unit that projects detection light onto an eyeball through a predetermined optical path, receives reflected light from the eyeball through the predetermined optical path, and detects a rotation direction and a rotation amount of the eyeball. A single housing having at one end and a line-of-sight-exposure part for projecting a light beam as the line-of-sight-exposed light at the other end; and a rotary stage for fixing the single case so as to be rotatable about two axes. A drive mechanism that drives this single housing in accordance with the output of the line-of-sight detection unit, and an image formed independently of the single housing in a predetermined optical path between the line-of-sight detection unit and the eyeball. An eye pointer characterized by comprising a lens. 2. The line-of-sight detection unit includes a light source that projects detection light, a half mirror that branches reflected light from an eyeball, and an optical position sensor that outputs a deviation signal between the branched light and a reference point. An imaging lens installed in a predetermined optical path forms an image of the light source on a center of curvature of the cornea of the eyeball, and reflects light reflected on the corneal surface on the optical position sensor via the half mirror. claim 1 Symbol mounting eye pointer, characterized in that for imaging the.