JP3523583B2 - Three-dimensional position / posture measuring apparatus and method, and recording medium - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for measuring a three-dimensional position and orientation required for inputting a three-dimensional position and orientation to an information processing device such as a computer or an amusement device.

[0002]

2. Description of the Related Art With the recent improvement in computer processing speed and the development of CG (computer graphics) technology, the number of applications involving three-dimensional input has been rapidly increasing. In particular, home entertainment equipment and VRML
The demand for a three-dimensional input device in an application on the Internet represented by (Virtual Reality Modeling Language) is remarkable. In the field of CG creation and CAD systems, a mouse, tablet, or the like is often used as an input device. Also,
Intuitive 3D position and position for the purpose of operating, confirming and correcting the created object model from any direction
The need for an inexpensive device that can input a posture has also become apparent.

At present, devices capable of three-dimensional input include a magnetic field measurement system, an optical stereo system, an acoustic system, and a joystick system. Both the magnetic field measurement method and the acoustic method perform desired coordinate measurement using a device that involves transmission and reception. The joystick method has a precise and mechanical movable part, and converts the operation information into three-dimensional position information. The optical stereo obtains three-dimensional position information from parallax information using a plurality of television cameras.

[0004] Further, an apparatus for determining the position and orientation of each light receiving element using four light receiving elements and a display unit has been proposed (see Japanese Patent Application Laid-Open No. 9-325007). This apparatus makes it possible to perform three-dimensional measurement with an inexpensive lens and a small number of light receiving elements by using the calculation method described in Japanese Patent Application Laid-Open No. 7-270124. This method is optical 3
Since dimension measurement is performed, stable three-dimensional input can be performed without being affected by a magnetic field, temperature, or the like.

[0005]

Conventionally, a magnetic field measuring system which is commercially available as the above-described three-dimensional input device measures fluctuations of a molten metal in a space, so that the effects of metal parts, electronic equipment, and geomagnetism are affected. receive. In addition, acoustic devices are affected by atmospheric pressure and temperature in order to measure the phase or arrival time of sound waves. In addition, both require a transmitting device and a receiving device, and the device scale becomes large. Furthermore, the joystick method has to perform parallel movement and rotational movement independently, and is not suitable for intuitive three-dimensional input.

A three-dimensional input device using four light receiving elements and a display unit has only a minimum number of necessary light receiving elements, so that three-dimensional measurement is impossible if even one detection point on the display deviates. Will be. Also,
The calculation method described in Japanese Patent Application Laid-Open No. 7-270124 discloses a method in which a display surface is directly opposed to a light receiving element, or a direction of a straight line connecting one of two directly facing pairs of four light receiving elements is on the projection surface side. However, when it becomes parallel to the display surface, there is a problem that the sensitivity is remarkably lowered and measurement becomes impossible, so that the use range is limited. Furthermore, since the interval of three-dimensional measurement cannot be performed unless all four light reception detections are completed, there is a problem that the interval is limited to the interval of the scan time of the display unit.

The present invention provides a three-dimensional input that is stable without being affected by a magnetic field, temperature, or the like, requires little installation space, and can be intuitively performed at arbitrary measurement time intervals and over a wide range. It is an object of the present invention to provide a three-dimensional position / posture measurement apparatus and method that can be performed.

[0008]

According to one aspect of the present invention, there is provided a three-dimensional apparatus for measuring a three-dimensional position and orientation of a light receiving device with respect to a display device that emits a bright spot. In the position / posture measuring device , a display device in which the bright spot is movable on a display surface of the display device, and a light source for condensing the bright spot on the display device and projecting the bright spot on a projection surface as spot light A light receiving device having a projection surface on which three or more light receiving elements capable of detecting the projection position of the projected spot light and a spot, a bright spot display position coordinate of the display device, and a spot light based on the light receiving element Associating means for associating the projected position coordinates of the display device, and the luminescent spot display position coordinates of the display device and the light receiving element associated with each other by the associating means.
Of the set of spot position projection coordinates of the spot light based on the
Yes a three-dimensional position calculating means for calculating the three-dimensional position and orientation of the light receiving device relative to the bright point moving screen of the display device from a set of Kutomo three pairs of bright spot display position coordinates and the projection coordinates And the three-dimensional position calculating means calculates the brightness of the display device.
Three-dimensional position of the light receiving device with respect to a point moving screen;
Using a translation matrix T and a rotation matrix R representing the posture,
From the center of the condensing part lens of the light receiving device to the light receiving element surface
Is the distance of f, and the n-th point on the display surface is
(Xpn _, Ypn _, 0) and its projected position on the light receiving element
Where (Xrn _, Yrn)

(Equation 4) 3D position and attitude based on the relationship
It is characterized by calculating .

With such a configuration, the three-dimensional position / posture calculation is performed by combining the projection positions of three or more arbitrary spot lights and the coordinates of the projection position of the spot light.
Intuitive three-dimensional input is possible over a wide range without being affected by the scanning interval of the display device.

The invention according to a second aspect is characterized in that the light receiving element capable of detecting the projection position of the spot light is a set of light receiving elements having a spot-shaped light receiving window.

According to a third aspect of the present invention, the light receiving element having the spot-shaped light receiving window is three light receiving elements, and the bright point display position coordinates and the projection position coordinates associated by the association means. Is three.

According to a fourth aspect of the present invention, the light receiving element having the spot-shaped light receiving window is five or more light receiving elements, and the bright spot display position coordinates and the projection thereof associated by the association means. The position coordinates are an arbitrary number of five or more.

According to a fifth aspect of the present invention, the associating means includes the step of projecting a bright point of the display device onto a position coordinate of a projected image on each light receiving element projected on the projection surface on the light receiving device. Means for associating time, means for recording the projection position coordinates and projection time, and correspondence position determination means for associating the luminescent spot display position coordinates at a plurality of times with the projection position coordinates in accordance with the recorded projection time. It is characterized by having.

The invention according to a sixth aspect is characterized in that the display device scans the bright spot and performs display.

The invention according to claim 7 is characterized in that a CRT display is used as the display device for scanning and displaying the bright spot.

The invention according to claim 8 is characterized in that a bright spot irradiating device and a projection plate are used as the display device for scanning the bright spot and performing display.

According to a ninth aspect of the present invention, as the light receiving device, a plurality of light converging portions having different optical axes and a projection surface are provided.

According to a tenth aspect of the present invention, the light receiving device has means for enabling the light receiving device to be attached to a moving body such as a robot or a vehicle, to a part of a body, or to be held by a part of a body. .

An eleventh aspect of the present invention is characterized in that the light receiving device is combined with means for converting a plane moving amount into a numerical value and converting the numerical value into two-dimensional information.

According to a twelfth aspect of the present invention, any one of a trackball, a mouse, and a laser sensor is used as the conversion means.

According to a thirteenth aspect of the present invention, the light receiving device is capable of freely moving the output signal transmission by wire or wireless.

According to a fourteenth aspect of the present invention, the light receiving element capable of detecting the projection position of the spot light has a planar light receiving portion and is a light receiving element capable of outputting the position coordinates of the spot light. And

According to a fifteenth aspect of the present invention, the associating means includes: a projection position coordinate and a projection time on each light receiving element of a spot light obtained by projecting a bright point of the display device on the projection surface of the light receiving device. Means for associating, the means for recording the projection position coordinates and the projection time, and a correspondence position determination means for associating the correspondence between the bright spot display position coordinates at a plurality of times and the projection position coordinates according to the recorded projection time. It is characterized by having.

The invention according to claim 16 is characterized in that a semiconductor position detecting element is used as the light receiving device.

According to a seventeenth aspect of the present invention, the display device scans the bright spot and performs display.

[0026] The invention according to claim 18 is characterized in that a CRT display is used as the display device for scanning and displaying the bright spot.

According to a nineteenth aspect of the present invention, the display device uses a bright spot irradiating device and a projection plate for scanning the bright spot and performing display.

According to a twentieth aspect of the present invention, as the light receiving device, a plurality of light converging portions having different optical axes and a projection surface are provided.

[0029] The invention according to claim 21 is characterized in that the light receiving device has a means for mounting on a moving body such as a robot or a vehicle, a part of the body, or a part of the body. .

The invention according to claim 22 is characterized in that the light receiving device is combined with a converting means for converting the amount of movement of a plane into a numerical value and converting it into two-dimensional information.

According to a twenty-third aspect of the present invention, any one of a trackball, a mouse, and a laser sensor is used as the conversion means.

According to a twenty-fourth aspect of the present invention, the light receiving device is capable of freely moving the output signal transmission by wire or wireless.

[0033] The invention according to claim 25, in the three-dimensional position and orientation measuring method for measuring a three-dimensional position and orientation of the light receiving device with respect to a display device which emits bright spot, said at the display surface of the display device The step of arbitrarily moving the luminescent spot, and the condensing part of the light receiving device condenses the luminescent spot on the display device, as a spot light on the projection surface of the light receiving device
Projecting, placed on the projection plane and projecting the spot light
Three or more light receiving elements capable of detecting the projection position of
Ri, a step of associating comprises obtaining a projected position coordinates of the spot light to be projected onto the projection plane, the projection position coordinates of the spot lights the display device bright spot display position coordinates to be based on the light receiving element, the correspondence Of the display device
Bright spot display position coordinates and spot light based on the light receiving element
Of the set of projection position coordinates, calculating the three-dimensional position and orientation of the light receiving device relative to the bright point moving screen of the display device from a set of at least three pairs of bright spot display position coordinates and the projection coordinates And the three-dimensional position / posture is measured.
Calculating based on a bright spot moving screen of the display device.
Parallel representing the three-dimensional position and orientation of the light receiving device
Using the movement matrix T and the rotation matrix R, condensing the light receiving device
F is the distance from the center of the lens of the part to the light receiving element surface
The n-th point on the display surface by (Xpn _, Ypn
_, 0) and its projected position on the light receiving element are (Xrn _, Y
rn)

(Equation 5) 3D position and attitude based on the relationship
It is characterized by calculating .

According to a twenty-sixth aspect of the present invention, the light receiving element capable of detecting the projection position of the spot light is a set of light receiving elements having a spot-shaped light receiving window.

According to a twenty-seventh aspect of the present invention, in the step of associating, the luminous point of the display device is projected to the position coordinates of a projected image on each light receiving element projected on the projection surface on the light receiving device. Associating a time, recording the projection position coordinates and the projection time, and associating a corresponding position according to the recorded projection times with the luminescent spot display position coordinates and the projection position coordinates at a plurality of times. It is characterized by having.

According to a twenty-eighth aspect of the present invention, the light receiving element capable of detecting the projection position of the spot light has a planar light receiving portion and is a light receiving element capable of outputting the position coordinates of the spot light. And

29. The invention according to claim 29, wherein the associating step comprises: projecting position coordinates and projection time on each light receiving element of a spot light obtained by projecting a bright spot of the display device on the projection surface of the light receiving device. Correlating the projection position coordinates and the projection time; and a correspondence position determination step of relating the luminous point display position coordinates and the projection position coordinates at a plurality of times according to the recorded projection time. It is characterized by having.

According to a thirtieth aspect of the present invention, there is provided a recording device which records a program for causing a computer to execute a three-dimensional position / posture measuring method for measuring a three-dimensional position and posture of a light receiving device with respect to a display device emitting a bright spot. A medium, the step of arbitrarily moving the luminescent spot on the display surface of the display device, and a light-collecting unit of the light-receiving device condenses the luminescent spot on the display device, and forms the spot light. Of the receiver
Projecting on a projection surface, and arranged on the projection surface, and
3 or more light receivers capable of detecting the pot light projection position
Obtaining , by an element, projection position coordinates of a spotlight projected onto the projection surface; associating a luminescent spot display position coordinate of the display device with a projection position coordinate of the spotlight based on the light receiving element; The attached table
The coordinates of the bright spot display position of the display
Of the set of projection position coordinates of Tsu bets light, three-dimensional position and orientation of the light receiving device relative to the bright point moving screen of the display device from a set of at least three pairs of bright spot display position coordinates and the projection coordinates Calculating the three-dimensional position.
The step of calculating the posture includes the step of moving the bright spot of the display device.
Displays the three-dimensional position and orientation of the light receiving device with respect to the plane
Using the translation matrix T and the rotation matrix R
The distance from the center of the lens of the
As f, the n-th point on the display surface is (Xpn _,
Ypn _, 0) and its projected position on the light receiving element are (Xr
n _, Yrn)

(Equation 6) 3D position and attitude based on the relationship
The calculation is performed, and the respective steps are made to function.

According to a thirty-first aspect of the present invention, the light receiving element capable of detecting the projection position of the spot light is a set of light receiving elements having a spot-shaped light receiving window. Associating the bright point of the display device with the position coordinates of the projected image on each light receiving element projected onto the projection surface on the light receiving device and the projection time; The method includes a step of recording a time, and a step of determining a correspondence position in which the coordinates of the bright spot display position and the projection position coordinates at a plurality of times are associated with each other according to the recorded projection time.

The invention according to claim 32, wherein the spot is
The light receiving element that can detect the light projection position has a planar light receiving unit.
And a light receiving element capable of outputting the position coordinates of the spot light , wherein the associating step is such that the bright spot of the display device is projected on the projection surface of the light receiving device on each light receiving element of the spot light. Associating the projection position coordinates with the projection time, recording the projection position coordinates and the projection time, and associating the luminescent spot display position coordinates and the projection position coordinates at a plurality of times according to the recorded projection time. And a corresponding position determination step.

In the present invention, as a three-dimensional device / posture input device, only a semiconductor element capable of detecting the position coordinates of a projected image and a display device for moving or scanning a luminescent spot on a two-dimensional display surface are realized. In addition, a three-dimensional measurement can be stably performed without requiring much installation space and without being affected by a magnetic field, temperature, or the like. Also, by measuring the three-dimensional position / posture at three points using three light receiving elements, the three-dimensional position / posture can be measured at a minimum time interval. Further, the correspondence between the luminescent spot display position coordinates at any plural times and the projection position coordinates on the light receiving device is associated according to the recording time, and the arbitrarily associated plural luminescent spot display position coordinates and the projection position coordinates. Calculate the three-dimensional position and orientation of the light receiving device based on the bright spot moving surface or the bright spot scanning surface of the display device from the set of By performing three-dimensional position / orientation calculation using, the intuitive 3
It has the effect that dimensional input is possible.

[0042]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram for explaining an embodiment of a three-dimensional position / posture measuring apparatus according to the present invention.
1 is a raster scan display (hereinafter abbreviated as a display), 2 is a light receiving device, and 3 is a three-dimensional position output device.

The basic principle of the present invention is that a figure or the like of a scanning trajectory drawn on the surface of the display 1 is observed by the light receiving device 2, and the observation information and the drawing information are processed by the three-dimensional position output device 3. It outputs the position and orientation of the light receiving device 2.

FIG. 2 is a configuration diagram of a light receiving device according to the first embodiment of the present invention. In this embodiment, a plurality of light receiving elements that convert the intensity of light into an electric signal are used as the light receiving elements of the light receiving device 2. In the figure, A is a lens, B
Is a projection plane of light passing through the lens A, C is a center point where the lens optical axis and the projection plane B of light intersect, D is a light receiving element, and F is a display surface of a raster scan display. Light receiving element D
Are installed on the projection surface. The number of the light receiving elements D may be at least three, but five or more are arranged,
By measuring the three-dimensional position / posture using points or four points, a wide range of measurements can be performed at arbitrary time intervals. The display surface F is irradiated with the electron beam at a constant cycle, and the high-brightness spot moves at high speed as shown by the arrow G. Here, when a high-brightness spot passes on an intersection (for example, H) of a display surface F with a straight line connecting an arbitrary light receiving element D and the center A of the lens, the output of the light receiving element D has a peak.

FIG. 3 is another configuration diagram of the light receiving device according to the first embodiment of the present invention. In the figure, reference numeral 81 denotes a concave lens (mirror), and reference numeral 82 denotes a hole (light receiving window). The difference from the light receiving device shown in FIG. 2 is that a concave lens (mirror) is
This is the point that is used.

FIG. 4 is another configuration diagram of the light receiving device according to the first embodiment of the present invention. In the figure, reference numeral 83 denotes a pinhole, and the difference from the light receiving device shown in FIG. The point is that a pinhole is used as an optical unit.

FIG. 5 is a block diagram showing the configuration of the first embodiment of the present invention. In this figure, reference numeral 4 is
A peak detector that detects a peak value of each light receiving element D based on a preset threshold. By determining the time delay between the time when the peak occurs and the vertical display start time on the display surface F of the display, it is possible to measure where on the screen the intersection was. Reference numeral 5 denotes a counter, which counts the vertical drawing start signal of the display 1 as a reset signal by incrementing by one at a predetermined cycle (or by counting a predetermined clock signal). Count the time. It is assumed that the counting speed is such that the irradiation position of the electron beam on the display 1 can be detected with sufficient accuracy. The output is sent to AND gate 6. Reference numeral 6 denotes an AND gate, which receives a signal from the peak detector 4 and inactivates (when a peak is not detected) or activates (when a peak is detected) the count output for each light receiving element D. Reference numeral 7 denotes a correspondence position determining means for associating the bright spot display position coordinates on the display device 1 and the position coordinates of the projected image on the light receiving device 2 at a plurality of times according to the recording time. Reference numeral 8 denotes a three-dimensional position calculation unit, which is associated with a plurality of (three or more points)
The three-dimensional position and orientation of the light receiving device 2 with reference to the bright spot scanning surface of the display device 1 is calculated from a set of the bright spot display position coordinates and the position coordinates of the projected image. Symbol 5 above
8 constitute the three-dimensional position output device 3 of the present embodiment. In addition, the code | symbol N has shown the light receiving part and M has shown the light condensing part.

FIG. 6 is a flowchart showing the operation of the first embodiment of the present invention. The operation will be described in detail with reference to FIG.

(First Step) It is assumed that the peak value of any one of the light receiving elements D, which is the peak detector 4, has been detected (S1).
By determining the time delay between the time when the peak value occurs and the vertical display start time on the display surface F, it is possible to measure where on the screen the intersection was. That is, when the peak value is detected here, it is sent to the AND gate 6. On the other hand, when detecting the synchronization signal (vertical drawing start signal) of the display 3 (S2), the counter 5 starts counting using this as a reset signal (S3). If no synchronization signal is detected, the output is sent to the AND gate 6 as 'counter value + 1' (S4). The AND gate 6 transmits the counter value to the corresponding position determining unit 7 for each light receiving element D that has detected the peak value (S5).

(Second Stage) The correspondence position determination unit 7 determines the counter value by using the spot display position (Xp,
Yp) (S6), and the position (Xr, Yr) of the light receiving element D that has output the peak value and the spot display position (Xp, Yp) at that time are recorded (S7).

(Third Stage) The three-dimensional position calculation unit 8 calculates the three or more points (Xp, Yp, X
r, Yr) and the focal length f of the light collecting unit of the light receiving device 2, the position and orientation of the light receiving device 2 can be calculated (S).
8). The specific computation method in the case of measurement by three points, the document "" unified solution method PnP camera problems due projective geometry "Koichiro Ideguchi processing Symposium, Vol.
90, pp41-50,1990 ”
In the case of measurement using four points, the literature (RJHolt and ANN
etravali. Camera calibration problem: Some new resu
lts. Computer Vision, Graphics, and Image Processi
ng: Image Understanding, Vol. 54, No. 3, PP. 368-383,
1991.) and references "Akira Kato et al." Camera Calibration by Active Graphic Update "IEICE, IEICE Technical Report, PRU94-112 (1995-01), pp. 39-46, 27
The three-dimensional position and orientation of the light receiving device 2 are calculated and output by a calculation method as disclosed in Japanese Patent Application Laid-Open No. 0124. Also, 3
The method of calculating the three-dimensional position and orientation using multiple points
Literature `` R, M, Haralick, C. Lee, K. Ottenberg, and M. Nolle,
“Analysis Solutions of The Three Point Perspective
Pose Estimation Problem ”pp.592-598,1991 IEEE ”
Is shown in

Hereinafter, a calculation method using three or more points will be described with reference to FIG.

Assuming that the direction of the lens optical axis of the condensing portion of the light receiving device is rotated by α around the X axis, β around the Y axis, and γ around the Z axis, the rotation matrix of the posture of the light receiving device is as follows.

[0055]

(Equation 7)

If the position of the center of the lens 32 with respect to the display surface 31 is (Xt _, Yt _, Zt), it can be defined as a translation matrix as follows.

[0057]

(Equation 8)

Here, assuming that the distance from the center of the lens 32 to the light receiving element surface 33 of the light receiving device is f, the display surface
Assuming that the n-th point on 31 is (Xpn _, Ypn _, 0) and the projection position on the light receiving element is (Xrn _, Yrn), the relationship is as follows.

[0059]

(Equation 9)

This relational expression indicates that R and T can be obtained as solutions of the simultaneous equations when n ≧ 3.
That is, if there are a plurality of points on the display surface of three or more sets and corresponding points on the light receiving element, the position and orientation of the light receiving device can be obtained.

FIG. 7 shows, as a second embodiment of the present invention, a semiconductor position detecting element (PSD: Position Sensitive Light D) as a light receiving element on the projection plane B in the light receiving device 2.
This is another embodiment using the E.C. The principle in this case is as follows. The PSD is a flat silicon plate and is composed of three layers of P, I and N. When a light spot is incident on the PSD, charges proportional to the light energy of the incident light are collected at the incident position, and this electric charge passes through the resistive layer (P layer in this case) as a photocurrent, and the position of the projected spot (X , Y). Therefore, if the resistance distribution of the resistance layer is made sufficiently uniform, the photocurrent is divided in inverse proportion to the distance to the electrode, that is, the resistance value, and is output from the output terminals at both ends. In this case, the incident position of light can be obtained regardless of the incident light energy.

Here, returning to the second embodiment, the semiconductor position detecting element 21 is installed on the projection plane B.
The display surface F is irradiated with the electron beam at a constant cycle, and the high-brightness spot moves at high speed as shown by the arrow G. Here, an intersection (for example, Xp,
When a high-brightness spot passes over Yp), the semiconductor position detection element 21 electrically outputs the spot position (Xr, Yr).

FIG. 8 is a block diagram showing the configuration of the second embodiment of the present invention. Reference numeral 9 denotes a detection position time determination unit, which transmits the spot light receiving position and the counter value of the counter 5 to the corresponding position determination unit 7 as (Xr, Yr, T). Other configurations are the same as those in FIG. 5,
The components 7, 8, and 9 constitute the three-dimensional position output device 3 of the present embodiment.

FIG. 9 is a flowchart showing the operation of the second embodiment of the present invention. The operation will be described in detail with reference to FIG.

(First Stage) First, the semiconductor position detecting element 2
When the spot light is projected onto the spot 1, the detection position coordinates (X, Y) of the spot are output and sent to the detection position time determination unit 9 (S11). On the other hand, when the counter 5 detects the synchronization signal (vertical drawing start signal) of the display 3 (S5).
12), counting is started using this as a reset signal (S13). If the synchronization signal is not detected, the output is sent to the detection position time determination unit 9 as 'counter value + 1' (S14). In the detection position time determination unit 9, the coordinates of the light receiving position of the spot light and the counter value (Xr, Yr, T)
Is transmitted to the corresponding position determining unit 7 (S15).

(Second stage) The correspondence position determining section 7 determines the counter value T by using the spot display position (X
p, Yp) (S16), and record the position (Xr, Yr, T) of the light receiving element D that has output the peak value and the spot display position (Xp, Yp) at that time (S1).
7).

(Third Step) The three-dimensional position calculating unit 8 calculates the position and orientation of the light receiving device 2 from three or more points (Xp, Yp, Xr, Yr) and the focal length f of the light collecting unit of the light receiving device 2. Can be calculated (S18).

FIG. 10 is a configuration diagram of a light receiving device according to the third embodiment of the present invention. A plurality of light collecting sections and light receiving elements are provided so that light can be collected over a wider range. In the example of FIG. 10, an example of an embodiment in which a set of four light-collecting units and light-receiving units is arranged so as to have optical axes in four directions different by 90 degrees is shown. And the angle between the optical axes is arbitrary. The operation principle is the same as in the first and second embodiments described above.

FIG. 11 is a block diagram for explaining a fourth embodiment of the present invention. In the present embodiment, the light receiving device used in the first and second embodiments is combined with a means for converting the amount of movement of a plane into a numerical value and converting it into two-dimensional information. A specific example of the means for converting the amount of movement of the plane into a numerical value and converting it into two-dimensional information is a trackball, a mouse, a laser sensor, or the like. In FIG. 11, the two-dimensional information conversion unit 10 composed of a mouse and the light receiving unit 11 are integrated.

FIG. 12 is a flowchart showing the operation of the fourth embodiment. The operation will be described in detail with reference to FIG.

The steps up to the step of calculating the three-dimensional position / posture by the light receiving device 11 are the same as those in the first and second embodiments (S21 to S28). Thereafter, it is detected whether or not the two-dimensional information conversion unit 10 using a mouse or the like is on the mouse pad 12 (S29). If not, the three-dimensional position / posture is output as it is, and the grounding is performed. If so, the movement information of the two-dimensional information conversion unit 10 on the mouse pad 12 is detected and output (S30). Thus, if a single input device is used, for example, when it is grounded to a mouse pad, screen operations can be performed in the same manner as a conventional mouse, and if it is held in the air away from the mouse pad, a three-dimensional position / posture can be input. And the operability is improved.

In each of the above embodiments, the light receiving device 2, the two-dimensional information converter 10 and the light receiving device 11 are
By transmitting the output signal wirelessly, the degree of freedom of movement can be further improved as compared with the case where the signal is transmitted by wire.

Further, in each of the above embodiments, the display device (display) that scans the luminescent spot at a constant period is used, but the projection position of the luminescent spot of the light receiving device and the luminescent spot display position of the display device are used. If the correspondence is uniquely determined, the scanning cycle may not be constant, and the scanning method is also arbitrary.

In each of the above embodiments, the light receiving device 2, the two-dimensional information conversion unit 10, and the light receiving device 11 are integrated with, for example, a part of a body such as a hand or a part of the body. By providing means that can be gripped by the part,
Usability can be improved.

Further, reference numerals 5 and 5 shown in FIGS.
The function of each unit of the apparatus 6, 7, 8, 9 can be realized by using a computer, or the computer can execute the processing procedures shown in FIGS. 6, 9, and 12. Needless to say, a program for realizing the function of each section by a computer, or a computer-readable recording medium such as an FD (floppy (registered trademark) disk) or M
It can be recorded, provided, and distributed on O, ROM, memory card, CD, DVD, removable disk, and the like.

FIG. 14 is a block diagram for explaining a fifth embodiment of the present invention, wherein 41 is a projection plate, 42 is a light receiving device, 43 is a three-dimensional position output device, and 44 is a light projecting module. .

The light projecting module 44 is placed on the surface of the projection plate 41.
The light-receiving device 42
The three-dimensional position output device 43 processes the observation information and the drawing information, and outputs the position and orientation of the light receiving device 42.

FIG. 15 is a structural view of a light emitting module according to a fifth embodiment of the present invention, wherein 51 is a magnetic circuit, 52 is a lens, 53 is a laser emitter, and 54 is a control signal generating module. The light emitting module 44 is disclosed in JP-A-6-3005.
Arbitrary laser scanning is performed in the space by a control method as described in “Shape feature extraction device and two-dimensional laser pattern generator using two-dimensional laser pattern” described in Japanese Patent Publication No. 42-42.

FIG. 16 is a block diagram showing the configuration of the fifth embodiment of the present invention. In this embodiment, as the light receiving elements of the light receiving device 42, a plurality of light receiving elements that convert light intensity into an electric signal are used. In the figure, A is a lens, B is a projection plane of light passing through the lens A, C is a center point where the optical axis of the lens and the projection plane B of light intersect, D is a light receiving element, and F is a display of a projection plate receiving laser irradiation. Reference numeral 61 denotes a display device, 62 denotes a light condensing unit, 63 denotes a projection surface, 64 denotes a peak detecting unit, 65 denotes a corresponding position determining unit, 66 denotes a three-dimensional position calculating unit, 67 denotes a light emitting module, and 68 denotes a control signal. It is a generating part.

A plurality of light receiving elements D are provided on the projection plane. The display surface F is irradiated with the laser beam at a constant period, and the high-brightness spot moves at high speed as shown by the arrow G. Here, when a high-brightness spot passes on the intersection (for example, H) of the display surface F with a straight line connecting the arbitrary light receiving element D and the center A of the lens, the output of the light receiving element D has a peak.

The peak detector 64 detects the peak value of each light receiving element D. By determining the correspondence between the time at which this peak occurs and the control signal (x, y) transmitted to the light projecting module 67 by the control signal generator 68, the corresponding position determiner 65 determines where the intersection point on the screen is. You can measure the warmth.

The three-dimensional position calculating means 66 calculates the three-dimensional position and orientation of the light receiving device with reference to the bright spot scanning plane of the display device from a set of a plurality of bright spot display position coordinates and their projection position coordinates. is there.

FIG. 17 is a flowchart showing the operation of the fifth embodiment of the present invention. The operation will be described in detail with reference to FIG.

(First Step) It is assumed that the peak value of any one of the light receiving elements D which is the peak detector 64 is detected (S3).
1). The laser scanning control signal (X) transmitted from the control signal generator 68 in FIG.
1, Y1) is received by the correspondence position determination unit 65.
(S32)

(Second Stage) The correspondence position determination unit 65
The laser scanning control signal is converted into a spot display position (Xp, Yp) on the projection plate (S33), and the position (Xr, Yr) of the light receiving element D that has output the peak value and the spot display position (Xp) at that time. , Yp) (S3)
4). The following equation is used for conversion from (X1, Y1) to (Xp, Yp), and conversion coefficients a0 to a5 are determined in advance.

[0086]

(Equation 4)

(Third Step) The three-dimensional position calculating unit 66
The position and orientation of the light receiving device 42 can be calculated from (Xp, Yp, Xr, Yr) above the point and the focal length of the light collector 62 (S35). This specific calculation method is shown in FIG.
The three-dimensional position and orientation of the light receiving device are calculated and output by the calculation method indicated by.

FIGS. 18 and 19 are structural views for explaining a sixth embodiment of the present invention. FIG. 18 is an overall perspective view.
FIG. 19 is a side view. Reference numeral 71 denotes a projection plate, 72 denotes a light receiving device, 73 denotes a three-dimensional position output device, 74 denotes a light projecting module, and 75 denotes a personal computer.

The projection plate 71 is translucent, and the bright points of the laser light emitted from the light projecting module 74 are diffusely reflected on the back surface of the projection plate 71, and only the bright points are displayed on the surface of the projection plate 71. is there. Further, the present apparatus is premised on three-dimensional input to a PC or the like without a raster scan display, and transmits the three-dimensional position and orientation of the light receiving device 72 with respect to the projection plate 71 to the connected personal computer 75.

The configuration diagram, the block diagram showing the configuration, and the flowchart showing the operation of the light projecting module in this embodiment are the same as those in the fifth embodiment shown in FIG.

[0091]

As described above, according to the present invention, 3
As an input device for three-dimensional position / posture, only a semiconductor device capable of detecting projected position coordinates and a display device that moves or scans a luminescent spot on a two-dimensional surface are used. 3D measurement can be performed stably without being affected by temperature and temperature. Also, by measuring the three-dimensional position / posture at three points using three light receiving elements, the three-dimensional position / posture can be measured at the minimum time interval. further,
The luminous point display position coordinates and the projection position coordinates at arbitrary plural times are correlated in accordance with the recording time, and the luminous point moving surface or the luminosity of the display device is obtained from the set of the plurality of luminous point display position coordinates and the projection position coordinates. By calculating the three-dimensional position and orientation of the light receiving device based on the point scanning plane,
Further, since a three-dimensional position / posture calculation can be performed using a combination of an arbitrary number and an arbitrary position of the projected image position coordinates and the bright spot display position coordinates, the three-dimensional position / posture can be calculated a plurality of times in one scanning interval of the display device. Since measurement can be performed, there is an effect that intuitive three-dimensional input can be performed over a wide range.

[Brief description of the drawings]

FIG. 1 is a schematic view of an apparatus for explaining an embodiment of the present invention.

FIG. 2 is a configuration diagram of a light receiving device according to the first embodiment of the present invention.

FIG. 3 is another configuration diagram of the light receiving device according to the first embodiment of the present invention.

FIG. 4 is another configuration diagram of the light receiving device according to the first embodiment of the present invention.

FIG. 5 is a block diagram showing a configuration of the first exemplary embodiment of the present invention.

FIG. 6 is a flowchart illustrating an operation of the first exemplary embodiment of the present invention.

FIG. 7 is a configuration diagram of a light receiving device according to a second embodiment of the present invention.

FIG. 8 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 9 is a flowchart showing the operation of the second embodiment of the present invention.

FIG. 10 is a configuration diagram of a light receiving device according to a third embodiment of the present invention.

FIG. 11 is a schematic view of an apparatus for explaining a fourth embodiment of the present invention.

FIG. 12 is a flowchart showing the operation of the fourth embodiment of the present invention.

FIG. 13 is a diagram for explaining a method of calculating a three-dimensional position / posture using three or more points.

FIG. 14 is a configuration diagram for explaining a fifth embodiment of the present invention.

FIG. 15 is a configuration diagram of a light emitting module according to a fifth embodiment of the present invention.

FIG. 16 is a block diagram showing a configuration of a fifth exemplary embodiment of the present invention.

FIG. 17 is a flowchart showing the operation of the fifth embodiment of the present invention.

FIG. 18 is an overall configuration diagram for explaining a sixth embodiment of the present invention.

FIG. 19 is a side view of FIG. 18;

[Explanation of symbols]

1 Raster scan display 2 Light receiving device 3 3D position output device 4 Peak detector 5 counter 6 AND gate 7 Correspondence position determination unit 8 3D position calculation unit 9 Detection position time determination unit 10 Two-dimensional information converter 11 Light receiving unit 12 mouse pads 21 Semiconductor position detecting element 31 Display surface 32 lenses 33 Light-receiving element surface 41 Projection plate 42 Receiver 43 3D position output device 44 Floodlight Module 51 Magnetic circuit 52 lenses 53 laser emitter 54 Control signal generation module 61 Display device 62 Condenser 63 Projection surface 64 peak detector 65 Correspondence Position Determination Unit 66 3D position calculation unit 67 Light emitting module 68 Control signal generator 71 Projection plate 72 Light receiving device 73 3D position output device 74 Light emitting module 75 Personal Computer 81 Concave lens (mirror) 82 holes (light receiving window) 83 Pinhole A lens B Projection surface of light passing through lens A C Center point where lens optical axis and light projection plane B intersect D Light receiving element F The display surface of a raster scan display N light receiving section M Condenser

Continuation of the front page (56) References JP-A-9-325007 (JP, A) JP-A-8-278846 (JP, A) JP-A-7-270124 (JP, A) JP-A-8-117448 (JP JP-A-8-76929 (JP, A) JP-A-10-333834 (JP, A) JP-A-11-249806 (JP, A) JP-A-11-219253 (JP, A) Koichiro Exit Unified Solution of PnP Camera Correction Problem Using Projective Geometry, Computer Vision '90-Vision and Environmental Understanding-Symposium Papers, Japan, Information Processing Society of Japan, August 29, 1990, p. 41-50 (58) Field surveyed (Int.Cl. ⁷ , DB name) G06F 3/033-3/037 G06F 3/03 G06F 3/00 G01B 11/00 A63F 13/00-13/12

Claims (32)

(57) [Claims] 1. A three-dimensional position and orientation measuring apparatus for measuring a three-dimensional position and orientation of the light receiving device with respect to a display device which emits bright spot, the bright spot at the display surface of the display device is movable A display device, a light-collecting unit that collects bright spots on the display device and projects the spot as light onto a projection surface, and a plurality of three or more light-receiving elements capable of detecting a projection position of the projected spot light are arranged. a light receiving device having the projection plane, the display bright spot display position coordinates of the device and the projection position coordinates of said based on the light receiving element spotlight and correlating means for correlating, said display associated with said associating means Equipment shine
Spot display position coordinates and spot light projection based on the light receiving element
Of the set of shadow position coordinates to calculate the three-dimensional position and orientation of the light receiving device relative to the bright point moving screen of the display device from a set of at least three pairs of bright spot display position coordinates and the projection coordinates 3 possess a dimension position calculating means, the three-dimensional position calculating means, bright spot of the display device moving screen
Represents the three-dimensional position and orientation of the light receiving device with reference to
The light receiving device using a parallel translation matrix T and a rotation matrix R
F is the distance from the center of the lens of the focusing part to the light receiving element surface.
And the n-th point on the display surface as (Xpn _, Y
pn _, 0) and its projected position on the light receiving element are (Xrn
_, Yrn), 3D position and attitude based on the relationship
A three-dimensional position / posture measuring device characterized by calculating . 2. The three-dimensional position / position according to claim 1, wherein the light receiving element capable of detecting the projection position of the spot light is a set of light receiving elements having a spot-shaped light receiving window.
Attitude measurement device. 3. The light-receiving element having the spot-shaped light-receiving window is three light-receiving elements, and the bright spot display position coordinates and the projection position coordinates associated by the association means are three. The three-dimensional position / posture measuring device according to claim 2, characterized in that: 4. The light-receiving element having the spot-shaped light-receiving window is five or more light-receiving elements, and the bright spot display position coordinate and the projection position coordinate associated by the associating means are five or more. 3. The three-dimensional position / posture measuring apparatus according to claim 2, wherein the number is arbitrary. 5. The associating means, wherein: means for associating a position coordinate of a projected image on each light receiving element projected on the projection surface on the light receiving device with a bright point of the display device and a projection time; Means for recording the projection position coordinates and the projection time, and correspondence position determination means for associating the luminescent spot display position coordinates at a plurality of times and the projection position coordinates with each other according to the recorded projection time. The three-dimensional position / posture measuring device according to claim 2. 6. The display device scans the bright spot,
The three-dimensional position / posture measuring apparatus according to claim 2, wherein the display is performed. 7. The display device scans the bright spot,
The three-dimensional position / posture measuring apparatus according to claim 6, wherein a CRT display is used for displaying. 8. The display device scans the luminescent spot,
The three-dimensional position / posture measuring apparatus according to claim 6, wherein a bright spot irradiation device and a projection plate are used for displaying. 9. The three-dimensional position / posture measuring apparatus according to claim 2, wherein the light receiving device includes a plurality of condensing portions and projection surfaces having different optical axes. 10. The light-receiving device according to claim 2, wherein the light receiving device has a unit that can be mounted on a moving body such as a robot or a vehicle, a part of a body, or can be gripped by a part of a body. Dimensional position / posture measuring device. 11. The three-dimensional position / posture measuring apparatus according to claim 2, wherein the light receiving device is combined with a unit that converts a plane moving amount into a numerical value and converts the numerical value into two-dimensional information. 12. The three-dimensional position / posture measuring apparatus according to claim 11, wherein any one of a trackball, a mouse, and a laser sensor is used as said converting means. 13. The three-dimensional position / posture measuring apparatus according to claim 2, wherein the light receiving device is capable of freely moving the output signal transmission by wire or wirelessly. 14. The light receiving element according to claim 1, wherein the light receiving element capable of detecting the projection position of the spot light has a planar light receiving section and is capable of outputting the position coordinates of the spot light. 3D position / posture measuring device. 15. The associating means, means for associating a projection position coordinate on each light receiving element of a spot light, which is obtained by projecting a bright spot of the display device on the projection surface of the light receiving device, with a projection time; Means for recording projection position coordinates and the projection time, and correspondence position determination means for associating the luminescent spot display position coordinates at a plurality of times and the projection position coordinates with each other in accordance with the recorded projection time. The three-dimensional position / posture measuring device according to claim 14. 16. The three-dimensional position / posture measuring apparatus according to claim 14, wherein a semiconductor position detecting element is used as the light receiving device. 17. The display device according to claim 14, wherein the display device scans the bright spot and performs display.
Dimensional position / posture measuring device. 18. The three-dimensional position / posture measuring apparatus according to claim 17, wherein a CRT display is used as the display device for scanning and displaying the bright spot. 19. The three-dimensional position / posture measurement apparatus according to claim 17, wherein a bright spot irradiating device and a projection plate are used as the display device to scan and display the bright spot. 20. The three-dimensional position / posture measuring apparatus according to claim 14, wherein the light receiving device includes a plurality of light condensing portions having different optical axes and a projection surface. 21. The light-receiving device according to claim 14, wherein the light receiving device has a unit that can be attached to a moving body such as a robot or a vehicle, a part of a body, or can be gripped by a part of a body. Dimensional position / posture measuring device. 22. The three-dimensional position / posture measuring apparatus according to claim 14, wherein said light receiving device is combined with a converting means for converting a plane movement amount into a numerical value and converting the numerical value into two-dimensional information. 23. A three-dimensional position / posture measuring apparatus according to claim 22, wherein any one of a trackball, a mouse, and a laser sensor is used as said converting means. 24. The three-dimensional position / posture measuring apparatus according to claim 14, wherein the light receiving device is capable of freely moving the output signal transmission by wire or wirelessly. 25. A three-dimensional position and orientation measuring method for measuring a three-dimensional position and orientation of the light receiving device with respect to a display device which emits bright spot, is arbitrarily moving the bright spot at the display surface of the display device And collecting light spots on the display device by a light collecting unit of the light receiving device and projecting the bright spots on the projection surface of the light receiving device as spot light.
And projecting the projected spot light disposed on the projection surface.
With three or more light receiving elements that can detect the shadow position,
Obtaining the projection position coordinates of the spotlight projected on the projection surface; associating the bright spot display position coordinates of the display device with the projection position coordinates of the spotlight based on the light receiving element; The bright spot display position coordinates of the display device;
A set of spot light projection position coordinates based on the light receiving element
Among them, a step of calculating the three-dimensional position and orientation of the light receiving device relative to the bright point moving screen of the display device from a set of at least three pairs of bright spot display position coordinates and the projection coordinates, the 3 Calculating the three-dimensional position / orientation,
3D position of the light receiving device with reference to the bright spot moving screen
Using a translation matrix T and a rotation matrix R representing the position and orientation
The light receiving element from the center of the lens of the light collecting section of the light receiving device.
Let n be the distance to the surface,
A point is projected on (Xpn _, Ypn _, 0) and its light-receiving element.
Assuming _{that the} shadow position is (Xrn _, Yrn), 3D position and attitude based on the relationship
A three-dimensional position / posture measurement method characterized by calculating . 26. The three-dimensional position / posture measurement method according to claim 25, wherein the light receiving element capable of detecting the projection position of the spot light is a set of light receiving elements having a spot-shaped light receiving window. 27. The associating step, comprising: associating the position coordinates of a projected image on each light receiving element projected on the projection surface on the light receiving device with a luminescent point of the display device and a projection time; Recording the projection position coordinates and the projection time; and determining a correspondence position according to the recorded projection times at a plurality of times. The three-dimensional position / posture measurement method according to claim 26. 28. The light receiving element according to claim 25, wherein the light receiving element capable of detecting the projection position of the spot light has a planar light receiving portion and is capable of outputting the position coordinates of the spot light. 3D position / posture measurement method. 29. The associating step, comprising: associating a projection time with a projection position coordinate on each light receiving element of a spot light obtained by projecting a bright point of the display device on the projection surface of the light receiving device; Recording a projection position coordinate and the projection time; and a correspondence position determining step of associating the luminous point display position coordinate at a plurality of times and the projection position coordinate with each other according to the recorded projection time. A three-dimensional position / posture measuring method according to claim 28. 30. A recording medium storing a program for causing a computer to execute a three-dimensional position / posture measuring method for measuring a three-dimensional position and posture of a light receiving device with respect to a display device that emits a luminescent spot, the display medium comprising: Arbitrarily moving the luminescent spot on the display surface of the device, and condensing the luminescent spot on the display device by the condensing part of the light receiving device, and projecting the luminescent spot on the projection surface of the light receiving device as spot light
And projecting the projected spot light disposed on the projection surface.
With three or more light receiving elements that can detect the shadow position,
Obtaining the projection position coordinates of the spotlight projected on the projection surface; associating the bright spot display position coordinates of the display device with the projection position coordinates of the spotlight based on the light receiving element; The bright spot display position coordinates of the display device;
A set of spot light projection position coordinates based on the light receiving element
Among them, a step of calculating the three-dimensional position and orientation of the light receiving device relative to the bright point moving screen of the display device from a set of at least three pairs of bright spot display position coordinates and the projection coordinates, the 3 Calculating the three-dimensional position / orientation,
3D position of the light receiving device with reference to the bright spot moving screen
Using a translation matrix T and a rotation matrix R representing the position and orientation
The light receiving element from the center of the lens of the light collecting section of the light receiving device.
Let n be the distance to the surface,
A point is projected on (Xpn _, Ypn _, 0) and its light-receiving element.
Assuming _{that the} shadow position is (Xrn _, Yrn), 3D position and attitude based on the relationship
A computer-readable recording medium for calculating and causing each of the steps to function. 31. The light-receiving element, wherein the light-receiving element capable of detecting the projection position of the spot light is a set of light-receiving elements having a spot-shaped light-receiving window, wherein the associating step comprises: Relating the position coordinates and the projection time of the projected image on each light receiving element projected on the projection surface on the light receiving device with the luminescent spot of the device; andrecording the projection position coordinates and the projection time, 31. The recording medium according to claim 30, further comprising: a correspondence position determination step of associating the coordinates of the luminescent spot display position and the projection position coordinates at a plurality of times according to the recorded projection time. 32. A projection position of the spot light can be detected.
Light receiving element has a planar light receiving section,
A light receiving element capable of outputting setting coordinates , wherein the associating step comprises: projecting position coordinates and projection time on each light receiving element of spot light obtained by projecting a luminescent spot of the display device on the projection surface of the light receiving device. A step of recording the projection position coordinates and the projection time; and a correspondence position determination step of associating the luminescent spot display position coordinates at a plurality of times and the projection position coordinates with the correspondence according to the recorded projection time. 31. The recording medium according to claim 30, comprising: