JP4046282B2 - Range finder - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a range finder that measures the shape of an object in a non-contact manner.
[0002]
[Prior art]
In recent years, with the development of CCD camera and computer image processing, three-dimensional measurement using images has become common. There is a range finder as a three-dimensional measuring apparatus using a CCD camera and computer image processing. The range finder is a device that irradiates a target with parallel light beams such as slit light from a projector, images the target with a camera, and measures the shape of the target from the captured image. The principle is that the surface position of the object is calculated based on triangulation from the positional relationship between the projector and the camera and the position of the parallel rays in the captured image (see, for example, Patent Document 1).
[0003]
[Patent Document 1]
Japanese Patent Laid-Open No. 9-287927 (paragraph 0025, FIG. 2, etc.)
[0004]
[Problems to be solved by the invention]
By the way, in the conventional range finder, the projector measures the shape on the premise that the light beam irradiates a parallel light beam from a fixed point and the light beam incident on the camera converges to the single point. That is, it is assumed that the camera lens system is based on a pinhole camera model. As shown in FIG. 7, this pinhole camera model is such that only light (incident light) incident through the base point position (needle hole: pinhole H) reaches the image plane, and the three-dimensional space is an image. A model associated with a two-dimensional space on a surface. Thus, the pinhole camera model assumes that an incident light beam passes through a single pinhole to form an image.
[0005]
However, not a pinhole camera but a camera having a lens system made of glass or the like does not converge to one point even if the incident light beam is extended. Therefore, non-linear distortion exists in an image captured by a camera having a lens system, and the distortion is larger in the peripheral visual field.
Therefore, in the conventional range finder, an error is generated in the position measurement result of the object by this distortion. This error also depends on the distance to the object, and the change in the error due to the distance of the object is non-linear. Therefore, if the distance of the object is known, it is possible to calibrate based on the relationship between the position measurement result and the actual position, but since the distance of the object is unknown until measurement, It could not be calibrated.
[0006]
The present invention has been made in view of the above problems, and in the conventional range finder, eliminates the root cause of errors based on the pinhole camera model, regardless of the distance between the camera and the object. An object of the present invention is to provide a range finder capable of performing accurate shape measurement.
[0007]
[Means for Solving the Problems]
The present invention was devised to achieve the above object, and first, the range finder according to claim 1 irradiates an object with parallel rays by a projector and the parallel rays with a camera . A range finder that images a target object and measures the shape of the target object from the positional relationship between the light emission position of the projector and the optical center of the camera, and the irradiation position of the parallel rays in the captured image, An image input unit that inputs an image captured by the camera, a pixel position detection unit that detects a pixel position of an irradiated parallel ray in the image input by the image input unit, and an image captured by the camera Calibration table storage means for storing a calibration table associating a pixel position with the direction of the incident light beam to the pixel and the amount of displacement from the reference position to the incident light beam; Based on the pixel position detected by the position detection means, referring to the calibration table, the direction and displacement amount of the incident light beam corresponding to the pixel position are obtained, and the direction and displacement amount of the incident light beam are used to And a position calculating means for calculating the irradiation position .
[0008]
In such a range finder , first, a parallel light beam is irradiated onto an object by a projector. This parallel light may be light that is irradiated from the projector without spreading, and spot light that is irradiated at one point, slit light that is band-shaped light, pattern light having a specific pattern, and the like alone. Or they can be used in combination.
Then, the image input means inputs the image captured by the camera into the range finder, and the pixel position detection means recognizes the pixel position of the parallel light beam emitted from this image. Further, the position calculating means refers to the calibration table to acquire the direction and displacement amount of the incident light beam corresponding to the pixel position, and uses the direction of the incident light beam and the displacement amount from the reference position to obtain the irradiation position, that is, the target. Calculate the surface position of the object.
Therefore, since the shape of the object can be measured based on the exact direction of the incident light incident on the lens, the shape of the object can be accurately measured. Incidentally, it with the optical center (opticalcenter) indicates the center position of the lens system, the lens system of the pinhole camera model, it corresponds to the position of the pinhole.
[0009]
The range finder according to claim 2 is the range finder according to claim 1, wherein the reference position represents a fixed point having a fixed relationship with the camera, and the amount of displacement is incident light from the reference position. The calibration table is represented by a vector directed to an arbitrary point on the line, the arbitrary point becomes a base point extending a vector of the line-of-sight direction from the camera, and the line-of-sight direction indicates the direction of incident light on the pixel. However, the displacement amount is associated with the reference position for each pixel position .
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
First, we will explain the non-pinhole characteristics of a camera that does not intersect incident light at a single point, which causes image distortion in a camera that is generally called a pinhole camera model, and quantifies the characteristics of the non-pinhole camera. The calibration data thus made will be described. Then, a method for generating calibration tables by measuring calibration data for each imaging pixel of the camera will be described, and a range finder having this calibration table will be sequentially described.
[0012]
[Non-pinhole characteristics of the camera]
First, with reference to FIG. 8, the cause of distortion in an image captured by a camera having a lens system will be described. FIG. 8 is a schematic diagram of a camera having a lens system. Here, in order to simplify the description, the lens system and the glass sheet G, it is assumed that the pinhole H is generated. Incident light beam r1 incident perpendicularly on the glass plate G of the camera C passes through the pinhole H and is imaged on the pixel R1 on the imaging surface I. In addition, incident rays r2 and r3 incident obliquely on the plate glass G are refracted in the plate glass G, pass through the pinhole H, and are imaged on the pixels R2 and R3 on the imaging surface I.
[0013]
However, in this camera C, the incident rays r2 ′ and r3 ′, which are the extension lines of the incident rays r2 and r3 before passing through the glass sheet G, and the incident ray r1 do not intersect at one point, but actually are a pinhole camera model. You can see that it is not. For this reason, the incident light beam r3 that is shifted by the distance D from the incident light beam rr assumed in the pinhole camera model is imaged on the pixel R3 on the imaging surface I.
[0014]
Thus, a camera that captures an image with incident light rays incident on the lens system (here, the glass sheet G) has a pinhole property (non-pinhole property). Hereinafter, a camera having a lens system is referred to as a “non-pinhole camera”.
[0015]
[About calibration data]
Next, calibration data obtained by quantifying the characteristics of the non-pinhole camera will be described with reference to FIG. FIG. 9 is an explanatory diagram for explaining the contents of the calibration data. As shown in FIG. 9, the incident light ray R incident on the lens 1 can be specified by two points. Here, when light emitted from the first light source position P1 and the second light source position P2 is captured by the same pixel (not shown), the incident light R corresponds to the pixel. To be identified.
[0016]
Here, the point at which the sum of squares of the distances to all incident rays is minimized is defined as the optical center O, and the point at which the distance between the incident ray R and the optical center O corresponding to each imaging pixel is minimized. This is defined as an incident light base point K of the incident light R.
[0017]
That is, the optical center O (x ₀ , y ₀ , z ₀ ) has a light source position P1 (x ₁ , y ₁ , z ₁ ) and a light source position P2 (x ₂ , y ₂ , z ₂ ) for all incident rays. The position where the sum of the squares of the distance d from the incident light ray R (equation (1)) is minimized is determined by the least square method.
[0018]
d ² =-(A ² / B) + C (1)
[0019]
However,
A = (x ₂ -x ₁ ) (x ₁ -x ₀ ) + (y ₂ -y ₁ ) (y ₁ -y ₀ ) + (z ₂ -z ₁ ) (z ₁ -z ₀ )
B = (x ₂ -x ₁ ) ² + (y ₂ -y ₁ ) ² + (z ₂ -z ₁ ) ²
C = (x ₁ -x ₀ ) ² + (y ₁ -y ₀ ) ² + (z ₁ -z ₀ ) ²
And
[0020]
As a result, the direction of the incident light (the direction specified by the light source positions P1 and P2), the amount of displacement from the optical center O to the incident light base point K (represented by a three-dimensional vector V _D (dx, dy, dz)), and By using the data associated with the as the calibration data, the characteristics of the non-pinhole camera can be digitized.
[0021]
Note that the calibration data is not limited to this. For example, in the above-described example, the optical center O is set as the reference position, and the vector from the optical center O to the foot of the perpendicular dropped to the incident ray is set as the displacement amount V _D. However, the reference position is not limited to the optical center, Any point can be used as long as it is a fixed point that is in a fixed relationship. The displacement amount V _D may be a vector that goes from the reference position to an arbitrary point on the incident light beam, and is not limited to a vector that goes from the reference position to the perpendicular foot dropped to the incident light beam.
[0022]
[Calibration table generation method]
Next, a method for generating a calibration table in which calibration data obtained by quantifying the characteristics of a non-pinhole camera is associated with each shooting pixel (corresponding to a projection pixel) of the camera will be described with reference to FIG. FIG. 10 is a conceptual diagram showing the principle of a method for generating a calibration table. FIG. 10A is a conceptual diagram showing the principle of measuring calibration data by changing the pan and tilt of the camera with respect to a specific incident ray, and FIG. 10B is a fixed diagram. It is a conceptual diagram which shows the principle which measures calibration data by changing incident light rays with respect to the camera which did.
[0023]
As shown in FIG. 10A, in order to generate a calibration table in which calibration data is associated with each imaging pixel, the light source position is set in one direction of P1 and P2 with respect to the camera C having non-pinhole characteristics. The camera is moved (one-axis movement) to determine the incident light ray R specified by the light source positions P1 and P2, and the light emitted from the light source positions P1 and P2 is incident on the imaging pixel to be measured. By adjusting the pan and tilt of C (biaxial rotation), the direction of the incident light beam R incident on each imaging pixel of the camera C is specified.
[0024]
Further, as shown in FIG. 10B, the camera C is fixed, and the light source positions P1 and P2 are set so that incident light rays R emitted at two points of the light source positions P1 and P2 are incident on the measurement pixel. Is moved in the XYZ directions (triaxial movement), and the direction of the incident light ray R determined by two points of the light source positions P1 and P2 incident on the measurement pixel may be specified.
[0025]
In FIG. 10 (a) or (b), based on the incident light ray R specified for each imaging pixel that has been measured, the direction of the incident light ray R and the incident light base point K from the optical center O (see FIG. 9). The calibration table can be generated by associating the amount of displacement to the image data (projection pixel) with each other as calibration data.
[0026]
[Range finder configuration]
Next, the range finder will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of a range finder according to an embodiment of the present invention. The range finder 1 shown in FIG. 1 picks up the light (parallel rays) irradiated to the object OB by the laser irradiation device 50 with the camera C, and detects the three-dimensional position of the object OB from the obtained image. To do. Then, the shape of the object OB is measured by measuring the three-dimensional position at a large number of points by swinging the direction of the light beam of the laser irradiation device 50 up and down and left and right.
Here, the range finder 1 includes a laser irradiation device (projector) 50, a camera C, an image input unit 10, a pixel position detection unit 20, a calibration table storage unit 30, and a position calculation unit 40. did.
[0027]
As shown in the block diagram of FIG. 2, the laser irradiation device 50 includes a laser light emitting unit 51 that emits laser light, and a condensing lens 52 that condenses the laser light emitted from the laser light emitting unit 51 into a thin beam. And a diffraction grating 53 for dividing the laser beam condensed by the condenser lens 52 into a plurality of beams. The diffraction grating 53 divides the beam in a direction perpendicular to the paper surface.
In addition, a beam diffusing lens 54 for diffusing the divided beams in one direction to generate slit light is provided at the subsequent stage of the diffraction grating. The beam diffusion lens 54 is configured by a cylindrical lens or the like. By the beam diffusion lens 54, each of the plurality of beams is diffused at an angle of 60 °.
FIG. 3 illustrates a laser light irradiation state. As shown in FIG. 3, the laser beam LB emitted from the laser irradiation device 50 is diffused in a conical surface, forms a quadratic locus RL on the plane PL, and is reflected and scattered. This reflection / scattering indicates the irradiation position in the claims. The laser beam LB is divided into five beams (laser beam LB) by the diffraction grating 53, and the five beams are diffused by 60 ° by the beam diffusion lens 54. These laser beam LB is emitted as radiating from different origin O _L in each beam which is divided into five.
In FIG. 3, the number of beams is “5” for easy viewing, but in practice, the laser light is divided at finer intervals. For example, the angle B shown in FIG. 3 is 32 °, and the angle C between adjacent laser beams is 1.6 °. That is, the number of beams is “21”.
[0028]
Further, the laser irradiation device 50 is not limited to the one that diffracts and diffuses in the lens system, but mechanically rotates the laser light that emits a single spot light in the upward direction (x direction) and the lateral direction (y direction). May be. In that case, if the rotation mechanism is configured such that the laser light is emitted radially from one point, the position of the object can be easily calculated.
[0029]
The image input means 10 inputs an image obtained by capturing the object OB with the camera C. The input image is input as a color image or a multi-tone monochrome image. The image input unit 10 includes a memory (not shown) that temporarily stores each image captured by the camera C. The image position stored in the memory is a pixel position detection unit 20 to be described later. Shall refer to.
[0030]
The pixel position detection unit 20 detects the irradiation position of the laser beam in the image input by the image input unit 10 and specifies the pixel position. Specifically, by looking at the brightness distribution on the image, the pixel having the peak brightness is detected. The pixel position of each image detected here is input to the position calculation means 40.
[0031]
The calibration table storage means 30 is a general storage medium such as a hard disk, and stores a calibration table 31 in which calibration data is associated with each imaging pixel of the camera C. For example, as shown in FIG. 4, the calibration table 31 has a displacement amount V _D (dx, dx, _d) from the optical center O as information for specifying a light ray incident on the pixel with respect to a combination of the x coordinate and the y coordinate of the pixel. dy, dz) and angles α and β are stored in association with each other. As shown in FIG. 5, the angle alpha, beta is the origin O _L in which the laser beam LB of the laser irradiation apparatus 50 is irradiated as the origin of the coordinate axes, X-axis relative to the orientation of the laser irradiation device 50, Y-axis, Z An axis is set, and the angle of incident light with respect to the XZ plane in FIG. 5 is α, and the angle of incident light with respect to the XY plane in FIG. 5 is β.
[0032]
The position calculation means 40 is based on the calibration data (the direction of the incident light and the displacement amount V _D ) of the calibration table 30 corresponding to the pixel position of each image detected by the pixel position detection means 20. The position (three-dimensional position) is calculated.
[0033]
[Rangefinder operation]
Next, the operation of the range finder 1 will be described with reference to FIGS. FIG. 6 is a flowchart showing the operation of the range finder 1.
First, the range finder 1 irradiates the object OB with the laser beam LB (step S1), and images the object OB with the camera C (step S2). Then, the captured image is input by the image input means 10 (step S3).
Next, in the range finder 1, the pixel corresponding to the irradiation position on the object OB of the laser beam LB is detected by the pixel position detecting unit 20 in each image input by the image input unit 10. The position is detected (step S4).
[0034]
Then, the range finder 1 acquires calibration data corresponding to the pixel position by the position calculation means 40 with reference to the calibration table 31 (step S5), and based on the calibration data, the camera base position P _C (Cx, Cy, Cz) is calculated (step S6). As shown in FIG. 5, the camera base point position is a position that becomes a base point for extending a vector in the viewing direction from the camera C. In the coordinate system of FIG. 5, when the coordinates of the optical center O of the camera C are O (Ox, Oy, Oz), the camera base position P _C (Cx, Cy, Cz) is
Cx = Ox + dx
Cy = Oy + dy
Cz = Oz + dz
Is represented by In the present embodiment, the pixel position and dx, dy, dz are associated with the calibration table 31 and Cx, Cy, Cz are calculated separately, but the pixel position and Cx are calculated in the calibration table 31. , Cy, Cz may be stored in association with each other. In that case, the reference position is referred to in the claims, the coordinate system of the origin of the 5, i.e., the origin O _L in which the laser beam LB is irradiated.
[0035]
Then, the irradiation position is calculated based on the camera base position P _C (Cx, Cy, Cz), the angles α and β acquired from the calibration table 31, and the angle ω and the angle φ that specify the direction of the laser beam. Note that when the distance to the object to be measured is limited in advance, the angle ω corresponds to the laser beam that can be determined from the pixel position because there is only a predetermined laser beam within a certain range on the image. The angle can also be determined. In addition, when the laser irradiation apparatus is configured to mechanically shake the spot light, the direction in which the spot light is directed is measured mechanically or electrically, and the angles ω and φ are synchronized with the image. What is necessary is just to take in the position calculation means 40 and to use for calculation.
[0036]
The position is calculated as follows, for example.
As shown in FIG. 5, the distance from the origin OL of the laser beam BL to the irradiation position P ₀ (X, Y, Z) is r, and the camera base point PC of the camera _C to the irradiation position P ₀ (X, Y, Z). The distance to is R.
Therefore, the irradiation position P ₀ (X, Y, Z) is geometrically expressed by the following equations (2) to (4) in relation to the direction vector of the laser beam BL and the distance r.
X = rcosω cos φ・ ・ ・ （２）
Y = rsinω (3)
Z = ‐rcosωsin φ・ ・ ・ （４）
In relation to the line-of-sight direction vector and the distance R, the following equations (5) to (7)
X = Rcosαcosβ + Cx (5)
Y = Rsinαcosβ + Cy (6)
Z = -Rsinβ + Cz (7)
There is a relationship.
From these equations (2) to (7), r and R are obtained.
(0 ≦ Cy and 0 ≦ ω) or (Cy <0 and ω <0)
R = (-B + √ (B ² -4AC)) / 2A
(0 ≦ Cy and ω <0) or (Cy <0 and 0 ≦ ω)
R = (-B-√ (B ² -4AC)) / 2A
When ω ≠ 0
r = (Rsinαcosβ + Cy) / sinω
However,
A = cos ² αcos ² β + sin ² β‐cos ² ωsin ² αcos ² β / sin ² ω
B = cosαcosβ Cx-sinβ Cz ‐ cos ² ωsinαcosβ Cy / sin ² ω
C = Cx ² + Cz ² ‐ cos ² ω Cy ² / sin ² ω
When ω = 0
R = -Cy / sin α cos β
It becomes.
[0037]
In this way, the range finder 1 can correct the non-pinhole property of the camera C and accurately detect the irradiation position. And if the three-dimensional position is calculated about the irradiation position of all the laser beams LB, the three-dimensional shape of the target object OB can be measured.
The range finder 1 can be used by being incorporated in a mobile robot, an automobile, or the like. For example, when the present invention is applied to a mobile robot and the mobile robot detects the shape of the floor, the unevenness of the floor can be recognized accurately, and the mobile robot can stably walk.
[0038]
【The invention's effect】
As described above, according to the range finder according to the present invention, in consideration of the non-pinhole property of the camera, the deviation of the optical center and the light actually incident on the lens system of the camera is corrected, and the object is accurately detected. The position and shape can be measured.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a range finder according to an embodiment.
FIG. 2 is a block diagram of a laser irradiation apparatus.
FIG. 3 is a perspective view showing a laser irradiation state.
FIG. 4 is a diagram illustrating an example of a calibration table.
FIG. 5 is a diagram illustrating a method for calculating an irradiation position.
FIG. 6 is a flowchart showing an operation of the range finder according to the embodiment.
FIG. 7 is a diagram illustrating the concept of a pinhole camera model.
FIG. 8 is a schematic diagram of a camera having a lens system.
FIG. 9 is a diagram illustrating the contents of calibration data.
FIG. 10 is a diagram illustrating a calibration data generation method.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Range finder 10 Image input means 20 Pixel position detection means 30 Calibration table storage means 31 Calibration table 40 Position calculation means 50 Laser irradiation apparatus C Camera

Claims (2)

The object is irradiated with a parallel light beam by the projector, and the target object irradiated with the parallel light is imaged by the camera. The positional relationship between the light emission position of the projector and the optical center of the camera, and the parallel in the captured image. A range finder for measuring the shape of the object from a light irradiation position,
Image input means for inputting an image captured by the camera;
In the image input by the image input means, pixel position detection means for detecting the pixel position of the irradiated parallel light beam,
Calibration table accumulating means for accumulating a calibration table in which the pixel position of the image captured by the camera, the direction of the incident light beam to the pixel and the amount of displacement from the reference position to the incident light beam are associated;
Based on the pixel position detected by the pixel position detecting means, the calibration table is referred to obtain the direction and displacement amount of the incident light beam corresponding to the pixel position, and use the direction and displacement amount of the incident light beam. A position finder for calculating the irradiation position. The reference position represents a fixed point that is in a fixed relationship with the camera,
The amount of displacement is represented by a vector from the reference position toward an arbitrary point on the incident ray,
The one arbitrary point is a base point for extending a vector in the sight line direction from the camera
The line-of-sight direction indicates the direction of light rays incident on the pixel,
The range finder according to claim 1, wherein the displacement amount is associated with the reference position for each pixel position in the calibration table.

Images