JP2695498B2 - Distance measurement processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial application] The present invention is a distance measuring process for automatically measuring three-dimensional shape information of a stationary rigid body from a plurality of images obtained by photographing the stationary rigid body from a plurality of viewpoints. Regarding the method.

[Conventional technology]

 Conventional techniques include the following two methods.

A method in which two to three cameras are installed apart from each other by a base line length L, and two to three images of a stationary rigid body are captured. A uniform length translational motion is performed by a baseline length L with one camera's visual axis facing the stationary rigid body. Then, a method of capturing N images of the stationary rigid body at a fixed time interval. In either method, the corresponding points between the captured images of the stationary rigid body are found, and the shape of the stationary rigid body is determined by the principle of triangulation. measure. Therefore, in order to increase the measurement accuracy, it is necessary to increase the base line length L as much as possible. In the method of (1), if the base line length L is set to be large, the corresponding points may be hidden, so that it is difficult to search for the corresponding points. On the other hand, since the corresponding point search problem can be replaced with the following straight line extraction problem, it is possible to assign corresponding points even if occlusion occurs.

As shown in FIG. 7 (a), the world coordinate system O-XYZ
Is set, the viewpoint is set on the X axis, and the visual axis is set to the positive direction of the Z axis. Assuming that the focal length is F, the three-dimensional object in the world coordinate system is centrally projected on a plane Z = F. Projection plane coordinates o-
xy has the origin at the intersection with the visual axis, and the direction of each coordinate axis matches the coordinate axis of the world coordinate system. While moving the viewpoint on the X-axis, the projected images are accumulated for each minute movement pitch δ _x . By arranging the obtained image sequences in order, a three-dimensional image (spatio-temporal image) as shown in FIG. 7B can be obtained. Let the X coordinate of the nth viewpoint position be U,
= N · δ _x . At this time, the point P in the world coordinate system
(X, Y, Z) is mapped to the point p (X ', Y', Z ') on the projection plane by the following equation.

Expressing this as projection plane coordinates (x, y), Equation (2b) means that the motion trajectory drawn by the projected point of the point P is constrained on a horizontal section, and equation (2a) is that the trajectory is a straight line, and the depth Z can be obtained from the slope of this straight line. Means that. Therefore, it is possible to measure the shape of a three-dimensional object in the world coordinate system by extracting a linear component from an image (epipolar image) obtained by cutting a spatiotemporal image along a horizontal line.

[Problems to be solved by the invention]

However, in the conventional method, when the straight line component is extracted, first, only the edge part having a large luminance change is extracted from the epipolar image by a differentiating process and the straight line component of this edge part is extracted. For this reason, it was impossible to extract the linear component in the part of the epipolar image where the change in brightness was gradual. The part where the change in brightness is gentle generally corresponds to the curved surface part of the stationary rigid body.
Therefore, the conventional method cannot measure the shape of the curved surface.

It is an object of the present invention to measure the shape of a stationary rigid body using a spatio-temporal image, and to enable shape measurement in a curved surface portion where the luminance change is gentle.

[Means for solving the problem]

In the present invention, when the linear component is extracted from the epipolar image, the epipolar image is once binarized at a certain brightness level V _q . In the binarized image, a false contour line appears in a portion where the luminance change is gentle. FIG. 1 is an explanatory diagram showing a situation where a false contour line appears, in which the horizontal axis represents position coordinates and the vertical axis represents luminance. This false contour line represents an equal luminance line in a portion where the luminance change is gentle. Therefore, binarization of the epipolar image is repeated while changing the brightness level V _q in the range from the minimum brightness level V _min in the epipolar image to the maximum brightness level V _max by δ _v , and each of the obtained binary values is obtained. By extracting the false contour line from the image, it is possible to measure the distance even in the part where the luminance change is gentle. That is, the epipolar image E is binarized by V _qi = i · δ _v + V _min (3) (1 = 0, ..., (V _max −V _min ) / δ _v −1). Obtained (V _max −
V _min ) / δ _v Each of the binary images E _qi has a brightness level V
Quantization of _qi produces false contours. Therefore, the false contour can be specified as a straight line by applying a differentiation process to the binary image E _qi and applying a straight line extraction method such as the Hough transform.

In the conventional technique, the edge portion is extracted by differential processing or the like without extracting the false contour line from the epipolar image, so that the information in the portion where the luminance change is gentle is removed, and the portion where the luminance change is gentle is removed. It was impossible to extract the linear component in.

[Action]

In the present invention, the false contour line seen in each binary image E _qi includes not only a pure false contour line in a portion where the luminance change is gentle but also a real edge portion where the luminance change is drastic as shown in FIG. Get lost. Therefore, if it is left as it is, there is a possibility that the same straight line may be extracted a plurality of times for the actual edge portion, which takes time and effort for calculation.

Therefore, by adding a real edge / false contour line separation process for separating a real edge portion and a false contour line portion in the following procedure,
More efficient straight line extraction is possible.

〔Example〕

First, a binary real edge image E _e is prepared in advance by extracting only the edge portion of the epipolar image E where the brightness changes drastically. The brightness change such as by differentiating the binary image E _qi extracted the intense partial binary image E _qi '
An exclusive OR (Exclusive-O) is performed for each pixel between (hereinafter referred to as a false contour line / real edge mixed image) and the real edge image E _e.
By taking R), it is possible to obtain the false contour line image F _{i from} which the edge part where the brightness change is drastically removed. FIG. 2 shows a processing flow for obtaining a false contour line image.

Since the point sequence on F _i is the constituent points of the straight line component by the false contour line, the Hough transform is applied to this F _i to extract the false contour component. For the edge part where the brightness changes drastically,
It is possible to easily extract the straight line component by separately applying the conventional straight line tracking algorithm to the real edge image E _e .

In the present invention, when a false contour line is extracted by the Hough transform, the Hough transform is generally performed by the following procedure.

The coordinate system of the false contour image F _i (or E _qi ) is ox−
If U, the straight line at F _i (or E _qi ) is expressed as x cos θ + U sin θ = ρ (4). In the Hough transform, the parameters of the straight line equation are transformed into θ and ρ as in equation (4). All the straight lines passing through a certain point (x ', U') on F _i are the loci represented by the equation (4) on the θ-ρ space. Therefore, the θ-ρ space is appropriately quantized to prepare a two-dimensional array K, and the loci represented by the equation (4) are accumulated on K for all points that are straight line candidates on F _i . Then, the linear component on F _i is detected as the maximum value of the cumulative frequency on K.

However, in the false contour line / real edge mixed image E _qi ′ (or the image F _i in which only the false contour line is extracted), the variation of the point sequence set forming the false contour line becomes considerably large. This is due to variations in the image sensor (CCD, etc.). Therefore, the criterion of direct detection in Hough transform is
Since it is simply “straight lines that maximize the number of passing point sequences”, it is not possible to find the exact slope of the false contour line (that is, to find the distance) with this alone. In the Hough transform, the cumulative frequency distribution on the two-dimensional array K forms Sidelobe around its maximum value. Therefore, in detecting the maximum value of the cumulative frequency, simply dividing the cumulative frequency by a threshold value may result in detection of multiple straight lines having similar parameters.

Therefore, straight lines are integrated by the following processing. Side
Due to the influence of lobe, a set C of constituent points of the false contour of F _i
For (f), if N straight lines l _i (i = 0, ..., N−1) are detected, if each straight line l _i is actually detected from the same false contour line Then, the set of passing point sequences C (fi)
Most of ⊆ C (f) are shared with each other. Therefore, straight lines are integrated and detected by the following procedure. FIG. 3 is an explanatory diagram for performing integration / detection of straight lines.

step1 (point sequence accumulation processing): As shown in FIG. 3, a band β in which each straight line l _i has a width d
Considering _i, we obtain the set of columns C (β _i) points contained in each band beta _i.

step2 (straight line integration process): If each of C (β _i ) is ‖C (β _i ) ∩C (β _j ) ‖> T (8), then the straight line l _j is integrated into l _i . However, T is a threshold value, and ‖ ・ ‖ is a set concentration.

step3 (least squares approximation process): A straight line L _i passing through the point sequence set C (β _i ) after the integration process is obtained by the least squares method.

In step 1 above, the passing point sequence set is obtained as a band in which the straight line l _i has a width d. By this processing, it is possible to enumerate the scattered point sequence sets forming the false contour line. In step 2, we investigated how much the enumerated point sequence was shared among the straight lines l _i , and integrated multiple lines that were erroneously detected due to the sidelobe effect. Then, in step 3, the final straight line is accurately detected using the method of least squares. The straight line integration processing enables accurate straight line extraction.

FIG. 4 shows a first embodiment of the present invention. In the figure, 1 is an epipolar image E, 2 is a minimum / maximum brightness level extraction process, 3 is a brightness level update process, 4 is a false contour line extraction process, 4-1 is a binarization process, and 4-2 is a binary image E. _qi , 4-3 is edge extraction processing, 4−
4 is the binarization processing, 5 is the false contour / real edge mixed image E _qi ′,
6 is a straight line extraction process, 7 is a straight line / three-dimensional information conversion process, and 8 is three-dimensional information.

First, a spatio-temporal image is constructed from a plurality of images taken in chronological order by an imaging device that translates a stationary rigid body at a constant speed.
One epipolar image E1 is extracted from the spatiotemporal image. The maximum brightness level V _max and the minimum brightness level V _{min of the} epipolar image E1 are extracted by the minimum / maximum brightness level extraction processing 2. The brightness level V _qi is generated from the minimum brightness level and the maximum brightness level by the brightness level update processing 3 shown in the expression (3). The false contour line extraction process 4 includes a binarization process 4-1, an edge extraction process 4-3, and a binarization process 4-4. The epipolar image is binarized 4-
A binary image E _qi 4 binarized by the luminance level V _qi by 1
-2. The binary image E _qi 4-2 is processed by edge extraction processing 4
-3, it is further processed by the binarization processing 4-4 to form the false contour line / real edge mixed image E _qi ′ 5. Straight line information is extracted from the false contour line / real edge mixed image E _qi ′ 5 by a straight line extraction process 6 such as Hough conversion, and three-dimensional information 8 is output by a straight line / 3D information conversion process 7. By the brightness level update process 3, the next brightness level V _{qi + 1} is generated according to the expression (3), and the same process is repeated. By performing the above processing on all epipolar images in the spatiotemporal image, the three-dimensional information of the stationary rigid body can be extracted.

With such a processing configuration, it is possible to extract straight line information even from a portion other than the edge where the change in luminance is gentle, and thus, the 3rd line in the curved surface portion of the stationary rigid body can be extracted.
It becomes possible to extract dimension information.

FIG. 5 shows a second embodiment in which real edge / false contour line separation processing is added to the configuration of the first embodiment of the present invention.
In the figure, 9 is a real edge / false contour separation process, 9-1 is a real edge extraction process, 9-1-1 is an edge extraction process, 9-1-2 is a binarization process, and 9-2 is a real edge. Images E _e and 9-3 are exclusive OR operations, and 10 is a false contour image F _i .

The real edge / false contour line separation process 9 is composed of a real edge extraction process 9-1 and an exclusive OR operation 9-3. The real edge extraction process 9-1 and the edge extraction process 9-1-1 are combined with each other. It is composed of the digitization processing 9-1-2.

First, a spatio-temporal image is constructed from a plurality of images taken in chronological order by an imaging device that translates a stationary rigid body at a constant speed.
One epipolar image E1 is extracted from the spatiotemporal image. Edge extraction processing 9-1-1 and binarization processing 9-1-2 are applied to the epipolar image E1 to generate a real edge image E _e 9-2. The real edge image E _e 9-2 represents only the real edge in the epipolar image E1. On the other hand, the epipolar image
The maximum brightness level V _max and the minimum brightness level V _{min of} E1 are extracted by the minimum / maximum brightness level extraction processing 2. The brightness level V _qi is generated from the minimum brightness level and the maximum brightness level by the brightness level update processing 3 shown in the expression (3). The false contour line extraction process 4 includes a binarization process 4-1, an edge extraction process 4-3, and a binarization process 4-4. The epipolar image is binarized by the binarization processing 4-1 at the brightness level V _qi to become a binary image E _qi 4-2. The binary image E _qi 4-2 is processed by the edge extraction process 4-3 and then the binary extraction process 4-4 to become a false contour line / real edge mixed image E _qi ′ 5. The false contour / actual edge mixed image E _qi ′ 5 has a mixture of real edges and false contours. Therefore, by performing an exclusive OR calculation 9-3 for each pixel between the false contour line / real edge mixed image E _qi ′ 5 and the real edge image E _e 9-2, a false contour line and a real edge are obtained. A false contour line image F _i 10 is generated by removing the real edges from the mixed false contour line / real edge image E _qi ′ 5. The false contour image F _i
The straight line information is extracted from 10 by the straight line extraction process 6 such as the Hough transform, and the three-dimensional information is output by the straight line / 3D information conversion process 7. By the brightness level update process, (3)
The next brightness level V _{qi + 1} is generated according to the formula, and the same processing is repeated. By performing the above processing on all epipolar images in the spatiotemporal image, the three-dimensional information of the stationary rigid body can be extracted. Separately, the real edge image E _e
By performing the straight line extraction process 6 on 9-2 and performing the straight line / three-dimensional information conversion process 7 on the obtained straight line information, the three-dimensional information 8 can be extracted from the actual edge in the epipolar image.

With such a processing configuration, it is possible to extract the three-dimensional information by separating the real edge portion and the false contour line portion, and it is possible to reduce the calculation labor.

FIG. 6 shows another embodiment of the straight line extracting process shown in FIGS. 4 and 5 of the present invention. In the figure, 6-1 is Hough transform,
6-2 is a point sequence accumulation process, 6-3 is a straight line integration process, 6-4 is a least square approximation process, and 6-5 is straight line information.

False contour line / real edge mixed image E in the first embodiment
_qi′5 or false contour image F _i 10 in the second embodiment
Or, for the real edge image E _e 9-2, first, the Hough transform 6
A straight line candidate is detected by -1. The point sequence accumulation process 6-2 is performed on each of the straight line candidates, and the point sequence set belonging to each straight line is counted. Then, in the straight line integration processing 6-3, the straight lines are integrated with respect to the shared point sequence exceeding the threshold value T. Finally, the straight line information 6-5 is extracted from the coordinate values of the point sequence belonging to the integrated straight line by the least square approximation process 6-4.

Due to this structure, the Hough
Even if a plurality of straight line candidates are detected due to the influence of delobe, it is possible to integrate them and to extract high-precision straight-line information, that is, high-precision three-dimensional information by least-squares approximation processing. .

〔The invention's effect〕

According to the present invention, it is possible to realize shape measurement of a portion, such as a curved surface shape of a stationary rigid body, which cannot be measured by a conventional measurement method using a spatiotemporal image.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing a situation where a false contour line appears, FIG. 2 is a processing flow for obtaining a false contour image, FIG. 3 is an explanatory diagram for performing integration / detection of straight lines, and FIG. 4 is a book. First embodiment of the invention, FIG. 5 is a second embodiment of the present invention, FIG. 6 is another embodiment of the straight line extraction processing used in the present invention, FIG. 7 (a)
(B) is a figure explaining a motion locus in a spatiotemporal image. In the figure, 1 is an epipolar image, 2 is minimum / maximum brightness level extraction processing, 3 is brightness level update processing, 4 is false contour line extraction processing, 5 is false contour line / real edge mixed image, 6 is straight line extraction processing, 7 is a straight line
/ 3D information conversion processing, 8 represents 3D information, and 9 represents real edge / false contour line separation processing.

Claims (3)

(57) [Claims] 1. A minimum brightness level and a maximum brightness level are extracted from the epipolar image using an epipolar image forming a spatiotemporal image obtained by time-sequentially imaging a stationary rigid body with an imaging device that performs translational motion at a constant speed. Minimum / maximum brightness level extraction processing, brightness level update processing for calculating a brightness level required for binarizing the epipolar image from the minimum brightness level and the maximum brightness level, and the brightness level update processing. False contour line extraction processing for obtaining a pseudo contour line / real edge mixed image by extracting isoluminance lines formed by the calculated brightness levels, and false contour line / real edge mixture obtained by the pseudo contour line extraction processing A straight line extraction process for extracting straight line information from the image and a straight line / 3D information conversion process for converting the straight line information extracted by the straight line extraction process into three-dimensional information are performed. Distance measurement processing method characterized by. 2. The false contour line according to claim 1 above.
A real edge / pseudo contour line separation process for separating a false contour line image and a real edge image from a real edge mixed image is added, and a straight line extraction process is separately performed on the false contour line image and the real edge image. A characteristic distance measurement processing method. 3. The Ho in which the straight line extraction processing according to claim (1) or (2) above extracts a straight line candidate from an image.
The ugh transformation, the point sequence accumulation process of counting the point sequence sets belonging to each of the straight line candidates, the line integration process of integrating the straight lines according to the degree of sharing of the point sequence set, and the straight line integrated by the straight line integration process. And a least-squares approximation process for calculating accurate straight-line information by the least-squares approximation from the coordinate values of the point sequence belonging to the distance measurement processing method.