JPH0623656B2 - Multi-view device - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-view device, and in particular, a pair of signals obtained from three or more imaging cameras correctly corresponds to a characteristic point of a target object. It is so arranged that whether or not it is a thing is judged in real time.

(Prior art) To control robots that bring automation to work,
A visual device capable of capturing three-dimensional information of a work target plays an important role. Methods for that purpose are classified into a range finder method in which light is actively projected onto an object and a method in which an image of the object obtained passively is processed.

Typical range finder methods are a triangulation method and a method of measuring from light propagation time. The advantages of this method are high measurement accuracy and simple processing. In particular, the triangulation method enables real-time measurement because it can perform accurate measurement with simple processing. On the other hand, the drawback of this method is that it is easily affected by the light environment, and if the image of the projected light cannot be separated from the image of the background light, or if the reflectance of the target object is low, the image position of the projected light is detected. If it cannot be done, correct measurement cannot be done.

As a passive method, a pair of image points corresponding to a characteristic point such as an edge of a target object is determined between a plurality of images captured from different directions, and three-dimensional coordinates of the point are obtained by triangulation. The image method is typical. There is also a method in which one camera is moved to pick up images a plurality of times to equivalently realize the functions of a plurality of cameras, but this can also be regarded as a kind of stereo image method. Since this method does not depend on the reflection state of light from a specific light source, there are relatively few restrictions on the light environment. However, although the determination of the corresponding points between images is the most difficult issue and many methods have been proposed, a method for accurately determining the corresponding points at a sufficient speed has not been established yet.

The basic principle of a normal corresponding point search is to focus on the fact that the images in the vicinity of the corresponding points are similar to each other, and use a method such as correlation to determine whether or not they are corresponding points. With this method, it is difficult to uniquely determine the correspondence when there are a plurality of similar image structures. Therefore, in order to determine the correct correspondence, it requires advanced processing such as combining a global search and a local search, or using a priori knowledge information. there were.

In this way, in addition to the technique of improving the performance by obtaining the signals from the two cameras and skillfully performing the processing,
A multi-view method in which an accurate correspondence search is performed by redundantly measuring a target object using three or more cameras has been devised. With this method, it is possible to perform accurate corresponding point search, but most of them require processing of a two-dimensional image, which makes the processing complicated and the execution time lengthy, making real-time measurement difficult. there were. Also, from this,
Since the execution time increases as the number of cameras increases,
There was a limit to the increase in the number of cameras. Furthermore, although a multi-eye visual device that uses a one-dimensional image sensor to perform measurement with relatively simple processing has been devised, this method has a limited function of measuring a distance in one direction, and the one-dimensional sensor is It was not a method that could measure many points in the observable plane. Therefore, in order to perform three-dimensional measurement, it is necessary to scan the entire device, and the high-speed camera for measurement has a limit.

(Problems to be Solved by the Invention) The present invention eliminates the drawbacks of the conventional multi-viewpoint corresponding point search method, and employs the multi-viewpoint method while performing high-precision corresponding point search in many measurements. You can go to the point in real time,
An object is to provide a multi-eye visual device.

(Means for Solving Problems) The multi-view visual device of the present invention uses three or more imaging cameras, the center point of the lens of each camera is on the y-axis, and the optical axis of each lens is the x-axis. Image sensors are arranged in parallel to the yz plane on the focal plane of each lens, and the main scanning direction is parallel to the y-axis so that all the cameras are synchronously scanned. Then, when the pair of signals obtained from the corresponding scanning lines on the same z-axis axis of each camera satisfies the following relation, it is determined that this is a pair of image points that correctly correspond to the characteristic points of the target object. It is equipped with a signal processing circuit.

(U _{i + 1} −u _i ) / l _i = Const. Where i represents the camera number (i = 1 to n), l _i represents the distance between the adjacent cameras _{i + 1} and _i ,
u _i represents the position of the signal on the corresponding scan line of camera i.

(Operation) According to the above-described configuration, for each of the actual characteristic points of the target object, there is a signal that satisfies a certain relationship determined by the relative positional relationship of the cameras for all cameras. If the pair of signals of is measured, it is determined to be correct. The probability that the signal of another feature point is arranged in the above-mentioned relationship will decrease sharply as the number of cameras increases, and it is almost impossible to make false correspondence. On the contrary, in response to an erroneous signal, a signal satisfying the above-mentioned certain relationship cannot be obtained, so that it is determined as an erroneous signal.

(Example) Hereinafter, an example of the present invention will be described in detail with reference to FIGS. 1 and 2.

FIG. 1 is an explanatory diagram showing the configuration of an example of a multi-eye visual device used in the present invention.

The visual device is composed of three or more cameras C ₁ , C ₂ ... C _n , each camera C _i ( _i = 1 to n) is the same for simplification of description, and the lens of each camera C _i is used. L
The center point of _i ( _i = 1 to n) is on the y-axis, and each lens L
The optical axis of _i is parallel to the x axis. The image sensor I _i ( _i = 1 to n) formed of, for example, a CCD arranged in the focal plane of each lens L _i parallel to the yz plane is also the same, and has the same relative distance to the optical axis of each lens L _i. They are arranged in a positional relationship, and the main scanning direction is parallel to the y-axis so that they are all scanned synchronously. Therefore, the position in the main scanning direction where the optical axis of the lens L _i intersects the image sensor I _i is the same for all cameras (this position is u ₀ ). Each image sensor I _i image of an arbitrary point P of arrangement is also projected that for on each image sensor I _i the same in the z-axis direction is the same height of each image sensor I _i (Z
Obtained on the scan line (coordinates are equal).

By the way, from the relative positional relationship of the cameras C ₁ , C ₂ ... C _n , the relationship as shown in the following expression (1) is established.

However, X is the x coordinate of the point P (hereinafter simply referred to as the distance of the point P), and l ₁ ... L _n-1 are adjacent cameras C ₁ , ... C _n.
The distance between, also an angle between α _{1 ...} α _n is the lens L ₁ ... L _n lines and the optical axis of the center point and a line connecting the point P is obtained by projecting onto the xy plane, respectively It represents.

Here, a relation that is made up of the characteristics of the cameras C ₁ , ... C _n ,
tan α _i = (u ₀ −u _i ) / f [where u _i ( _i = 1
Up to n) represent the position of the image of the point P on the image sensor I _i of the camera C _i , and f represents the focal length of the lens L _i . f is the same for all lenses L _i ], the following formula (2) is obtained.

What can be said from the above is that an image of a certain feature point satisfies the relative positional relationship of the above equation (2) on the scanning lines of the same height with respect to all the cameras C ₁ , ... C _n . It means that it will be measured. Conversely, by checking whether or not this relational expression (2) holds for the signals of the image points obtained by all the cameras C ₁ , ... C _n ,
That is, it is possible to determine whether or not each of the obtained signals relates to an accurate corresponding point. With normal binocular vision,
In many cases, the feature points of different images are made to correspond to each other and incorrect distance measurement is performed.However, according to this method, such a correspondence relationship of erroneous feature points is Since the correspondence does not satisfy the expression (2), it can be removed.

In this method, the probability of false positive occurrences decreases exponentially as the number of cameras increases. Practically, it is possible to search corresponding points with almost no misjudgment using a multi-view system of about 4 to 6 units. In addition, in the visual device shown in FIG. 1, particularly in the case of a configuration satisfying l ₁ = l ₂ = ... -l _n-1 , the image of the point P on the light receiving surface of the adjacent cameras C ₁ , ... C _n The phase difference (u _{i + 1} −u _i ) of the position is
As is clear from the equation (2), since all are equal, the determination is further simplified, and the value of this phase difference uniquely gives the value of the distance X. Further, in the above description, it is assumed that all cameras are the same, but even if a combination of cameras having different sizes is used, a relationship similar to the expression (2) can be obtained, and the present method is applied. be able to.

FIG. 2 shows an example of a specific circuit for executing the above-mentioned discrimination process. For simplification of description, it is assumed that the number of cameras is 5, and that the video signal has been subjected to the feature point extraction processing such as edges. The image position values u of the characteristic points obtained from the cameras C ₁ and C ₅ (in this case, assume values of 0 to 255) are stored in the order of addresses in the RAMs ₁ and ₅ , respectively. Further, the signals of the scanning lines obtained from the cameras C _{2 to} C ₄ are converted into a sequence of 0 and 1 according to the presence / absence of edges, and RA sequences are respectively calculated.
It is stored as 0 or 1 in the address corresponding to the position u (0-255) of each signal of M2-4. Then, the following processing is performed in this circuit. First, the cameras C _{1 on} both ends
From C ₅ of RAM1, RAM5, a 2-byte counter 6
U ₁ and u ₅ are sequentially read by using the upper byte and the lower byte of the output of ₁ as the respective addresses (the maximum value of the image position u is 255, so 1 byte is sufficient as the address).

Next, it is assumed that u ₁ and u ₅ read out at the same time have a correct correspondence relationship and are signals from the same feature point, and the feature is set at any position of the scanning signals of the intermediate cameras C _{2 to} C _4. Estimate whether or not the indicated signal should be generated according to equation (2). In this case, u ₅ −u ₄ = u ₄ −u ₃ = u ₃ −u ₂ = u ₂ −
Since u ₁ = (u ₅ −u ₁ ) / 4, the subtractor 7 outputs u ₅
And u ₁ are input to output (u ₅ −u ₁ ) and constants of 1/4, 1/2, and 3/4 are respectively added to the multipliers 8, 9, and
Each phase difference (u ₂ −u ₁ ) obtained by multiplying by 10,
(U ₃ −u ₁ ) and (u ₄ −u ₁ ) are added to u ₁ by the adders 11, 12, and 13 to derive u ₂ , u ₃ , and u ₄ . And
The calculated addresses u ₂ , u ₃ , and u ₄ are input to and read from the RAMs ₂ , ₃ , and ₄ of the cameras C ₂ , C ₃ , and C ₄ , and characteristic points are shown at corresponding positions on all light receiving surfaces or in the vicinity thereof. Check if the signal is being received. If there is a signal that correctly corresponds to all the cameras C ₂ , C ₃ , and C ₄ , a signal I is obtained from each of the RAMs ₂ , ₃ and ₄ , and therefore an output is obtained from the AND circuit 14, and this signal causes the latch circuit 15 to be output. It is set and u ₁ and u ₅ at that time are output. Thus u
The combination of ₁ and u ₅ is determined to be a signal obtained from the correct feature point, and the result is output. This processing is performed for the combinations of the values of the _u 1, _{u 5,} it is possible to obtain a correct signal pair _(u 1, u _5).

The processing time of this circuit is 10MHz clock frequency,
Assuming that N number of characteristic points are obtained by the camera C ₁ and M number of characteristic points are obtained by the camera C ₅ , the processing time can be estimated as 0.1 · N × M μsec. Therefore, when N = M = 128, the processing time is 1.6 msec, but in the normal case, if about 50 pieces of data are input, the processing can be performed in about 250 μsec. Since the latter processing time is equivalent to four scanning lines of a normal television camera, sufficient real-time property can be obtained.

As described above, in the present invention, as a feature point in the camera image, for example, attention is paid to an edge (a feature point can also be a signal at a position that crosses the set threshold value intensity of a specific wavelength, etc.), It is determined whether or not all cameras hold the relationship shown in equation (2). This determination process is remarkably simple, easily implemented in hardware, and can be processed in real time, as in the example described above. The number of combinations of data to be compared is M × N at maximum, _assuming that M and N feature points are detected by the cameras C ₁ and C _n at both ends. This number of combinations is an upper limit value when only the characteristic points that can be measured by all cameras are measured, and does not depend on the total number of cameras. Therefore, if the number of cameras is increased, it is possible to provide a sufficiently high erroneous judgment removal capability in the same processing time.

By the way, if a projection device for projecting an appropriate pattern is added to the above-mentioned configuration, the edge of the projected pattern can be detected as a feature point.
As a result, since a large number of feature points are generated even for a target object having a single color and a smooth surface and having few feature points, a range finder device capable of measuring a wider area is configured. can do.

Further, one of the cameras is replaced with a beam light source or a slit light source capable of knowing the projection direction, and the projection angle signal is regarded as a definite detection signal obtained from the camera, and the reflection point is used as a feature point. If you process
Since most of the disturbance signals detected by other cameras do not satisfy the equation (2) and are removed, it is possible to detect the projection signal that is hardly influenced by the background light. Therefore, it is not affected by the background light environment,
It is possible to construct a highly reliable range finder device.

Although the above description has been made assuming that a two-dimensional image sensor is used, in the present invention, since each scanning line is processed independently, it means that essentially one-dimensional processing is performed. Therefore, the present invention is not limited to two-dimensional sensors,
The same can be applied to a visual device including a one-dimensional sensor.

The present invention is not limited to the above embodiment,
Various modifications are possible based on the spirit of the present invention, and these are not excluded from the scope of the present invention.

(Effects of the Invention) As described in detail above, according to the present invention, since multi-view is the basis, accurate corresponding point determination is possible in real time, and at that time, simple determination is possible. Since only the processing is performed, the signal processing circuit to be used can be easily implemented as hardware. Further, there is a low possibility that an erroneous measurement will occur due to the background light environment, and highly reliable measurement can be performed at many measurement points.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing a configuration of an example of a multi-eye visual device used in the present invention, and FIG. 2 is a circuit diagram showing an example of a signal processing circuit used in the present invention. C ₁ , C ₂ ... C _n ... Camera, L ₁ , L ₂ ... L _n ... Lens, I ₁ , I ₂ ... I _n ... Image sensor, P ... Characteristic point of target object, 1, 2, 3, 4, 5 ... RAM,
6 ... Counter, 7 ... Subtractor, 8, 9, 10 ... Multiplier, 11, 12, 13 ... Adder, 14 ... AND circuit, 15 ...
Latch circuit.

Claims (3)

[Claims] 1. Using three or more imaging cameras, the center point of the lens of each camera is on the y-axis, the optical axis of each lens is arranged parallel to the x-axis, and each lens is parallel to the yz plane. An image sensor is arranged on the focal plane, its main scanning direction is parallel to the y-axis, and all cameras are synchronously scanned, and corresponding scanning lines on the same z-coordinate axis of each camera. A multi-eye visual device characterized by including a signal processing circuit for determining a pair of signals obtained from the image point pair that correctly corresponds to a characteristic point of the target object when the following relation is satisfied. . (U _{i + 1} −u _i ) / l _i = Const. However, i represents the camera number (i = 1 to n), l _i represents the distance between the adjacent cameras _{i + 1} and _i ,
u _i represents the position of the signal on the corresponding scan line of camera i. 2. The multi-eye visual device according to claim 1, wherein all the l _i are arranged so as to be equal to each other. 3. A phase difference (u) between adjacent cameras from the positions u ₁ and u _n of the signals of the cameras at both ends of the signal processing circuit.
_{i + 1-} u _i ), from which the position of the signal of the remaining camera should be derived, and whether or not there is an actual signal at the derived position is determined. The multi-eye visual device according to claim 1 or 2.