JPH0827840B2 - Range image generator - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a three-dimensional object recognition technique using a light-section method applied to, for example, the vision of a robot. When applying the light cutting method by irradiating slit light all at once, by associating each slit light with its slit image and labeling each slit image, the distance to the object is calculated and the distance is obtained. The present invention relates to a range image generation device for generating an image.

<Prior Art> In general, the “light cutting method” irradiates a plate-shaped slit light toward a three-dimensional object to generate an intersecting line of the slit light on the object surface, and the intersecting line is imaged by a television camera. The ray image (hereinafter referred to as “slit image”) is obtained by calculating the three-dimensional coordinates of the points on the intersection line from the coordinates of each point on the slit image and the equation of the plane of the slit light. is there.

However, in the case of this kind of method, only the information about the portion irradiated with the slit light can be obtained, which makes it difficult to recognize the entire object.

Therefore, in order to obtain information on the entire object, a large number of slit light beams are generated by scanning the slit light with a reflecting mirror, and each slit light is applied to the surface of the object. A method of forming the film by distributing it on the surface was proposed.

However, in the case of this method, the intersecting lines are imaged one by one at each scanning position of the slit light, and the distance image is generated by stereoscopically viewing the points on each slit image, so that the number of images of the intersecting lines increases, There is a problem that the time required for the recognition process becomes long.

In order to solve this problem, in recent years, a method has been proposed in which a large number of slit lights are generated at the same time and these are irradiated onto the object surface all at once.

When adopting this type of irradiation method, it is necessary to perform a process (which is referred to as "slit image labeling") for determining which slit light each of the plurality of slit images obtained by the television camera corresponds to. As one of the methods, a method has been proposed in which slit light is spatially coded with a gray code to illuminate an object (The 59th General Meeting of the Institute of Electrical Communication of Japan, "Distance Image Generation by Gray Code Space Projection").

<Problems to be Solved by the Invention> However, this method has a drawback that it is susceptible to external factors such as variations in the reflectance of the object surface.
Moreover, at least N (e.g., 8) imagings are required to obtain 2 ^N -1 (e.g., 256) slit images, and there is a limit in reducing the number of imagings.

The present invention is for solving the above-mentioned problem, and is a novel distance image generation device capable of labeling each slit image and generating a distance image with only two imagings, regardless of the number of slit images. The purpose is to provide.

<Means for Solving Problems> In order to achieve the above object, in the present invention, a projection means for simultaneously irradiating an object with a plurality of slit lights each having radiant energy independently set twice before and after. , The image pickup means for obtaining the slit image of each slit light by imaging the intersection line of each slit light generated on the object surface, and the brightness of the corresponding pixel for the two images obtained by the light irradiation twice before and after Table storage means for storing a label setting table for labeling each slit image from the relationship between the depth and the set value of the radiant energy before and after the radiant energy for each slit light, and each of the corresponding relationships. A distance calculation means for calculating the distance to the object based on the coordinates of each pixel and the label output from the table storage means is provided.

<Operation> When a plurality of slit light beams are simultaneously emitted to the object by the light projecting means, intersection lines of the slit light beams are simultaneously generated on the surface of the object. These intersecting lines are imaged by the image pickup means to obtain the slit image of each slit light. The same operation is performed twice before and after, whereby two images are obtained.

The brightness of the pixels corresponding to these two images is given to the table storage means, and the slit image of the slit image determined from the relationship between this brightness and the set value of the radiant energy before and after twice for each slit light is set. The label is output from the table storage means. The label output and the coordinates of each pixel are given to the distance calculating means, and the distance calculating means calculates the distance to the object from these data and generates a distance image.

Therefore, in the case of this device, it is possible to label each slit image and generate a distance image by only two times of imaging regardless of the number of slit images, and it is possible to shorten the processing time for object recognition.

<Embodiment> FIG. 1 shows a three-dimensional object recognition system in which the distance image generation device of the present invention is incorporated, and includes a multi-slit light source 1, a television camera 2, and a distance image generation unit 3. .

The multi-slit light source 1 of the illustrated example irradiates the three-dimensional object 5 to be recognized with a plurality of plate-shaped slit lights 6 in parallel from an obliquely upper position, and generates an intersecting line 7 where the slit lights intersect on the object surface. To do. In this case, each slit light 6 is simultaneously irradiated toward the object 5, and this light irradiation is performed twice before and after one object recognition process. Energy of each slit light 6 radiated within one frame period is controlled to a set value by the distance image generation unit 3, and as a control method therefor, a method of controlling the length of the lighting time within one frame period (PWM method) Either the method of controlling the number of pulses to be lit in one frame period (PNM method) or the method of controlling the pulse lighting frequency in one frame period (PFM method) is adopted. The setting of the radiant energy value for each slit light 6 is independent for each slit light,
Moreover, it will be carried out each time the light irradiation is performed twice before and after.

The television camera 2 images a plurality of intersecting lines 7 that are simultaneously formed on the surface of the object by irradiating the light twice from the position directly above the object 5 each time to generate a slit image of each slit light 6.

FIGS. 2 (1) and 2 (2) show an example of two images 9A and 9a obtained by imaging twice.

The first image 9A shows slit images A _{1 to} A ₃ of each slit light 6 by the first light irradiation, and the second image 9a shows the slit image a of each slit light 6 by the second light irradiation. _{1 to} a ₃ ,
Each slit image includes one or more segments (for example, the slit image A ₁ has three segments A ₁₁ , A ₁₂ , and A).
Consisting of ₁₃ ).

Returning to FIG. 1, the distance image generation unit 3 uses the television camera 2
From the above images 9A, 9a are sequentially captured, and each slit image A ₁
To A _3, a ₁ ~a each segment of ₃ per after performing labeling processing for each slit image by utilizing the brightness information, using the coordinates of the point on the label outputs and the slit image Then, the distance to the object 5 is obtained and a distance image is generated.

Assuming that the radiant energy of the existing slit light is P and the brightness of the slit image of this slit light is E, the following relationship is established between the two.

In the above formula, Is the position and orientation of the TV camera, Is the position / orientation of the object, α is the reflectance of the object, and f
Are these It is a function of α.

Next, when it is assumed that the brightness of the slit image changes from E ₁ to E ₂ when the radiant energy of the slit light is changed from P ₁ to P ₂ , the relation of the following equation holds.

According to this equation, the change rate of the radiant energy of the slit light and the corresponding change rate of the brightness of the slit image are the position and orientation of the TV camera. Position / posture of light source Object position / posture It can be seen that they agree with each other without depending on the reflectance α of the object.

The following equation is a more generalized expression of the above equation. In this equation, when m = 1, a = 1, b = 0, d = 1, e = 1, the relation of the above equation is derived. Get burned.

Thus, in the apparatus according to the present invention, when a plurality of slit light beams 6 are radiated to the object 5 simultaneously twice before and after, the relation between the radiant energies of the first time and the second time for each slit light (the relation on the right side of the equation). And when each slit image agrees with the relationship between the brightness of the first time and the brightness of the second time (the relationship on the left side of the equation), the label of the slit light is assigned to the slit image. In the case of this embodiment, when the rate of change in brightness for each slit image (left side of the formula) and the rate of change in radiant energy for each slit light (right side of the formula) match, the two correspond to each other. Then, it decides and labels it.

When the slit light label corresponding to the slit image is found from the above, the coordinates (I, J) on the image of the point on the slit image, the position / orientation of the TV camera Position and orientation of light source, Using, the three-dimensional coordinates (X, Y, Z) of the corresponding point on the object can be obtained by the following equation.

In the case of the device according to the present invention, the coordinate data I of each pixel,
The Z coordinate of the point on the corresponding object is obtained from J, and Z
By forming an array of = D (I, J), the distance image regarding the Z coordinate of the object is obtained.

FIG. 3 shows a specific example of the distance image generating apparatus according to the embodiment of the present invention.

In the figure, a multi-slit light source 1 is for irradiating a plurality of (for example, 128) slit lights whose radiant energies are independently set to an object simultaneously twice before and after, and a TV camera 2 is generated on the object surface. This is for obtaining the slit image of each slit light by imaging the intersecting line of each slit light.

The radiant energy of the k-th slit light at the time of generating the i-th image is represented by P (i, k), and the parameters representing the plane of the slit light are a (k), b (k), c (k). , d (k), the equation of the plane of the slit light in the xyz coordinate system is given by the following equation.

a (k) x + b (k) y + c (k) z + d (K) = 0
In the case of this embodiment, the positional relationship between the image pickup surface of the television camera 2 and the plane of the slit light is as shown in FIG.
The slit image 11 obtained by irradiating the plane of z = h is
Set so that it is perpendicular to the I axis in the IJ coordinate system. In the figure, 12 is a slit image obtained by irradiating a plane of z = 0, which is also perpendicular to the I axis. 13 is the slit image 11
Indicates the arbitrary pixel position above, and the brightness of the parallax is IMG
(I, I, J).

The television camera 2 outputs a time series signal including a video signal for the first image and a video signal for the second image in time series to the distance image generation unit 3.

The distance image generation unit 3 includes a switch 15 and two A / D converters 1
6, 17, an image memory 18, a sync generator 19, a label setting table memory 20, a stereoscopic view table memory 21, and a distance image memory 22. These circuit parts, the TV camera 2 and the multi-slit light source 1 are provided in the CPU 23, ROM 24. Then, a microcomputer composed of RAM 25 is configured to control in series.

The switching unit 15 is switched by the CPU 23 to the terminals S ₁ and S ₂ , whereby the time-series signal is separated into video signals of two images. The video signal of the first image is sent to one A / D converter 16, and the video signal of the second image is sent to the other A / D converter 16.
The signals are supplied to the / D converters 17 and converted into digital signals. The operation timings of these A / D converters 16 and 17 are appropriately determined by the synchronizing signals G ₁ and G ₂ from the sync generator 19, and the sync generator 19 receives the start signal ST from the CPU 23 and outputs the end signal ED. In the meantime
Generates and outputs various signals.

The output of one A / D converter 16 is given to the image memory 18, and the first image is written in the image memory 18 by the R / W signal (read / write signal) from the sync generator 19. The output of the other A / D converter 17 is given as it is to the table memory 20 for label setting. In this case, the sync generator 19 causes the A / D converter 1 to the image memory 18.
Pixel coordinate data I, J are given in synchronism with the operation timing of 7, the brightness IMG (1, I, J) of each pixel is sequentially read by the R / W signal, and the read output is the label setting table. Output to bell memory 20. The output of the A / D converter 17 is represented by IMG (2, I, J), and as a result, the brightness IMG (1, I, J) applied to the same pixel in each of the first and second images. ), IMG
(2, I, J) are sequentially applied to the table memory 20 in synchronization.

The brightness of each pixel has a data structure of, for example, 8 bits, and two brightnesses (8 bits + 8 bits = 16 bits) are given to the table memory 20 as address inputs. In this table memory 20, the slit light labels, which are respectively assigned based on the above-described principle, are stored in advance as a table for all combinations of two pieces of brightness information.

This label setting table stores two pieces of brightness information
If 1 and E2 are satisfied, the table data LAB (E1, E2) is set to m when the condition of the following formula is satisfied, and table data LA is not satisfied when the condition is not satisfied.
It is formed by setting 0 as B (E1, E2).

Note that ε is a threshold value.

In the above equation, abs is a function that takes an absolute value, Indicates the minimum value when k = 1 to N, respectively.

When the brightness IMG (1, I, J), IMG (2, I, J) of each pixel position (I, J) is given to the label setting table memory 20 as an address input, the position (I, J) The label m of the slit light corresponding to the pixel) is expressed by the following equation, and is output from the label setting table memory 20 to the next stereoscopic table memory 21.

m = LAB (IMG (1, I, J), IMG (2, I, J)) ... The stereoscopic table memory 21 stores a stereoscopic table A in advance. A calculates the z coordinate of the corresponding point on the object from the I coordinate of the image point on the imaging surface and the label m of the slit light by the stereoscopic technique using the light section method, and sets this in the table memory 21. It is formed by

Therefore, when the label m is supplied from the label setting table memory 20 to the stereoscopic table memory 21 and the I coordinate of the pixel is supplied from the sync generator 19 as an address input, the z coordinate of the point on the corresponding object is stereoscopically viewed. The read data is read from the usage table A according to the following formula, and the read data is output to the distance image memory 22.

z = A (m, I) ... The distance image memory 22 inputs the pixel coordinates (I, J) from the sync generator 19 as an address and the z coordinate as data. A range image D represented by the following equation is generated on 22 over the entire area.

D (I, J) = z ... Next, the operation of the entire apparatus will be described with reference to FIG.

First, the radiant energy values P (1, k) of the plurality of slit lights 6 are individually set, and then all the slit lights 6 are simultaneously radiated from the multi-slit light source 1 toward the object 5. On the surface of 5, the intersection line of brightness according to the magnitude of the radiant energy of each slit light 6 is generated. Each intersection line is imaged by the television camera 2, an image including all slit images is generated on the imaging surface of the television camera 2, and the video signal thereof is sent to the distance image generation unit 3.

Next, the radiant energy values P (2, k) of the plurality of slit lights 6 are individually set again, and the slit lights 6 are simultaneously radiated to the object 5 from the multi-slit light source 1. In this case, the radiant energy value P (2, k)
Is determined based on the rate of change set to a different value for each slit light.

An intersection line 7 of the slit lights is generated on the surface of the object 5 by the simultaneous irradiation of the slit light 6, the second slit image is captured by the television camera 2, and the image including all the slit images is displayed on the television camera. 2 is sent to the distance image generation unit 3.

Thus, in the distance image generation unit 3 having the configuration shown in FIG. 3, two brightnesses IMG (1, I,
J) and IMG (2, I, J) are taken out sequentially at the same time, and while labeling each pixel, a range image is generated at high speed.

<Effects of the Invention> As described above, the present invention irradiates a large number of slit lights at the same time to perform image processing thereof, and further, labels each slit image with only two imaging operations regardless of the number of slit images. Since the range image is generated, the number of times of imaging can be significantly reduced as compared with the conventional method, and the recognition processing time can be shortened. In addition, in labeling, there is a remarkable effect that the object of the invention is achieved, such that it is not adversely affected by external factors such as variations in the reflectance of the object surface and there is no risk of misjudgment.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of an object recognition system in which a distance image generating device of the present invention is incorporated, FIG. 2 is an explanatory diagram showing images obtained by two times of imaging, and FIG. 3 is a distance image generating. FIG. 4 is a circuit block diagram showing an embodiment of the apparatus, and FIG. 4 is an explanatory diagram showing a relationship between an image pickup surface of a television camera and a plane of slit light. 1 …… Multi-slit light source 2 …… TV camera 3 …… Distance image generator 20 …… Label setting table memory 21 …… Stereoscopic table memory

Claims (1)

[Claims] 1. A light projecting means for irradiating an object with a plurality of slit lights each having radiant energy independently set twice before and after, and an intersection line of the slit lights generated on the surface of the object. And the image pickup means for obtaining the slit image of each slit light, the relationship between the brightness of the corresponding pixels in the two images obtained by the two light irradiations before and after, and the radiant energy before and after the radiant energy for each slit light. Based on the table storage means for storing the label setting table for labeling each slit image from the relationship of the set values, and the coordinates and the label output by the table storage means for each pixel in the corresponding relationship. A distance image generation apparatus comprising: a distance calculation unit that calculates a distance to an object.