JPH0474155B2 - - Google Patents

Description

[Detailed Description of the Invention] [Field of Application of the Invention] The present invention detects an optical image of a target object and analyzes it to detect the position and orientation of the target object. The present invention relates to an apparatus for detecting the position and orientation of a target object and a method thereof.

[Prior art]

Conventionally, the method of detecting the position of an object is to uniformly illuminate the entire object, capture this optical image with a TV camera, binarize the image thus obtained, and process this binary image. This method has been widely used. However, in this method, the color of the object is significantly different from the background, and it is necessary to separate and extract only the object in the image through binarization. Also, since the position in the depth direction cannot be detected with this method,
If an object has to be grasped or a part attached to it, such as in robotic assembly, this object must be at a known distance from the detector, thus limiting the scope of work. Ta.

Also, in the Showa era, it was used for automatic welding by robots.
11th International Symposium on Industrial Robots held in 1956 (11th Int.Symp.on
Industrial Robots (1981)) pp. 151-158
T.Bamba published on page
et al., “A Visual Sensor for Arc-
Welding Robots”, a paper titled “Welding Robots” shows that an optical cutting detection head is attached to the robot’s hand.
There is a method of automatically detecting the welding position by processing the detected optical cutting waveform. However, this method cannot be used for assembly work because the work target is limited to groove-shaped objects.

Furthermore, an attempt to apply this method to assembly work was made at the 9th International Symposium on Industrial Robotics (G th Int.
Symp. on Industrial Robots (1979), pp. 123 to 230, by GJ Van der Brug and others, “A Vision System for Real Time.
There is an NBS method proposed in a paper titled "Control of Robots." In order to completely detect the position and orientation of an object three-dimensionally using this method,
This method has the disadvantage that detection takes time because it is necessary to move the robot arm carrying the detector and view the target from several different angles.

The object of the present invention is to eliminate the drawbacks of the prior art described above, and to provide a target object that can detect the position and orientation of the target object at high speed using a small and lightweight visual device without being affected by the color of the target object. An object of the present invention is to provide a position and orientation detection device and a method thereof.

[Summary of the invention]

In order to achieve the above object, the present invention provides a device and method for detecting the position and orientation of a target object. an imaging means for capturing an optical image of the line of intersection between a light beam and the surface of the target object and converting it into an electronic signal; a light beam switching means for switching and irradiating two light beams irradiated from the light irradiation means; a separating and extracting means for separating and extracting a bright cutting line signal on an image from an electrical signal obtained from the means; a storing means for storing a correspondence relationship between positional information on the image captured by the imaging means and actual positional information; The gist of the present invention is to include a calculation means for analyzing the position and orientation of the target object from the cutting line signals separated and extracted by the separation and extraction means and the correspondence stored in the storage means. The present invention also provides a light irradiation means for irradiating at least two flat light beams that are not parallel, and an optical image of the intersection line between the two light rays irradiated by the light irradiation means and the surface of the target object. When converting into an electrical signal, an image means is provided on the operating member of the robot, and a light beam switching means is provided to switch and irradiate two light beams irradiated from the light irradiation means, and an image is converted from the electric signal obtained from the image pickup means. a separating and extracting means for separating and extracting bright cutting line signals; a storing means for storing the correspondence between positional information on the image captured by the imaging means and actual positional information; a calculation means for analyzing the position and orientation of the target object from the generated cutting line signal and the correspondence relationship stored in the storage means; and a calculation means for analyzing the position and orientation of the target object based on the position and orientation of the target object analyzed by the calculation means. This is a position and orientation detection device for a target object, characterized in that it is equipped with a robot control means for controlling the object. Further, the present invention resides in that the optical axis of the imaging means is arranged on a plane that bisects the angle formed by the two planes formed by the two flat light rays. In an advantageous embodiment of the present invention, the line of intersection of the planes formed by the two planar light rays coincides with the rotation center axis or the geometrical symmetry axis of the hand of the robot, and the optical axis of the imaging device also coincides with the line of intersection of the planes formed by the two flat light beams. passing through the line.

The present invention is based on the following knowledge of the inventors.

As a method for detecting the position and orientation of an object without being affected by its color, there is a method that applies light sectioning.

As shown in FIG.
A detector composed of a and an imaging device 2 is used.
Flat light beam 3 projected from the slit light source 1
When the intersection line between the slit light (hereinafter referred to as slit light) and the surface of the object, that is, the light cutting line 4, is imaged by the imaging device 2 from an oblique direction that has a certain angle with the optical axis of this light beam, for example, as shown in FIG. As shown, the cross-sectional shape of the object is obtained. The farther a point is from the detector, the more
Since it is located above on the image, by extracting this light cutting waveform and detecting and processing the positions of points a and a' on the image in Figure 2, we can determine the distance to the object and the left and right sides. The position of the direction can be detected. Also, the angle θ
By detecting , the rotation angle of the object around the axis perpendicular to the plane formed by the slit light can be detected.

As shown in FIG. 3, slit light sources 1a and 1b
By combining two slit beams, cross-shaped slit beams 3a and 3b which intersect with each other are created.
The optical cutting line is imaged by the imaging devices 2a and 2b.
With such a configuration, the rotation angle of the object on two non-parallel planes can be detected, so the attitude of the object can be detected three-dimensionally without moving the detector. In addition to the distance to the object, the size and position of the object in two directions can also be detected.

However, in such a configuration, two slit light sources and two imaging devices are required, so the detector becomes large and difficult to attach to the robot hand. Therefore, it cannot be used for applications in which the relative positional relationship between the workpiece and the robot's hand is detected by the hand's detector, feedback is given to the robot, and the robot's hand is precisely positioned on the workpiece.

Therefore, in the present invention, as shown in FIG. 4, the two slit lights are rotated in the same direction around their intersection line, so that the optical cutting line of the two slit lights can be detected with one imaging device. The detector is designed to be smaller in size. When two slit light sources can be installed, the two slit light sources can be switched to emit light one by one in order to extract the two light cutting lines independently, simplifying subsequent processing and speeding up the detection time. Plan.

In the present invention, the cross-shaped slit light shown in FIG. 4 is further obtained by arranging two slit light sources in an inverted V-shape as shown in FIG. That is, in FIG. 4, the slit light source is located on the plane 3.
a, 3b, and those planes 3a, and 3
As long as the angle between the intersection line b and the optical axis 8 of the imaging device is smaller than 90°, it can be moved and placed anywhere.

Hereinafter, the present invention will be explained in more detail using examples with reference to the drawings, but these are merely illustrative and it is understood that various modifications and improvements may be made without going beyond the scope of the present invention. Of course.

[Embodiments of the invention]

Here, in order to make the explanation easier to understand, an example of an apparatus configuration in which the x-axis and the y-axis are symmetrical with respect to a plane passing through the intersection line of the imaging device and two planes shown in FIG. 4 will be explained. The same applies to the case shown in FIG. 5, which shows the configuration of the apparatus of the present invention.

In the principle diagram of the detector shown in Fig. 4, when two slit light sources each emit light one by one, the correspondence relationship between the light cutting line detected by the imaging device and the actual position of the object is shown. has a diagonal lattice shape as shown in FIGS. 6a and 6b. Figure 6 a is the xz plane,
b represents correspondence on the yz plane, solid lines are equispaced lines, and broken lines are equidistant lines. This correspondence is determined in advance using an object whose dimensions are known while changing its position from the detector, and is stored in a storage device. When detecting the position and orientation of an object, the actual position of the object is determined by comparing this pre-stored correspondence with the position on the image of the optical cutting line detected by the imaging device. You can find the position and tilt. In the present invention, two slit lights are switched and projected, and the distance, position, and inclination to the object on two non-parallel planes are determined, so the three-dimensional position and orientation of the object can be uniquely determined from these. be able to.

In addition, in the present invention, the intersection line 5 of two slit lights
Assume that the detector or the object is roughly aligned in advance so that it intersects the object surface,
It is assumed that the relative position and orientation of the detector and the object are precisely detected using the apparatus of the present invention.

Next, a method for determining the correspondence between the position of the detected light section line on the image and the actual position of the object will be explained using FIG. Here, only the case of one of the slit light sources shown in FIG. 4 1a will be explained. The same applies to the case of FIG. 4 1b and the case of FIG. As shown in FIG. 7, a coordinate system xyz is assumed in which the line of intersection of two slit lights coincides with the z-axis, and the slit light according to the slit light source 1a coincides with the xz plane. First, as shown in FIG. 7, a plane 6 perpendicular to the z-axis and wider than the field of view of the imaging device is erected on a scaled rail 7 so that it can be moved in parallel in the z-axis direction. If the position of the light section line on the detected image is determined while moving the plane 6 away from the detector in increments of, for example, 1 cm, an equidistant diagram as shown in FIG. 8 will be obtained. Next, the plane 6 is made into a rectangle with a constant width, and such rectangles are prepared in several widths, and they are also erected perpendicularly to the z-axis on the rail 7 and moved in the z-axis direction. plane 6
For example, if the width measured in the x direction is set at 2 cm increments, and each plane is moved in the z-axis direction, the locus of the end point of the detected light section line is found to be a monospaced line as shown in Figure 9. A diagram (in terms of x-coordinate) is obtained. In the equidistant line diagram shown in Figure 8 and the equispaced line diagram shown in Figure 9, a certain horizontal line h on the diagram,
Looking at changes in distance along h' and position in the x direction, the results are shown in FIGS. 10 and 11, respectively.
If these are expressed as a function z of distance at coordinate j in the vertical direction on the diagram and a function x of position in the x direction, and coordinate i in the horizontal direction on the diagram is expressed as a variable, zj = f(i) = a It can be approximated as _j0 +a _j1 i+a _i2 j ² +a _j3 i ³ + ...(1) xj=g(i)=b _j0 +b _j1 i+b _i2 i ² +b _j3 i ³ + ...(2). The degree of approximation is determined based on the required detection accuracy. For every j on the diagram (for example, j = 1 to 256), a _jo ,
If b _jo (n = 0, 1, 2, ...) is determined in advance using the above method, equations (1) and (2) can be obtained from the detected optical cutting line.
Conversely, its position in the z and x directions can be determined using . In addition, when determining the inclination of a plane constituting the detection target, the actual coordinates (x, z) of the end points of the optical cutting line determined as above
From (x', z'), the angle between that plane and the x-axis can be found using θ _x = tan ^-1 z'-z/x'-x (3).

Next, a method for extracting a light section line from a detected image will be specifically explained. In this case as well, the case of the one-sided slit beam 3a shown in FIG. 4 will be explained. The optical cutting line imaged by the imaging device 2 generally has a certain width due to the width of the slit light, the blur of the optical system, etc. Therefore, the horizontal scanning direction of the imaging device is arranged to match the directions h and h' (horizontal direction) of the diagrams shown in FIGS. 8 and 9, and the peak position of each horizontal scanning signal (i.e., the brightest Let the coordinate i of point ) be the position i of the light section line at each j. (FIG. 12) In this way, the coordinates of the position of the optical cutting line for calculating equations (1) and (2) are extracted.

Next, the process of detecting the three-dimensional position and orientation of an object by the apparatus of the present invention will be described using as an example the detection of a cylindrical object 9 shown in FIG. 13 and a circular hole 10 formed on the surface shown in FIG. explain.

First, a method for detecting the position and orientation of the cylindrical object 9 will be explained. Suppose that when the slit light shown in FIG. 5 3a is projected, a light cut image as shown by the solid line in FIG. 15 is detected by the imaging device 2, for example. In the figure, as explained earlier, when the position of the brightest point in the horizontal direction is detected and the shape of the light cutting line is extracted, the 16th
It will look like the figure. This waveform is approximated by a broken line, and the coordinates (i, j) of the end points of each line segment are calculated in advance using equation (1)
When converted to actual coordinates (x, z) using (2),
The result will be as shown in Figure 17. In this figure, the most z
An approximate center position M in the x direction of the upper surface of the cylindrical object is determined from the midpoint of the line segment A with the smallest coordinates, that is, the line segment closest to the detector. Also, equation (3)
The inclination θ _x of the object in the xz plane is determined using . Next, switch the slit light source, Figure 5 3
By performing exactly the same detection on the optically sectioned image (broken line in Figure 15) obtained for the other slit beam in b, the approximate center position of the upper surface of the cylindrical object in the y direction and the object on the yz plane can be determined. Find the slope of Furthermore, based on the above detection results,
Distance z from the detector to the center position of the top surface of the object
If the top surface of the cylinder is almost perpendicular to the z-axis, then z=z ₁ +z ₂ /2+z ₄ −z ₃ /y ₂ −y ₁・y ₁ +y ₂ /2 (4) or z=z ₃ +z ₄ /2+z ₂ −z ₁ /x ₂ −x ₁・x ₁ +x ₂ /2 (5) Here, x ₁ and x ₂ are the x of the end point of the optically sectioned image of the xz plane on the top surface of the cylinder (Fig. 15A).
Coordinates y ₁ and y ₂ represent the y coordinates of the end points of the optically sectioned image of the yz plane (FIG. 15B), and z ₁ , z ₂ , z ₃ and z ₄ represent the z coordinates of the four end points, respectively. From the above detection results, if the inclinations of the top surface of the cylinder in the xz plane and zy plane are significantly different from 90°, the method of determining the center position from the midpoint of the light section line and the calculation of the distance using equation (4) or (5). If it is determined that the error cannot be ignored, the robot
The arm is moved so that the z-axis, that is, the line of intersection of the slit beams, is approximately perpendicular to the surface, and the center position and distance are detected again.

Next, a method for detecting the center position of a circular hole perpendicular to a plane and the direction of the hole will be described. FIG. 18 shows an example of a light section image detected by the imaging device. In this case, as in the previous example, two sets of optically sectioned images on the xz plane and the yz plane are independently detected by switching the slit light source. A solid line and a broken line in FIG. 18 respectively indicate two sets of optically sectioned images obtained in this manner. In the same figure, the line segment connecting points a ₁ and a ₂ and the line segment connecting points b ₁ and b ₂ are respectively the 15th line segment of the previous example.
By performing processing corresponding to line segments A and B in the figure, the center position of the hole and the direction in which the hole is formed are detected in exactly the same way.

A first embodiment of the detection device according to the present invention will be described below.
This will be explained with reference to FIG.

The detector includes a TV camera 18 and a slit light source 1.
It consists of parts a and 1b, and is mounted on the arm 11 of the robot in the following arrangement.

That is, (1) The intersection line 5 of the plane formed by the two slit lights 3a and 3b is made to coincide with the rotation axis 12 of the robot's hand.

(2) The optical axes 8a, 8b of the slit light sources 1a, 1b and the optical axis 8c of the TV camera 18 intersect at approximately one point on the intersection line 5 of the slit lights.

(3) Slit light sources 1a, 1b and TV camera 1
8 should be lined up horizontally.

(4) The angle formed by the slit lights 3a and 3b is 90°.

The entire device is controlled by a microcomputer (or minicomputer) 15. That is, the slit light switching device 13 is controlled so that the slit light sources 1a and 1b emit light one by one,
The optically sectioned image at that time is detected by the TV camera 18. The detected TV image signal is input to the optical cutting line extraction circuit 14, where the optical cutting waveform is extracted and output to the microcomputer 15. As described above, this optical cutting line extraction circuit 14 compresses two-dimensional image data into one-dimensional waveform data, and is, for example, the device shown in Japanese Patent Application No. 146,427/1983. The two light-cutting waveform data obtained in this way are processed in the microcomputer 15, and as explained earlier, the waveform data polygon line approximation formulas (1) to (5) are used.
is calculated to detect the three-dimensional position and orientation of the target object, and the data 17 is output to the control device of the robot arm mechanism. Coefficients when calculating equations (1) and (2)
a _jo , b _jo (j=1, 2, 3,..., n=0.1, 2,...)
is determined in advance by the method described above, and is stored in storage device 1.
6, and can be taken out and used as needed.

Next, the operation of the robot visual system according to this embodiment will be explained.

Before detecting the object, as already explained, it is necessary to store the coefficients shown in equations (1) and (2) (and the coefficient of y _j obtained in the same manner) in the storage device. Using the device shown in Fig. 7, the robot is operated and fixed so that the line of intersection of the slit lights is parallel to the rail 7, and the equidistant line diagram (Fig. 8) and the equispaced line diagram are drawn as already explained. (Figure 9) is obtained. This operation is switched by the slit light switching device 13 and is performed for each of the two slit lights 3a and 3b. Next, based on these data,
The computer 15 performs polynomial approximation to obtain the coefficients of equations (1) and (2) for each j and store them in the storage device 16.

In the target object detection process, as already explained using FIGS. 13 and 14 as examples, the computer 15
controls the slit light switching device 13 to obtain the extracted waveform of the optical cutting line when the two sets of slit light sources emit light from the optical cutting line extraction circuit 14, and after this waveform is approximated by a broken line by the computer 15. , the actual position coordinates of each line segment are determined using equations (1) and (2) (and similar equations for calculating y _j ) based on the data stored in the storage device 16. When the two sets of slit light sources are activated on the computer 15, the line segments on the target object are selected one by one using the distance z as a criterion (usually the line segment closest to the detector and in the field of view). Line segment near the center), from each of the four end points, formula (3) ~
(5) and the midpoint of the line segment are calculated to detect the position and orientation of the target object and output them to the control device of the robot arm mechanism.

In this embodiment, since the TV camera and slit light source are arranged as shown in (2) to (3), the detector can be made small and compact. In addition, since the slit light is arranged as shown in (1) to (4), the z-axis always points in the same direction with respect to the rotation of the robot hand, and the coordinate system xyz becomes a rectangular coordinate system, which allows the robot arm mechanism to It is easy to convert the detection results output to the control device into the coordinates of the robot arm mechanism system. Also, the expression
As shown in (1) and (2), since the equidistant line diagram and the equispaced line diagram are approximated by functions, the amount of data can be smaller than if each line diagram is stored as is.

(For example, when i=1 to 256 and a fourth-order approximation is performed, about 1/50) [Effects of the Invention] As explained above, according to the present invention, the light source is arranged so that the two slit lights intersect with each other, By analyzing each optical section line, the position and orientation of the object on two non-parallel planes can be detected without moving the detector or the target object, so the three-dimensional position and orientation of the object can be detected at high speed. There is an effect that can be done. In addition, since the cross-shaped slit light is obtained from two sets of slit light sources arranged in an inverted V-shape, it is possible to integrate the imaging device and the slit light source, making it compact and lightweight. This has the effect that detection is possible without being photographed by the grabbed object. Furthermore, since the optical cutting lines in two non-parallel planes are detected by one imaging device, the detector can be made smaller and lighter. Furthermore, the effect of the light sectioning method, which is a premise of the present invention, makes it possible to detect the position and orientation of an object without being affected by its color or surface condition.

[Brief explanation of the drawing]

Figure 1 is a perspective view of a detector for explaining the light sectioning method, Figure 2 is a diagram showing an example of a detected image obtained by the detector shown in Figure 1, and Figure 3 is a diagram showing two sets of the first image. FIG. 4 is a perspective view of a detector based on the principle of the present invention; FIG. 5 is a perspective view of the basic configuration of a detector according to the present invention; FIG. The figure shows equidistant lines and equispaced lines obtained by the detector shown in Fig. 4, Fig. 7 is a perspective view of a device for obtaining equidistant lines and equispaced lines, and Figs. 8 and 9 show equidistant lines and equispaced lines. Figure 10 shows an example of an equidistant line diagram and an equispaced line diagram obtained using the apparatus shown in Figure 7, respectively.
Figures 11 and 11 are graphs showing the changes in the values in Figures 8 and 9, respectively. Figure 12 is a diagram for explaining the method of extracting the optical cutting line. Figures 13 and 14 are graphs showing the changes in the values in Figures 8 and 9. Perspective view of target example, No. 15
16 and 17 are diagrams for explaining the process of detecting the detection target shown in Figure 13, Figure 18 is a diagram showing the detected image of the detection target shown in Figure 14, and Figure 19 1 is a perspective view of a detector according to the present invention attached to a hand of a robot, including a block diagram showing the configuration of an image information processing device. 1, 1a, 1b...slit light source, 2, 2a,
2b...Imaging device, 3, 3a, 3b...Slit light, 4...Light cutting line, 5...Intersection line of slit light,
6...Flat plate, 7...Rail with scale, 8a, 8b,
8c... Optical axis, 9... Cylindrical object, 10... Circular hole, 11... Robot arm, 12... Rotation axis of robot hand, 13... Slit light switching device, 14...
... Optical cutting line extraction circuit, 15 ... Computer, 1
6... Recording device, 17... Detection output.

Claims (1)

[Scope of Claims] 1. A light irradiation means that irradiates at least two flat light beams that are not parallel, and an optical image of the intersection line between the two light rays irradiated by the light irradiation means and the surface of the target object. an imaging means for converting the light beam into an electrical signal, a light beam switching means for switching and irradiating two light beams emitted from the light irradiation means, and separating a bright cutting line signal on the image from the electrical signal obtained from the imaging means. a separating and extracting means for extracting; a storing means for storing the correspondence between positional information on the image captured by the imaging means and actual positional information; and a cutting line signal separated and extracted by the separating and extracting means and the above. A device for detecting the position and orientation of a target object, comprising: calculation means for analyzing the position and orientation of the target object from the correspondence stored in the storage means. 2. The target object according to claim 1, wherein the optical axis of the imaging means is arranged on a plane that bisects an angle formed by two planes formed by the two flat light rays. position and orientation detection device. 3. An apparatus for detecting the position and orientation of a target object according to claim 1, wherein the optical axis of the imaging means is arranged to pass on a line of intersection where the two planar light rays intersect. 4. A light irradiation means that irradiates at least two flat light beams that are not parallel, and an optical image of the intersection line between the two light rays irradiated by the light irradiation means and the surface of the target object, and converts it into an electrical signal. An imaging means is provided on the operating member of the robot, and further includes a light beam switching means for switching and irradiating two light beams irradiated from the light irradiation means, and a bright cutting line signal on the image from an electric signal obtained from the imaging means. Separation and extraction means for separating and extracting; storage means for storing the correspondence between positional information on the image captured by the imaging means and actual positional information; and a cutting line signal separated and extracted by the separation and extraction means. calculation means for analyzing the position and orientation of the target object from the correspondence stored in the storage means; and robot control means for controlling the robot based on the position and orientation of the target object analyzed by the calculation means. An apparatus for detecting the position and orientation of a target object, characterized in that it is equipped with: 5. The target object according to claim 4, wherein the optical axis of the imaging means is arranged on a plane that bisects an angle formed by two planes formed by the two flat light rays. position and orientation detection device. 6. An apparatus for detecting the position and orientation of a target object according to claim 4, characterized in that the optical axis of the imaging means is arranged so as to pass on a line of intersection where the two planar light rays intersect. 7. The light irradiation means is arranged on the operating member of the robot so that the line of intersection of the two plate-like light rays coincides with the rotation center axis or the geometrical axis of the robot's hand. An apparatus for detecting the position and orientation of a target object according to claim 4. 8. At least two non-parallel flat light beams are irradiated by the light irradiation means, and an optical image of the two irradiated light beams and the surface of the target object is captured by the imaging means and converted into an electrical signal, and the above-mentioned The two irradiated light beams are switched and irradiated, and the bright cutting line signal on the image is separated and extracted from the electric signal obtained by the above-mentioned imaging, and the correspondence relationship between the position information on the above-mentioned image to be taken and the actual position information is determined. is stored in a storage means, and the position and orientation of the target object is analyzed and detected by a calculation means from the separated and extracted cutting line signal and the stored correspondence relationship. Method.