JPH0663733B2 - Method for detecting multiple holes - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for detecting multiple holes that are concentrically and stepwise reduced in diameter in a member such as a mechanical component in a stepwise manner in the axial direction.

Conventional technology For mechanical parts, there is a structure in which the nut is fixed concentrically facing the hole, and there is also a structure in which a plurality of holes with decreasing radii are formed in the axial direction. It is necessary to screw a bolt into a hole having the smallest diameter among the multiple holes having a plurality of holes and to insert a pin. In the prior art, a member having such multiple holes is imaged by a two-dimensional sensor such as a television camera, and the hole on the center side having the smallest diameter is detected.

Problems to be Solved by the Invention In such a prior art, although the contour of the outermost hole is relatively easy to detect and the center position of the outermost hole can be detected relatively easily, it has a smaller diameter. Regarding the detection of the inner hole, when light is irradiated from an angle inclined with respect to the axis of the multiple hole, a shadow of the stepped peripheral edge of the outer hole is generated, and a clear outline of the inner hole can be captured. Generally difficult. Further, the multiple holes should be arranged concentrically, but the actual work has a misalignment, and even if the center of the outermost hole is detected, it does not coincide with the center of the inner hole. Therefore, it is difficult in the prior art to accurately determine the center of the inner hole.

It is an object of the present invention to provide a method for detecting multiple holes, which is capable of accurately detecting each of the multiple holes.

Means for Solving the Problems The present invention is a method for detecting a multiple hole formed concentrically and stepwise in the axial direction so that the diameter is reduced to face one surface of a member. Then, the one surface of the member having the multiple holes is irradiated with a pair of slit light beams intersecting each other across the multiple holes, and the one surface of the member is imaged by a single TV camera. The axis of is almost parallel to the axis of the multiple hole, and the three-dimensional coordinates of the end points of the slit holes on the periphery of each hole are detected based on the imaged image, and the coordinate value of the multiple hole in the axial direction is The same or very similar end points are discriminated as the end points on the same plane, and the three-dimensional coordinates of the center of each hole are determined based on the three-dimensional coordinate values of the end points on the same plane. Many characterized by seeking A method for detecting holes.

Operation According to the present invention, slit light is radiated so as to cross each stepped hole of the multiple holes, and the slit lights intersect with each other, for example, a cross shape or a T shape, and With one television camera, one surface of the member facing the multiple holes is imaged, and the three-dimensional coordinates of the end points of the slit light at the periphery of each hole are detected based on the imaged result. Among the three-dimensional coordinate values of each end point, the coordinate values in the axial direction of the multiple holes, that is, the depth direction are compared, and the end points where the coordinate values in the axial direction of the multiple holes are the same or very close to each other are the same. It is possible to discriminate as the endpoints on the plane, and thus the endpoints on each plane are grouped and separated, and thus it is possible to determine which hole is the endpoint of each hole. Based on the three-dimensional coordinate values of the end points on the same plane, the three-dimensional coordinate of the center of each hole is obtained.

The axis of the television camera is substantially parallel to the axis of the multiple hole, and therefore, among the three-dimensional coordinate values of the end points, the coordinate values of the multiple hole in the axial direction can be easily compared.

The slit light may have a cross shape as described above, or may have a T shape, or may have other shapes.

Embodiment FIG. 1 is a sectional view showing the structure of an embodiment of the present invention.
A member gripped by a working end 101 of an industrial robot 100.
A nut 106 is concentrically provided with a hole 104 formed in a flat plate 103, and a nut 106 having a screw hole 105 having a smaller diameter than the hole 104 is fixed by welding or the like. FIG. 2 is a front view showing the hole 104 and the screw hole 105.

Two-dimensional sensor 107 realized by a TV camera or the like
Captures the left surface of the member 102 in FIG.
The hole 104 and the screw hole 105 constitute a multiple hole 108, and the holes 104 and 105 are concentric with each other, and the diameters thereof are reduced stepwise in the axial direction thereof from the left side to the right side in FIG. Are formed in a stepped manner. That is, the screw hole 105 has a smaller diameter than the hole 104. The optical axis of the sensor 107 is substantially parallel to the axis of the multiple hole 108 in this embodiment, for example. The output from the sensor 107 is given to a processing circuit 109 realized by a microcomputer or the like. The slit light source 110 is arranged on the one surface side of the member 102, that is, on the side where the sensor 107 is arranged. The slit light source 110 emits cross-shaped slit light beams that intersect each other at 90 degrees, and the state in which the slit light beams are emitted is shown in FIG. The surface of the flat plate 103 is vertically (3rd
The slit light 111 extending in the vertical direction in the drawing is emitted, and the slit light 112 extending in the horizontal direction (the horizontal direction in FIG. 3) is also emitted. Further, in the nut 106 having the screw hole 105,
The slit light 111 is emitted as indicated by reference numerals 111a and 111b, and similarly, the slit light 112 is displayed on the nut 106 by the reference numerals.
Irradiation is performed as indicated by 112a and 112b.

The operation of the processing circuit 109 will be described with reference to FIG. From step n1 to step n2, the slit light source 110 irradiates one surface of the member 102 with the cross-shaped slit light 111, 112 so that the slit light 111, 112 crosses the multiple hole 108. When the slit light 111, 112 does not reach the holes 104, 105 and does not cross these holes 104, 105, the processing circuit 109 operates the robot 100 in step n3 to detect the degree of engagement of the slit light 111, 112 with respect to the holes 104, 105 by the sensor 107, The direction of movement of the member 102 by the working end 101 of the robot 100 is determined, and the slit lights 111 and 112 are used as holes.
A movement command signal is given until the object crosses 104 and 105, and thus the member 102 is moved by the robot 100.
After the slit light beams 111 and 112 have crossed the holes 104 and 105 in this way, at step n4, four end points A of each hole 104 are formed.
˜D and the three-dimensional coordinates of the end points E to H of the hole 105 are detected. In the coordinate system of this sensor 107, the optical axis direction of the sensor 107 is the Z-axis direction, the direction perpendicular to the Z-axis direction in the first paper plane is the Y-axis direction, and the direction perpendicular to the paper plane of FIG. Axial direction.

Therefore, in step n5, the coordinate values of the respective end points A to D and E to H in the optical axis direction of the sensor 107, that is, the axial line direction of the multiple holes 108, that is, the Z axis direction are compared with each other. The coordinate values in the axial direction of the end points A to D in the hole 104 of the flat plate 103 are the same or very similar. The coordinate values in the axial direction of the end points E to H in the hole 105 of the nut 106 are the same or very close to each other. The coordinate values of the end points A to D and the end points E to H in the axial direction are deviated by the thickness t1 of the flat plate 103. Therefore, the end points A to D of the hole 104 and the end point E of the hole 105 are determined by determining whether the coordinate values of the end points A to D and the end points E to H are the same or similar. ~ H can be distinguished.

At step n6, the center position of each of the holes 104 and 105 can be calculated and calculated based on the coordinate values of the end points A to D and the end points E to H thus obtained. In this way, the center positions of the holes 104 and 105 can be accurately detected.

FIG. 5 is a perspective view showing a structure for three-dimensionally capturing the multiple holes 108 by the three-dimensional sensor 107. Although the following description will be made with reference to the hole 104 in particular, the same applies to the other hole 105. A perfect circular hole 104 is formed facing the surface of the member 102, which is a plane. A plurality (two in this embodiment) of slit light is applied to this hole 104. It should be noted that the camera and two projectors that emit slit light
3 and 4 are integrated. The slit light is a plane indicated by reference numerals 5 and 6, respectively. The light section lines on the surface of the member 102 are missing in the holes 104 and their endpoints are indicated by the reference signs A, B, C, D respectively. The surface of the member 102 is imaged by the industrial television camera 7. The camera 7 includes an image pickup surface 8 of a charge storage element (abbreviated as CCD) and a lens 9 that forms an image of the surface of the member 102 on the image pickup surface 8. The three-dimensional coordinate system of the member 102 is shown by X, Y, Z, and the camera coordinate system (CCD of the camera 7
The coordinate system set on the image pickup plane of is indicated by Xc and Yc. The output from the camera 7 is given to the processing circuit 10.

FIG. 6 is a block diagram showing the electrical configuration of the sensor 107 shown in FIG. The projectors 3 and 4 are driven by drive circuits 11 and 12. The television camera control 13 provided in the processing circuit 10 provides a sync signal to the charge storage element of the camera 7 via the line 14, whereby the video signal obtained from the charge storage element is processed via the line 15. Circuit 10
Is supplied to the analog / digital conversion circuit 16 and is converted into a digital value. The output from the analog / digital conversion circuit 16 thus obtained is compared with the threshold value which is the discrimination level from the threshold value setting device 17 in the comparator 18, and a binary signal is obtained from the line 19. . The binarized signal is stored in the frame memory 20. The content of the memory 20 is given to the processing means 22 via the bus 21, and the data can be transferred to the external processing circuit 109 via the communication controller 23. In one embodiment of the present invention having such a basic configuration, first, the central position of the hole 104 is measured (Chapter I to II described later), and then the inclination of the surface of the member 102 which is a plane, that is, The posture angle is measured (Chapter III, which will be described later), and the distance between one surface 102 of the member and the camera 7 is also measured (Chapter IV, which will be described later).

First, the principle of measuring the center position of the circle of the hole 104 will be described. In the processing circuit 10, the process shifts from step u1 to step u2 in FIG. 7, two intersecting slit lights 5 and 6 are projected onto the plane including the hole 104, and four slit lights are missing from the line of the hole 104. End points A, B,
Measure the 3D position of C and D.

I, measuring method of three-dimensional position of points A, B, C, D by slit light 5,6.

As shown in FIG. 8, the slit light projector 3 and the camera 7 are arranged, and one point P on the slit light plane 5 (this P
Is (X, Y, Z) in the object coordinate system (A, B, C, D are representatively described above), and the coordinate of the image of the point P on the imaging plane 8 is Q in the camera coordinate system. (Xc, Yc). The perspective transformation of the camera 7 is shown in the first equation.

The equation of the slit light plane 5 is shown in the second equation.

a * X + b * Y + Z = d (2) Therefore, the coordinates (X, Y, Z) in the object coordinate system of P can be obtained by solving the first equation and the second equation simultaneously. Basically, the three-dimensional coordinates of all points on the slit light planes 5 and 6 can be obtained.

The coefficients (C _{1 1 to} C _{3 4} , h, a, b, d) are obtained in advance before solving the simultaneous equations including the first and second equations. The method is shown below.

(1) Regarding the camera parameter calibration.

C _{1 1 to} C _{3 4} in the first expression are called camera parameters.
The camera parameters are the focal length of the lens 9 and the lens 9
Is a value that is determined depending on the position of the principal point, the distance between the lens 9 and the light receiving surface, that is, the imaging surface 8. Since it is difficult to measure these values, the following method is used.

By expanding the first equation and eliminating the coefficient h, C _{1 1} * X + C _{1 2} * Y + C _{1 3} * Z + C _{1 4} −C <sub>3 1
* Xc * X -C 3 2 *</sub> Xc * Y-C 3 3 * Xc * Z-C 3 4 * Xc = 0 ...
_{(3-1) C 2 1 * X} + C 2 2 * Y + C 2 3 * Z + C 2 4 -C 3 1
_{* Xc * X-C 3 2} * Xc * Y-C 3 3 * Xc * Z-C 3 4 * Xc = 0 ... (3-
2) Therefore, 6 known points that are not on the same plane
The dimensional coordinates and the camera coordinates corresponding to each are calculated in 3-1.
Twelve unknowns (C _{1 1 to} C _{3 4} ) are obtained by substituting the equations and the equation 3-2 and solving the 12-element simultaneous equations.
Here, in order to improve the accuracy of the calculation of the camera parameters, the measurement of n points (n> 6) whose three-dimensional coordinates are known is performed,
It is obtained by the least squares method.

From the third equation, the following 12-element 2n simultaneous equations relating to the coefficients C _{1 1 to} C _{3 4} are obtained.

However, G = (Et * E) by _{C 3 4 = 1 ... (7-2} ) least squares ^{- 1} * Et * F ... Calculating the (8), G is obtained.

(2) Calculation of the coefficient of the equation of the plane of slit light.

By substituting the known three-dimensional positions of three points on the plane of the slit light into the second equation, a simultaneous equation of three elements related to a, b, d can be obtained, and by solving this, a, b, d can be calculated. . Here, in order to improve accuracy, the known three-dimensional coordinates of n points (n> 3) are substituted into the second equation, and the following three-dimensional n simultaneous simultaneous equations are solved by the least squares method.

If it puts to as J * K = L ... (10 ), K = (Jt * J) - 1 * Jt * L ... determined from (11).

(3) Calculation of three-dimensional coordinates of feature points.

If C _{1 1 to} C _{3 4} , a, b, d are obtained by the method described above, the three-dimensional coordinates of the feature points can be obtained by solving the following equations by combining the second equation and the third equation.

M * N = R (12) _{However, C 3 4 = 1 ... (} 15-2) than the 12 ^{equation, N = M - 1 * R} ... (16) II, points A, B, C, measurement method of the center of the circle passing through D.

The center of the circle passing through the points A, B, C, D is (1a) on the plane passing through the four points A, B, C, D.

(2a) Distances from points A, B, C, D are equal.

It is obtained from the two conditions 1a and 2a.

(1) A plane P1 including the four points A, B, C, D which is the surface of the member 102
3, that is, the calculation of the coefficient of the equation on the paper of FIG.
Figure u3).

The normal vector component of the plane including the four points A, B, C, D has a large Z component and a small X, Y component and distance, so the equation of the plane is expressed by the following equation.

a ₁ * x + b ₁ * y + z = d ₁ (17) Since 4 points A, B, C, D are points on this plane, As a result, a, b, d are calculated by the least squares method, or in the case of three points A, B, C, the components of one row are ignored and a ₁ , b ₁ , d ₁ are calculated by the inverse matrix. calculate.

(2) Calculation of the coefficient of the equation of the plane in which the distances from the two points among the points A, B, C and D are equal.

The points (x, y, z) that have the same distance from each point can be expressed by the following equation.

_{^{(X-x 1) 2 +}} (y-y 1) 2 + (z-z 1) 2 + r
² (19) (i = 1 to 4) In order to calculate with high accuracy, calculation is performed using two points having a large distance from each other. Here, a pair of points A and B and points C and D is used.

With reference to the plan view of FIG. 9, points equidistant from points A and B are given by the following equation (step u4 in FIG. 7).

-2 * x ₁ * x + x ₁ ² -2 * y ₁ * y + y ₁ ² -2 * z ₁ * z + z ₁ ² = -2 * x ₂ * x + x ₂ ² -2y ₂ * y + y ₂ ² -2 * z ₂ * z + z ₂ ² ...
_{(20-1) 2 (x 1 -x} 2) * x + 2 (y 1 -y 2) * y + 2 (z 1 -z 2) * z = (x 1 2 -x 2 2) + (y 1 2 -y ₂ ² ) + (z ₁ ² −z ₂ ² ) ... (20-2) This Equation 20-2 is expressed as a ₂ * x + b ₂ * y + c ₂ * z = d ₂ (20-3). The 20th formula is a formula on the plane P11.

The points C and D are similarly calculated (step u5 in FIG. 7).

2 (x ₃ −x ₄ ) * x + 2 (y ₃ −y ₄ ) * y + 2 (z ₃ −z ₄ ) * z = (x ₃ ² −x ₄ ² ) + (y ₃ ² −y ₄ ² ) + ( z ₃ ² −z ₄ ² ) (21-1) This is defined as a ₃ * x + b ₃ * y + c ₃ * z = d ₃ (21-2). Equation 21-2 is an equation for plane P12.

(3) Calculation of the center O of the circle.

The intersection of the three planes of Equation 17, Equation 20-3, and Equation 21-2 is the center of the circle. Therefore, the coordinates of the center of the circle (xc1, yc1, zc
1) can be obtained by solving the following simultaneous equations (step u6 in Fig. 7).

Than this III, measurement of the posture angle α around the X axis and the posture angle β around the Y axis of the plane P13.

In FIG. 10 (1), the plane P13a is + Δα around the X axis.
When only the angle displacement is made and the posture of the plane P13b is reached, the light cutting line 26 on the plane P13a of the slit light 5 becomes as the light cutting line 27 on the plane P13b. On the imaging surface 8 of the camera 7,
The image of the light section line 26 at α = 0 is indicated by reference numeral 26a, and the image of the light section line 27 after its rotation is indicated by reference numeral 27a. Also the first
As shown in FIG. 1 (1), when the plane P13c is angularly displaced about the Y axis by an angle Δβ and becomes a plane P13d, the light cutting line 28 on the plane P13c becomes the light cutting line 29 on the plane P13d. Become. Therefore, on the imaging surface 8 of the camera 7, the light cutting line
The image 28a of 28 becomes the image 29a of the light section line 29. In this way, the images 27a and 29a on the imaging surface 8 can be calculated by calculating the mutual angle Δα between the planes P13a and P13b and the angle + Δβ between the planes P13c and P13d. The definitions of Δα and Δβ are shown in FIGS. 10 and 11, and the method for obtaining the inclination of the plane will be specifically described with reference to FIGS. 12 to 14.

(1) In determining the posture angle α of the target surface P13 about the X axis and the posture angle β of the target surface P13 about the Y axis shown in FIG. 13, first, the horizontal slit on the imaging surface 8 of the camera 7 is to be calculated. obtained in advance an equation of the optical cutting line 30 of the light, and the equation of the optical cutting line 30, from the previously obtained above camera parameter C _{1 1} ~C _{3 4} Prefecture, optical cutting line 30 of the lens 9 Find the equation of the plane P14 passing through the principal points (steps m1 and m2 in Fig. 12). Further, the equation of the plane P15 of the slit light is obtained in advance (step m3 in FIG. 12).

(2) The equation shown in the previous paragraph (1),
And the camera parameters, the normal vector of each plane of the planes P14 and P15 is obtained, and the normal vector is ₂ .
₃ and the direction vector of intersection 31 of planes P14 and P15 Then, And ₂ and ₃ are orthogonal, This allows Is required (step m4 in Fig. 12).

(3) Similarly, from FIG. 14, the equation of the plane P16 is obtained from the light section line 32 and the camera parameters, and if the equation of the plane P17 is also obtained, the normal vectors of the planes P16 and P17 are respectively obtained.
₆ , ₇ , the direction vector of ray 33 As This allows Is required (step m7 in Fig. 12).

In FIG. 14, a plane P16 is a plane passing through the principal point of the lens 9, and P17 is a plane formed by the slit light of the projector 3.

(4) The normal vector ₀ of the target surface P13 = (s ₀ , t ₀ , 1) (30) is Because it is orthogonal to This gives ₀ (step m in FIG. 12).
8).

The attitude angles α (rotation angle around the X axis) and β (rotation angle around the Y axis) of the image pickup surface 8 of the sensor with respect to the target surface P13 are obtained by the following equations (step m9 in FIG. 12).

_{^{α = tan - 1 (t 0}} ) ... (33) β = tan - 1 (s 0) ... (34) IV, a distance measuring method.

As shown in FIG. 15, when measuring the distance between the plane P13e and the imaging surface 8 of the camera 7, if the plane P13e is displaced within a detectable range as indicated by P13f and P13g,
As shown in FIG. 2B, on the imaging surface 8, the light cutting lines 34, 35, 36 of the slit light 5 of the projector 3 are detected as images 34a, 35a, 36a. In this way, the images 34a, 35 on the imaging surface 8 are
By detecting a and 36a, the distance between the planes P13e, P13f, and P13g can be measured. This method will be described more specifically with reference to FIGS. 16 and 17.

(1) The equation of the optical axis of the camera 7 is x = y = 0 (35), and the distance d between the imaging surface 8 and the target surface P13 is the optical axis of the lens 9 of the camera and the target surface P13. It is defined as the Z coordinate of the intersection point of.

If the coordinates of one point on the target surface P13 are obtained, the equation of the plane of the target surface P13 can be determined from the normal vector of the target surface P13. That point is obtained as the intersection of the planes P14, P15, P17,
Assuming that point to be (x ₀ , y ₀ , z ₀ ), the equation of the target surface P13 is s ₀ (x−x ₀ ) + t ₀ (y−y ₀ ) + 1 · (z−
z ₀ ) = 0 (36) (steps r1 and r2 in FIG. 16).

(2) The distance d is obtained as d = s ₀ X ₀ t ₀ Y ₀ + Z ₀ (37) (steps r3, r4 in FIG. 16).

V, measurement of surface position.

Referring to FIG. 18, in measuring the position of the surface 13, the slit light 6 from the single light projector 4 is projected, and the optical axis 37 of the camera 7 is the XY plane of the object coordinate system. Shall be perpendicular to. At this time, the three-dimensional position of one point 39 on the intersection line 38 between the plane P13 to be measured and the slit light plane 6 is measured, and the Z
The axis component is the surface position (that is, height).

The present invention can not only measure the central position of the hole 104, but also can widely calculate and calculate the area of the hole 104 and other physical quantities, and such modification is easy for those skilled in the art. .

As another embodiment of the present invention, three or more holes 104, 105 may be formed so that the system becomes smaller sequentially in the axial direction.

As described above, according to the present invention, it is possible to accurately detect each hole of the multiple holes, and in particular, the axis of the television camera and the axis of the multiple holes are made substantially parallel so that on the same plane. It is easy to group and separate the endpoints at.

[Brief description of drawings]

FIG. 1 is a sectional view showing the overall construction of an embodiment of the present invention,
FIG. 2 is a front view of the member 102 as seen from the sensor 107 side, and FIG. 3 is a front view showing a state in which the slit light sources 110 and 112 emit slit lights 111 and 112 to the multiple holes 108 of the member 102, and FIG. Flow chart for explaining the operation of the robot, No. 5
FIG. 6 is a simplified perspective view of the three-dimensional sensor 107 according to one embodiment of the present invention, FIG. 6 is a block diagram showing the electrical configuration of the three-dimensional sensor 107 shown in FIG. 5, and FIG. FIG. 8 is a flow chart showing the procedure for calculating the center position, and FIG. 8 is a perspective view showing a method for measuring the three-dimensional position of the point P by the slit light 5.
9 is a plan view of the plane P13, FIG. 10 is a diagram showing the definition of the posture angle α of the plane, FIG. 11 is a simplified diagram showing the definition of the posture angle β of the plane, and FIG. 12 is a posture angle α. , Β is a flow chart showing the procedure for measuring β, and FIGS. 13 and 14 are simplified diagrams showing a method for measuring the posture angle α around the X axis and the posture angle β around the Y axis of the target surface P13. , Fig. 15 shows plane P13e, P
A simplified diagram showing the principle of measuring the distances of 13f and P13g, and FIG. 16 is a flow chart showing the procedure for calculating the distance d in the plane,
FIG. 18 is a diagram showing a simplified structure for measuring the distance d, and FIG. 18 is a simplified diagram showing the principle of measuring the position of the surface 13 according to still another embodiment of the present invention. 100 ... Robot, 102 ... Member, 107 ... Two-dimensional sensor, 108 ... Multi-hole, 109 ... Processing circuit, 111,112 ... Slit light, A to D, E to H ... End points

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hiramatsu Shin 1-1 Kawasaki-cho, Akashi-shi, Hyogo Kawasaki Heavy Industries Ltd. Akashi Plant (72) Inventor Yasuo Nakano 1-1, Kawasaki-cho, Akashi-shi, Hyogo Kawasaki Heavy Industries Inside the Akashi Plant (72) Inventor Sumihiro 1-1 Kawasaki-cho, Akashi City, Hyogo Prefecture Kawasaki Heavy Industries Ltd. Inside the Akashi Plant (56) References JP-A-56-132506 (JP, A) JP-A-56 -35005 (JP, A)

Claims (1)

[Claims] 1. A method for detecting multiple holes formed concentrically and stepwise in the axial direction so that the diameter of the multiple holes decreases toward one surface of the member, the multiple holes being provided. The one surface of the member is irradiated with a pair of slit lights intersecting each other across a multi-hole, and the one surface of the member is imaged by a single TV camera, and the axis of the TV camera is a multi-hole. It is almost parallel to the above-mentioned axis and the three-dimensional coordinates of the end points of the slit holes at the periphery of each hole are detected based on the imaged image, and the coordinate values in the axial direction of the multiple holes are the same or very close to each other. It is characterized in that the respective end points are determined as being end points on the same plane, and the three-dimensional coordinates of the center of each hole are obtained based on the three-dimensional coordinate values of the end points on the same plane. Method for detecting multiple holes.