JPH0241789B2 - - Google Patents

Description

Detailed Description of the Invention (a) Technical Field of the Invention The present invention is based on slit light image information obtained by irradiating slit light onto a surface constituting an object.
The present invention relates to an image processing device that calculates the inclination and coordinates of the surface in three-dimensional space.

(b) Background of the technology For example, when moving a robot or having a robot perform a task, it is necessary to know the three-dimensional position, posture, and shape of objects within the robot's work area. That is, three-dimensional object recognition is necessary, and in particular, obtaining information about the distance from the robot to the object and the inclination of the object's surface is an important issue.

(c) Prior Art and Problems Conventionally, there is a method of segmentation that uses distance information to extract surface information of an object from image information obtained by photographing the object in three-dimensional space.

For example, a three-dimensional space is irradiated with slit light and scanned, an image of the slit light is received by a TV camera, etc., and distance information is obtained from the position information.

However, this method requires measuring the distances of all points within the TV camera's shooting screen.
It has the disadvantage that it takes a very long time to find the distance to the object.

In other words, much more calculation time is required to perform processing from obtaining distance information to segmentation, such as extracting a plane region, inclination of the surface, and integrating the regions, which means that the robot cannot be run in real time. The disadvantage was that it could not be controlled.

(d) Purpose of the invention In view of the above-mentioned conventional drawbacks, the purpose of the present invention is to
An object of the present invention is to provide an image processing device that can calculate the inclination and coordinate information of a surface of an object in a three-dimensional space in a short time.

(e) Structure of the Invention The object is to provide a photographing means for photographing an object in a three-dimensional space, and a surface extraction section for extracting a surface constituting the object based on image information obtained by the photographing means. , an irradiation section that irradiates slit light only onto a desired surface based on the surface information obtained by the surface extraction section; and a slit light image obtained by photographing a slit light image irradiated only onto the desired surface. This is achieved by providing an image processing device that includes a surface information calculation section that calculates the inclination and three-dimensional spatial coordinates of the surface based on optical image information.

(f) Embodiments of the Invention Hereinafter, embodiments of the image processing apparatus according to the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram for explaining an embodiment of an image processing apparatus according to the present invention.

In the figure, 1 is a TV camera, 2 is an A/D converter, 3 is a grayscale image processing section, 4 is a straight line extraction section, 5 is an area extraction section, 6 is a surface inclination/distance measurement section, and 8 is a scanning mechanism control section. It is.

Further, SW, SW, and SW are respective changeover switches, and for example, the central processing unit CPU21
It is switched by a control signal from.

The operation of the image processing apparatus shown in FIG. 1 will be explained.

The operation is divided into the following three processes.

(1) Extract the surface that constitutes the object to be measured from the image information obtained by photographing a three-dimensional space with the TV camera 1. (2) Irradiate the cross slit light within the obtained surface. (3) In order to calculate the inclination and coordinates of the surface, the inclination of the slit light irradiated onto the surface and the cross point of the cross slit light are detected.

First, a description will be given of the process (1) of extracting a surface constituting an object to be measured from image information obtained by photographing a three-dimensional space with the TV camera 1.

First, with the changeover switch SW set to the "1" side, a three-dimensional space is photographed with the TV camera 1,
The obtained analog image signal is converted into a digital image signal by the A/D converter 2 and then input to the grayscale image processing section 3.

The operation of the grayscale image processing section 3 will be explained below.

FIG. 2 is a diagram for explaining the grayscale image processing section 3. As shown in FIG.

Its basic configuration is a first spatial filter calculation section 1
2. Second spatial filter calculation unit 13, pole extraction unit 1
4, a threshold processing unit 15 and a noise removal unit 16, the first spatial filter calculation unit 12 and the second spatial filter calculation unit 13 are connected in series or in parallel by a changeover switch unit 17, and in the case of a parallel connection state, the absolute A value calculation section 18 is inserted.

The first spatial filter calculating section 12 and the second spatial filter calculating section 13 have the same configuration, and the details thereof will be representatively explained in the case where the first spatial filter calculating section 12 has a filter size of 3×3.

The spatial filter calculation section performs the calculation given by the following equation (1).

Ym, n= ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 Xm+1, n+1 ・fi, j/H...(1) Here, Xm, n, Ym, n are each screen The values before and after filtering of the gray level value of the pixel in the upper m rows and n columns are shown, fi and j are weighting coefficients, and H is a normalization coefficient. This filter is
The value of Ym,n is calculated from the pixel of interest Xm,n and its surrounding 8 pixels Xm+1, n+1 (i=1, 0, -1, j=1,
0, -1 except i=j=0) and the weight (fi,
j) It is determined by an addition operation, and the fourth
As shown in the figure, the characteristics of the filter can be changed in various ways by selecting the values of fi and j.

Therefore, when the loading coefficients fi and j are determined as shown in Figure 3B, and the normalization coefficient H is determined as H=16,
Ym and n are determined by the following equation (2).

Ym, n=1/16 (X _n-1,o-1 ×1+X _n,n-1 ×2+X _n+1,o
-1 ×1 +X _n-1,o ×2+X _n,o ×4+X _n+1,o ×2+X _n-1,o+1 ×1
+X _n,o+1 ×2 +X _n+1,o+1 ×1)...(2) In FIG.
A case will be described in which the second spatial filter calculation section 13 and the second spatial filter calculation section 13 are connected in series by the changeover switch section 17. At this time, the first spatial filter calculation unit 12 in the first stage is used for a smoothing filter, the second spatial filter calculation unit 13 in the second stage is used for an image enhancement filter (high pass, Laplacian), and the scanning signal of the TV camera is By synchronizing with , it is possible to reduce image noise and extract the outline of the object in real time.

Now, the hardware configuration for realizing the calculation of equation (1) above will be explained. As shown in FIG. 5, this spatial filter calculation section is composed of four parts: a data shift circuit 30, a multiplication circuit 31, an addition circuit 32, and a division circuit 33.

The data shift circuit 30 stores and stores data of 2 rows + 3 pixels using a shift register, as shown in FIG. 6, in order to perform 3×3 spatial filtering processing on image data for one screen. , Xm+1, n+j (i=1,
It has a function of transmitting nine pixels (0, -1, j=1, 0, -1).

In FIG. 6, the case of screen data with 1 row of p pixels and 2 ^q gray levels will be described. The data displayed by the A/D converter 2 in FIG. 1 with q bits per pixel is held using shift registers for 3 pixels in the first stage and p pixels in the second and third stages.

The last three pixels of each stage are connected to latches or flip-flops R1 to R1 so that the data of each pixel can be simultaneously sent to the multiplier circuit 31 of the next stage.
9 in cascade or serial-in-parallel
9 by using the out shift register.
Pixel transmission can be realized. The second and third stages are shift registers 34 and 3 for (p-3) pixels.
5 is used to hold data of pixels other than those sent. The data shift circuit 30 shifts one pixel at a time according to the scanning signal of the TV camera 1.

As shown in FIG. 7, the multiplier circuit 31 applies weighting coefficients f _i,j (i=-1, 0,
1, j=-1, 0, 1) to obtain the following equation (3).

W _n+i,o+j =X _n+i,o+j ×f _i,j …(3) (i=-1, 0, 1, j=-1, 0, 1) Therefore, the seventh As shown in the figure, the first multiplier 3
Using nine multipliers, 6th to 9th multipliers 44, the first
A weighting coefficient f _-1,1 is set in the multiplier 36, and the pixel X _n-1,o-1 sent out from the data shift circuit 30
Multiplication is performed. A weighting coefficient f _-1,0 is set in the second multiplier 37, and multiplication with the pixel X _n-1,o sent from the data shift circuit 30 is performed. In this way, the multiplication circuit 3 shown in FIG.
6 to 44 can perform the multiplication in equation (3) above.

The adder circuit 32 calculates the nine values W _n+i,o+j (i=-1, 0, 1, j=-
1, 0, 1), it has an addition function that calculates the following equation (4), which is the sum of these.

Z _n,o = ₁ 〓 ^i=-1 ₁ 〓 ^i=-1 W _n+i,o+j …(4) (i=-1, 0, 1, j=-1, 0, 1) Therefore As shown in FIG. 8, the first adder 45
~8th adder 52 is used.

Furthermore, the division circuit 33 converts the gray level Z n,o increased by the multiplication operation and addition operation in the multiplication circuit 31 and the addition circuit 32 to the gray level Z _n,o of the image memory 19 and the monitor TV (not shown) in FIG. It divides by the normalization coefficient H so that it falls within the range of Y=Z _n,o /H (=Equation (1))...(5). Here, the normalization coefficient H is selected to be a value of 2 (k=0, 1, 2, . . . ) close to and larger than ₁ 〓 ^i=-1 ₁ 〓 ^j=-1 f _i,j for ease of implementation. As shown in FIG. 9, this division circuit 33 is composed of a first data selector 53 to an eighth data selector 60.
The final calculation result of the equation Y _n,o is obtained.
Figure 9 shows 8-bit data obtained by multiplying and adding 8-bit gray pixels to 16-bit gray level data _Zn,o.
In this example _, 2 to 9 bits of the 16-bit data Z _n,o are input to the first data selector 53, and the 16-bit data Z n, _{o is input to the second data selector 54.} Input 3 to 10 bits of _o , and similarly input 4 to 11 bits to the third data selector 55.
5 bits are input to the fourth data selector 56.
~12 bits, 6~ to the fifth data selector 57
13 bits, and 7 to 14 bits to the sixth data selector 58.
8 to 15 bits are input to the seventh data selector 59, and 9 to 16 bits are input to the eighth data selector 60. and selector line
In response to the 3-bit selection signal transmitted from the SL, each data selector 53-60 selects and outputs any one of the 1st to 8th bits among the 8-bit inputs. For example, when quoting Z _n,o by 2 ⁷ , each data selector 53 to 60 is controlled to output the first bit from each data selector 53 to 60. The first bit of 2 to 9 bits, that is, the 2nd bit of 16 bits, is output, and the second data selector 54 outputs 3 to 10 bits.
The third bit out of the bits is output, and the third data selector 55 outputs the first bit out of the 4th to 11th bits.
The 4th bit of the 16 bits is output, and the 9th bit of the 9th to 16th bits is output in the same way from the 8th data selector 60. As a result, each data selector From 53 to 60, 2 to 9 bits, which is the value obtained by multiplying 16 bits Z _n,o by ²⁷ , are output.

Similarly, when Zn _,o is multiplied by ²⁷ , it is sufficient to output the second bit from each data selector 53 to 60, that is, 3 to 10 bits of 16-bit data, and the quotient is ²⁵ . In this case, it is sufficient to output the third bit from each data selector 53 to 60 and output the fourth to 11th bits from the 16-bit data. In this way, the ^215-N to ^{28-N bits of the 16-bit data are input to the Nth data} selector, and the value of k is set in the selector line corresponding to the dividing number ^2K . Division can be realized at high speed.

In this way, by using the data selector, it is possible to speed up division.

By appropriately selecting the weighting coefficients of the first spatial filter calculating section 12 and the second spatial filter calculating section 13 in FIG. 2, it is possible to reduce random noise on the image and emphasize sudden changes in brightness.

This load coefficient can be retrieved by the CPU 21 from the load coefficient table of the load coefficient holding section 22, for example.

For example, FIG. 10b shows the state after smoothing + Laplacian processing is applied to the original image in FIG. 10a.

Figure 10 a-1 shows the case where the light hits the cylindrical object 74 from diagonally upward to the right, Figure 10 a-2 shows the cylindrical object 74 viewed from directly above, and Figure 10 a-3 shows the same a
-2 in. It shows a change in brightness in a one-dimensional direction indicated by X-Y.

In addition, Fig. 10b-1 is a three-dimensional display of the gradation level after smoothing + Laplacian calculation;
2 shows isobright lines when viewed from directly above. Furthermore, b-3 in the same figure is X-Y in b-2 in the same figure.
This shows a cross section in the one-dimensional direction indicated by .

Here, extraction of the target outline from FIG. 5b-1 by threshold processing will be explained.

When three levels A, B, and C are selected as slice levels and binarized, different contour lines are obtained as shown in FIGS. 11a to 11c. Figure 11a
Then the level would be too high and the outline would be broken,
In c of the same figure, the level is too low and becomes thick.

Therefore, it is necessary to repair the discontinuity or thin the contour line again. These processes are not suitable for real-time processing because their algorithms are complex and time-consuming.

In addition, if you can set just the right slice level, you can achieve clear binarization as shown in Figure 11b, but the shading level of the original image depends on the surface condition, color, and shape of the target object, lighting conditions, and lens brightness. It is difficult to find an appropriate slice level because it differs each time due to the aperture, etc. After all, it is difficult to extract an object as a series of line drawings and obtain an image with a good S/N ratio using only threshold processing.

Therefore, contour candidate points are first extracted by extreme point extraction processing.

The principle of this polar point extraction operation will be explained using FIG. 12. In order to simplify the explanation, an example will be described in which the extreme point R is determined in the case where the original image P in FIG. 12a has the one-dimensional density distribution as shown.

The original image P is shifted to the left by one pixel Δ and the left-shifted image is designated as P _L (c in the figure), and the right-shifted image in which the original image P is shifted by one pixel Δ to the right is designated as P _R (FIG. 2e). Then, the original image P and the left-shifted image P _L are compared, and the density of the original image is greater or equal (hereinafter, this is referred to as P≧
_{P _ _ _} ) Extract the part Q _R (f in the same figure).

The extreme point (maximum point) R found at this time is considered to be at least larger than or equal to the neighboring pixels on both sides, so by extracting the portion Q _L and Q _R as shown in Fig. 12h, the extreme point R can be found. However, the case where P=P _L =P _R is excluded.

By the way _, when there are poles in a pattern as _shown in L in FIG.
Since the pole edges 1 ₂ and 1 ₄ cannot be extracted,
By scanning this also in the Y direction, the pole side 1 ₂ ,
1 ₄ can be extracted.

Therefore, if we use an extraction window W' having pixels A to I as shown in FIG. The diagonal pole of 45° is
The 135° diagonal poles can be extracted using A, E, and I, respectively.

Now, one frame shown in FIG. 12j is sequentially scanned by scanning lines h ₀ , h ₁ , h ₂ . horizontally by comparative processing,
Pole points in the vertical, 45°, and 135° directions can be sequentially extracted.

The polar point extraction section 14 shown in FIG. 2, which performs the processing described above, is comprised of a data shift circuit 62 and a data comparison circuit 63, as shown in FIG.

As detailed in FIG. 14, the data shift circuit 62 sequentially holds data of 2 rows + 3 pixels using a shift register, and the data comparison circuit 63 of the next stage
It has a function of transmitting nine pixels A to I relative to each other, and is configured similarly to the data shift circuit shown in FIG. The output of the spatial filter 61 consisting of the first spatial filter calculation unit 12 and the second spatial filter calculation unit 13 is input to registers r1 to r9 and a shift register circuit 62 with a P-3 bit length. The pixels A to I are outputted from the data comparison circuit 63 and inputted to the data comparison circuit 63.

As shown in FIG. 15, this data comparison circuit 63 includes comparisons C-1 to C-8, OR circuits OR-1 to
OR-9, AND circuit AN-1 to AN-4, NAND circuit NAN-1 to NAN-4, peak register
It is made up of PR, etc. and comparator C-
1, pixels D and E of registers r4 and r5, which are spatial filter processing data, are compared, and the comparator C-
2, pixels F and E are similarly compared.

Therefore, if pixel E is the pole in the horizontal direction (X direction), E>D, E>F, and D≠E≠F, so from OR-2, OR-3, and NAN-1, all of them are "1". " is output, and "1" is output from OR-1, so the pixel E as the horizontal peak value is held in the peak register PR, and this is output as the extreme point.

Furthermore, the comparator C-3 compares pixels B and E,
Comparator C-4 compares pixels E and H, so
When pixel E is a vertical pole, AN-2 outputs "1" and OR-1 outputs "1", so that pixel E is held in peak register PR. Similarly, pixels C, E, and G in the 45° diagonal direction are compared by comparisons C-5 and C-6, and if pixel E is the pole, "1" is output from AN-3, and comparator C-7 , C-8 compares pixels A, E, and I in the 135° diagonal direction, and if pixel E is the pole in this 135° diagonal direction, "1" is output from AN-4. Become.

Therefore, when the pixel E is the extreme point in any one of the horizontal, vertical, 45°, and 135° diagonal directions, OR-1 outputs "1". At this time, the switching circuit 200 is switched to OR- by the switching signal from the host device.
Since it is switched to output the signal from 1, pixel E is held in the peak register PR and is output as the pole. In this way, if the pixel of interest E is an extreme point, it can be extracted.

As shown in FIG. 16, the threshold processing section 15 consists of a data shift circuit and a threshold circuit, and the data shift circuit 62 uses the same circuit as that of the pole extraction processing section. The latter has a comparator 66, an AND circuit 67, and a register 68, and the comparator 66 compares a preset slice level with the pixel of interest (for example, E), and when the pixel of interest has a density greater than the slice level, Comparator 66 outputs "1",
The AND circuit 67 is turned on. By controlling the level of the pixel of interest E using this signal, it is possible to extract only those pixels that are at a certain level or higher and are extreme points in each of the horizontal, vertical, 45°, and 135° directions. Therefore, the discontinuity N0 that occurs when contour extraction processing is performed on an image obtained by photographing a case where a circular object and a triangular object overlap as shown in FIG. 17 is eliminated.

In this way, the extreme point extraction section 14 extracts contour candidate points, and the threshold processing section 15 performs threshold processing.

In other words, the contour of the object to be contour extracted is considered to correspond to a portion of the original image where the brightness is rapidly changing. Therefore, a Laplacian filter (second derivative) or a gradient filter (1
In the data after applying (order differentiation, boundary line extraction), the part corresponding to the contour of the object has a higher level than other parts.

Therefore, for the data after filtering,
Pole processing is performed in the horizontal, vertical, 45° diagonal, and 135° diagonal directions to extract contour candidate points whose brightness changes suddenly on the original image. In addition, instead of the four directions mentioned above, there are two directions.
It can also be a direction.

If this is all that is done, a lot of background noise due to external factors will be extracted.

However, since the level of background noise is usually lower than the contour points of interest, it can be removed by thresholding.

That is, the slice level may be such that the sequence of contour candidate points extracted by the extreme point extraction process is not interrupted and that background noise can be removed.

In addition, since this threshold processing can be performed in parallel with the extreme point extraction processing, the extreme point extraction section 14 and the threshold processing section 15
The speed can be increased by arranging them in parallel.

As a result, points at the extreme points in the horizontal, vertical, 45° diagonal, and 135° diagonal directions and at a certain slice level or higher can be extracted as contour candidate points as a binary image.

By the way, in the threshold processing in the threshold processing unit 15, the slice level was set so that the contour candidate point sequence was not interrupted, so that some whisker-like noise N1 and isolated noise N2 of several pixels or less remained as shown in FIG. I get tired.

Therefore, assuming that the contour candidate point sequence extracted as a binary image constitutes a closed loop with at least 8 connections, we can eliminate whisker-like noise and isolated Perform noise removal. Removal of each of these noises is performed by the noise removal section 16.

Before explaining the detailed structure of the noise removal section 16, its operating principle will be explained.

As shown in FIGS. 19-19, if the pixel of interest E at the center of the 3×3 window W is not noise, it is always a closed-loop image. Since the closed-loop image is continuous, the closed-loop image always crosses the window W, and it can be said that at least two of the surrounding pixels within the window W must be pixels above which the closed loop is located.
Considering the above preconditions, it can be said that the two peripheral pixels on the closed loop image must have a non-adjacent relationship.

From these facts, in order for the pixel of interest E to be a pixel on a closed-loop image, when the pixel of interest E is in state 1, at least two neighboring pixels that are not adjacent to each other must be in state 1. There are 16 cases that meet this condition, and each case is represented by a window W in FIG. 19.

In this case, to simplify the explanation,
If the closed-loop image has a corner, it is assumed that the pixel corresponding to the corner is treated as not a pixel on the closed-loop image.

Thereafter, a pattern in which the basic pattern does not occur within the window W shown in FIG. 19 is examined and set as a noise removal condition for removing noise of several pixels or whisker-like noise.

In this case, as the first noise removal condition, the second
In the window W in Figure 0a, ~ in Figure b
As shown in , when the pixel of interest E is in state 1, at least the surrounding pixels along two adjacent sides of window W are in state 0, that is, along the top side 1 ₁ and left side 1 ₂ of window W. Surrounding pixels A, B, C, D, G are in state 0, peripheral pixels A, D, along left side 1 ₂ and bottom side 1 ₃ of window W,
G, H, I are in state 0, peripheral pixels along lower side 1 ₃ and right side 1 ₄ of window W G, H, I, F, C are in state 0, peripheral pixels along right side 1 ₄ and upper side 1 ₁ of window W One example is that I, F, C, B, and A are in any state 0.

The second condition is that when the pixel of interest is in state 1, as shown in FIG.
At least the peripheral pixels along one side of the window W and the peripheral pixels adjacent thereto are in the state 0, and the peripheral pixels located at the center of the opposite side of the window W are in the state 1, for example, the upper side of the window W 1 ₁ For example, peripheral pixels A, B, and C along the line and peripheral pixels D and F adjacent thereto are in state 0, and peripheral pixel H located at the center of the lower side ₁₃ of window W is in state 1.

Note that the pixels marked with an x in FIG. 20 indicate that they may be in either state 1 or state 0.

Therefore, the window W for moving the binary image
If the pattern within meets either of the first and second noise removal conditions, even if the pixel of interest E is in state 1, the pixel of interest E is not detected as a pixel of a closed-loop image but as noise. By setting this to "0", this noise can be effectively removed.

For example, as shown in FIG. 18, when there is noise N2 in which four or fewer pixels are clustered together,
If there is whisker-like noise N1 extending from the closed-loop image that is removed under the first noise removal condition, it will be removed under the second noise removal condition.

An example of a circuit for carrying out the noise removal method as described above is shown in FIG. 21, and will be described below.

As shown in FIG. 21, this noise removal section 16
It is comprised of a data shift circuit 70 to which the binary image from the spatial filter/threshold processing section 69 is input, and a logic operation circuit 71.

First spatial filter calculation unit 12, second spatial filter calculation unit 13, pole extraction unit 14, threshold processing unit 15
As shown in FIG. 22, the output of the spatial filter/threshold processing section 69 consisting of
9', N-3 bit length shift register 72,7
3, pixels A to I are outputted to each register R1' to R9', and inputted to a logic operation circuit 71.

Here, the logic operation circuit 71 is configured as shown in FIG. 23, and changes the level of the target pixel E to the state when the first noise removal condition or the second noise removal condition shown in FIGS. 20b and 20c is satisfied. In other cases, the level of the target pixel E is output as 1, and the logical operation circuit 71 sequentially obtains binary image data from which noise has been removed.

Specifically, the logical operation circuit 71 is configured to satisfy the following logical expressions (a) to (d).

E^ ₁ {(A∪B∪C∪D∪G)∩(A∪D∪G∪H∪I)
∩(G∪H∪I∪F∪C)∩(A∪B∪C∪F∪
I)}∩E = {(A∪B∪C∪D∪G)∩(A∪D∪G∪G∪H
∪I)∩ (G∪H∪I∪C∪F∪I)∩(A∪B∪C∪C∪
F∪I)}∩E = {(X ₁ ∪X ₂ )∩(X ₂ ∪X ₃ )∩(X ₃ ∪X ₄ )∩(X ₄ ∪X ₁
)}∩E…(a) E^ ₂ = [{(A∪B∪C∪D∪F)∪}∩{(A∪D
∪G∪B∪H)∪｝∩ {(D∪F∪G∪H∪I)∪｝∩{(C∪F∪I∪
B∪H)∪)]∩E = [{(X ₁ ∪D∪F)∪｝∩{(X ₂ ∪B∪H)∪
｝∩ {(X ₃ ∪D∪F)∪｝∩{(X ₄ ∪B∪H)∪}
〕∩E…(b) E^=E^ ₁ ∩E^ ₂ …(c) X ₁ = A∪B∪C, X ₂ = A∪D∪G, X ₃ = G∪H∪I, X ₄
=C∪F∪I...(d) In formulas (a) to (d), A to I are pixels A to I.
represents the data of pixels B, D,
Indicates the inverted data of F and H, E^ ₁ and E^ ₂ are index values indicating whether the first noise removal condition and the second noise removal condition are satisfied, respectively, and E^ is the attention level E
This is an index value indicating whether or not is noise.

A specific configuration of the logical operation circuit 71 that satisfies this logical formula is shown in FIG. In Figure 23, OR
-1' to OR-14' are OR circuits, AN-1' to
AN-3' is an AND circuit, NT-1' to NT-
4' is a knot circuit.

For example, if the pixel of interest E and other pixels are
In the case of the conditions shown in, OR-1', OR-2'
Since “0” is output from each, OR−
7' also outputs "0", resulting in AN-1', AN
-3' outputs "0", so the pixel of interest E is output as "0".

Furthermore, in the case of the conditions shown in Figure 20c,
OR-1' and OR-5' each output "0",
Since the NOT circuit NT-1' also outputs "0", the OR circuit OR-11' outputs "0". As a result, AN-2' and AN-3' output "0", so the pixel E of interest is output as "0". Similar to the other circuits described above, such control is performed sequentially in synchronization with the horizontal synchronizing signal output from the scanning signal generating section 20 of FIG. 2.

In this way, the logic operation circuit 71 performs the second
The conditions shown in Figure 0 b and c, that is, when there are no pixels on the top and right sides of the window W, on the right and bottom sides, on the bottom and left sides, or on the left and top sides (noise removal condition 1); If the pixel of interest is not above or below the left side but to the right, or to the left of the bottom side rather than to the left or right (noise removal condition 2), the pixel of interest can be regarded as noise and removed.

In this way, the contour points of the object from which noise has been removed can be extracted, and the coordinates of the extracted contour points on the TV camera can be determined.

The coordinate information (Xi, Yi) of the contour point thus obtained is stored in the image memory 19 together with the density level information.

Note that for the contour point coordinates (Xi, Yi) obtained by the grayscale image processing section 3, only the contour address is outputted to the straight line extraction section 4 by the address generator 7.

Next, set the changeover switch SW to the "1" side. As a result, the straight line extracting section 4 extracts
A histogram is created by mapping onto the Hough plane θ-ρ with θ as a parameter.

This process will be explained.

In Figure 1, 75 is First In First Out
It is a (FIFO) type data buffer, and the contour addresses output from the grayscale image processing section 3 are sequentially stored. 76 is a ρ calculation unit which calculates each ring address (Xi, Yi) stored in the data buffer 75.
is read out and the calculation of the following equation is performed.

ρn=Xicosθn+Yisinθn θn=nπ/512, n=0 to 511 77 is a first-in-first-out (FIFO) type data buffer in which ρn and θn calculated by the ρ calculation unit 76 are stored. 78 is a histogram creation unit which uses ρn and θn stored in the data buffer 76 as addresses to write the histogram memory 7.
9 is read, "1" is added, and the result of the addition is written again to the same address.

FIG. 24 shows a detailed circuit diagram of the ρ calculation section 76.

In the figure, 80 indicates a cosθ table 81 and a sinθ table 82, which will be described later, in sequence by a clock pulse.
an address counter that generates the address θn of 8
1 is a cosθ table in which values obtained by dividing nπ into 512 equal parts are stored. 82 is a sinθ table in which values obtained by dividing nπ into 512 equal parts are listed. 83 is one of the contour addresses from the data buffer 75.
A register 84 stores the Xi address, Yi of the contour address from the data buffer 75.
A register in which the address is stored, 85 is a register in which the value from the cosθ table is stored, 86 is sinθ
87 is a multiplier that multiplies the values stored in register 83 and register 84; 88 is a multiplier that multiplies the values stored in register 85 and register 86; 89 is a register in which the value from the table is stored; a register 90 for storing the multiplication result of the multiplier 87;
is a register that stores the multiplication result of the multiplier 88;
1 is an adder that adds the values stored in register 89 and register 90, 92 is a register that stores the addition result of adder 91, and 93 is a delay device 9 that delays address θn output from address counter 80.
3 for synchronizing the contents stored in the register 92 and storing it in the data buffer 76.

With the configuration described above, n (512) coordinates on the .theta.-.rho. plane can be determined for the contour address at the same time as the clock pulse is input. By repeating this process for each contour address, a Hough transform image for the contour point coordinates obtained by the grayscale image processing section 3 is obtained.

Then, the Hough transform image obtained in this manner is stored on the histogram memory 79 by the histogram creation section 78, as described above.

By performing the above processing, the histogram of the Hough transformed image corresponding to each contour point is stored on the histogram memory 79.

A point with a high number of histograms on this histogram memory 79 corresponds to a line segment that makes up the surface of the object, but on a screen with a lot of noise, a point with a high number of histograms corresponds to a line segment that makes up the surface of the object. Points with a high number of histograms appear.

Therefore, it is necessary to extract only the peak points corresponding to each line segment.

This peak point extraction process will be explained below.

First, in FIG. 1, the straight line extracting section 4 is further provided with a memory reading section 93 for reading out the contents of the histogram memory 79, and this output is sent to the selector switch.
Connect to the SW input “2” side.

Therefore, when performing the peak point extraction process, the changeover switch SW is switched to the "2" side.

Then, the Hough transform image stored in the histogram memory 79 is input to the grayscale image processing section 3, and peak candidate points are extracted by performing the above-described processing of the grayscale image processing section 3 on this Hough transform image. . However, in the extreme point extraction process, by switching the switching circuit 200 in FIG. 15 to output the AND circuit AN-5, peaks that are maximum in all four directions are output.

Then, in order to perform peak point integration processing based on the obtained peak candidate points, the changeover switch is switched to the "2" side, and the peak point candidate points are input to the peak point integration calculation section 94 provided in the straight line extraction section 4.

The peak point integration calculation unit 94 integrates adjacent peak candidate points among the input peak candidate points and extracts only the peak points corresponding to the contour line.

The process of extracting this peak point will be explained.

FIG. 25 is a diagram for explaining the peak point extraction process. In the figure, the threshold calculation unit 95
An image memory 96 corresponds to the threshold processing section 15 in the figure.
corresponds to the image memory 19 in the figure. Furthermore, the second
5, the noise removal section 16 shown in FIG.
are not shown.

The threshold calculation unit 95 extracts peak candidate points Xi (i=1 to N) that are equal to or higher than a predetermined threshold level. The extracted peak candidate point Xi is stored in _the image memory 96 along with its coordinate information X ^x _i , X ^y _i and density level X ^Li .

The peak point integration calculation unit 94 includes a clustering calculation unit 97, a maximum value calculation unit 98, and memories 99 and 10.
It consists of 0.

The clustering calculation unit 97 includes a threshold calculation unit 95
Class classification is performed for the N peak candidate points extracted. In other words, in the histogram obtained by Hough transformation, other peaks often appear near the peak of one mountain due to noise etc., as shown in the a-b plane shown in Figure 26, and multiple peaks form a single straight line. In many cases, this corresponds to Therefore, if it is possible to classify peak candidate points Xi as to whether they belong to the same mountain or not, it is possible to extract correct peaks that correspond one-to-one to the straight line. In addition, the number of clusters Nc of the classification result is determined by the clustering calculation unit 9.
7 is unknown at startup.

The clustering calculation section 97 includes, for example, as shown in FIG. 27, an ordering calculation section 101, a class classification calculation section 102, an evaluation criterion calculation section 103, and a threshold value updating section 104. Image memory 19 and memory 99 in FIG. 5 correspond to FIG. 25.

Image memory memory 19, memory 105, and memory 99 may be the same memory.

The ordering calculation unit 101 refers to the image memory 19, compares the density levels X ^L _i of each point X1, X2,..., Store in.

The class classification calculation unit 102 includes the ordering calculation unit 1
Points Y1, Y2,..., ordered by 01
For Yn, the Euclidean distance from other points is calculated in order from the beginning, and by comparing the distance with a threshold value, those having a distance less than the threshold value are labeled and classified into classes.

The evaluation criterion calculation unit 103 is the class classification calculation unit 1
In this method, an evaluation criterion is introduced that minimizes the variance within each class classified by 02 on average, and the evaluation criterion is calculated.

The threshold processing unit 104 updates the threshold used for class classification so that the evaluation standard becomes a smaller value. By changing the threshold value, different classification results can be obtained by the class classification calculation unit 102.

The processing contents of the ordering calculation section 101, the class classification calculation section 102, the evaluation criterion calculation section 103, and the threshold value updating section 104 will be explained with reference to FIGS. 28 to 31.

The ordering calculation unit 101 performs processing as shown in FIG. 28, for example. That is, while changing the loop counter j from 1 to N-1 and the loop counter i from 1 to N-j, the adjacent Xi, Xi+
The density levels X ^L _i of 1 and X ^L _i+1 are compared in size.

If the density level X ^L _i+1 is greater than the density level X ^L _i , the positions of the points Xi and X _i+1 are exchanged. Through this repeated process, the contents of the image memory 19 are arranged in ascending order of density level like the memory 105. Of course, various other sorting means may also be used.

The class classification calculation unit 102 performs processing as shown in FIG. 29, for example. First, the concentration level is the highest.
Let Y1 be the center C1 of the first class. And C
₁ ^and _{other peak} ^points Yj ( ^{j =} 2 ^, 3,... ^N ₎ seek. When this distance Dj1 is smaller than the threshold Tm, it is assumed that Yj belongs to the same first class as Y1.

Next, assuming that classification is possible up to the (k-1)th class, the point with the highest concentration level among the points that do not belong to the classes from the first class to the (k-1)th class
Let Yi be the center Ck of the k-th class. Then, the Euclidean distance Djk between Ck and other points Yj (j=1+1 to N) is determined.

Djk= |Yj−Ck| =√( ^x _j − ^x _k ) ² + ( ^y _j − ^x _k ) ² ) As a result, when the distance Djk is smaller than the threshold Tm,
It is assumed that Yj belongs to the k-th class.

Repeat the process in the same way with k=k+1, and if all Yj belong to one of the classes,
Finish classification. That is, among the unfractioned points, the one with the highest density level is taken as the center of a new class, and classification is performed in order.

The evaluation criterion calculation unit 103 performs processing as shown in FIG. 30, for example. That is, assuming that clustering is good clustering so that the variance within each class is minimized on average, an evaluation criterion Jm expressed by the following equation is introduced.

Jm=〓〓|Yj−mk| ² mk=1/N _k 〓〓Yj Here, Nc is the number of classes, N _k is the number of peak points of the k-th class, and 〓k is the sample set of the k-th class. The calculation of this evaluation standard Jm is executed and the result is transmitted to the threshold updating unit 104.

The threshold value updating unit 104 performs processing as shown in FIG. 31, for example. That is, the threshold value Tm is gradually changed until the evaluation standard Jm becomes less than or equal to a certain predetermined value ε _J or the difference dj from the previous J _n=1 becomes less than or equal to a certain predetermined value εd _J , and the class classification calculation unit 102 Start.

When the evaluation criterion Jm or the difference dj from the previous time becomes a sufficiently small value, it is assumed that good clustering has been performed, and all clustering processing is terminated.

When clustering points in a two-dimensional histogram such as Hough transformation, each point is distributed around the maximum value within the class. Therefore, as mentioned above, by sorting all target points in order of size in advance, the center of the class during clustering can be selected from the beginning with the maximum value. Better clustering can be obtained compared to cases where center points are assumed in order. Therefore, the number of times the threshold is updated until finally satisfactory clustering is obtained can be significantly reduced.

The maximum value calculation unit 98 is the clustering calculation unit 9
The maximum value Zi (i=1 to Nc) of each class is determined by referring to the clustering results of the memory 99 outputted by 7. When the clustering calculation unit 97 is configured as shown in FIG. 27, the peak candidate points Yi on the memory 99 are arranged in ascending order of density level.

Therefore, by searching from the direction of i=1, the maximum value Zi of each class can be easily found.

That is, for example, as shown in FIG. 32, the class number k is changed from 1 to Nc, and the peak candidate point that appears first for each class is extracted.

In this way, the maximum value Zi of each class obtained in the memory 100 becomes a peak point that corresponds one-to-one to a straight line in the case of a Hough transform histogram or the like.

By performing the above processing, only the peak points corresponding to each line segment can be extracted.

A linear equation is calculated based on the peak points (θi, ρi) obtained in this way.

This calculation is performed by the linear equation calculation section 106 in the straight line extraction section 4 shown in FIG.

A flowchart for calculating this linear equation is shown in FIG. 33.

As shown in the figure, the obtained peak point (θi,
ρi, i=1 to M), ai=−cosθi/sinθi bi=ρi/sinθi is calculated based on the linear equation Y=aiX+bi.

In addition, if cosθi=0, Y=ρi/sinθi=ci Also, if sinθi=0, X=ρi/cosθi=di Calculate.

The ai, bi, ci, and di obtained by the linear equation calculating section 106 are output to the region extracting section 5.

The area extraction unit 5 extracts an area based on each of the input values.

This area extraction process will be explained.

In FIG. 2, 107 is a line segment extraction unit, 108
109 is a closed-loop extraction unit, and 109 is a projection point calculation unit.

First, the operation of the line segment extraction section 107 will be explained using the flowchart shown in FIG.

The line segment extraction unit 107 includes a linear equation calculation unit 10.
The slope and intercept information obtained in step 6 and the contour point coordinates stored in the image memory 19 in the grayscale image processing section 3 are input.

Then, as shown in FIG. 35, the input contour point coordinates are developed on the XY plane, and a straight line based on the slope and the intercept is generated in a pseudo manner on the XY plane.

Then, a window consisting of a K×L matrix is scanned along this straight line.

The number of contour points existing within this window is sequentially scanned from, for example, the upper left corner of the window to check whether contour points are stored in the image memory 19 corresponding to each coordinate within the window.

Then, if it is a contour point, the counter
Add CONT by "1", and if it is not a contour point, move to the coordinate position in the next window.

This process is then sequentially repeated until it reaches the lower right end of the window calculation circuit. If the value of the counter CONT after the processing is over the predetermined value T,
The coordinates are stored in memory as the coordinates (xi, yi) of the point of interest on the straight line. A circuit that executes this window processing is a window calculation circuit.

This process is performed until the end position of the straight line on the XY plane, and then the next straight line is generated in a pseudo manner on the XY plane, and the same process is repeated in sequence to store the line segment coordinates in memory. .

Then, based on the coordinates of each line segment, the end point A and end point B of each line segment and the length L of the line segment are extracted, and the third
Create the line segment table shown in Figure 6.

Next, the operation of the closed loop extraction section 108 will be explained using the flowchart shown in FIG. 37.

First, the end points A, B, and length L of each line segment are extracted from the line segment table, rearranged in descending order of length L, and assigned numbers in descending order.

Then, one end point A of the line segment with the maximum length
Search for a line segment that has an end point within radius R using as a reference. If there is no endpoint, the radius R is made slightly larger and a line segment is searched in the same way. If a line segment cannot be found even if the radius R is increased by more than a predetermined value (Rmax), this line segment is excluded (for example, the invalid flag is set to “1”).
). If the end point of another line segment can be found within the radius R, assign a continuous line segment number to this line segment, and perform the same operation using the other end point of the found line segment as a reference. Execute repeatedly.

Then, when returning to the first line segment, this operation is completed and one closed loop is extracted. This closed loop stores line segment numbers for each closed loop, as shown in FIG.

By repeating this, all closed loops in the three-dimensional space can be extracted.

The projection point calculation unit 109 includes a closed loop extraction unit 108
The center of gravity of each closed loop region obtained by is calculated.

As explained above, the projection plane extraction process can be performed.

Next, the process (2) of irradiating cross slit light within the obtained plane will be explained.

Incidentally, the process of projecting the slit light is shown in the flowchart of FIG.

Here, for convenience, it is assumed that the cross point of the cross slit light is irradiated to the center of gravity coordinates in the projection plane.

In FIG. 2, the scanning mechanism control section 7 includes a scanning controller 110 and a motor 111 that drives a cross slit light projector 112 that emits cross slit light having a wavelength in the infrared region.

First, the scanning controller 110 drives and controls the motor 111 so that the cross point of the cross-shaped slit light becomes the barycentric coordinate of the desired closed-loop area calculated by the projection point calculation unit 109 of the area extraction unit 7, and Move the optical projector 112.

At the same time, the S/ of the cross slit optical image is
A visible light cut filter is attached to the front of the TV camera 1 in order to improve the N ratio and facilitate slit image detection. Note that any material may be used as long as it can be optically separated. This is done by the separation mechanism 113 shown in FIG. Further, the separation mechanism 113 operates based on a signal from a higher-level central processing unit (not shown). Furthermore, set the changeover switch to “1”
Then, the changeover switch is set to the "1" side, and the circuit configuration of the surface inclination/distance measuring section 6 is changed to a circuit configuration for determining the rough crossing point of the cross slit light.

In this state, a three-dimensional space is photographed by the TV camera 1 to obtain a slit image. Then, as described above, the gradation image processing section 3 extracts the skeleton line coordinates (corresponding to contour point coordinates) of this slit image.

Based on the skeleton line coordinates obtained by the gray scale image processing section 3, the rough cross point calculation section 114 in the surface inclination/distance measurement section 6 calculates the rough cross point coordinates of the cross slit light.

This process will be explained using FIG. 40.

First, from among the skeleton line coordinates of the extracted slit light, the minimum value Xmin of the X coordinate, the maximum value Xman, and the Y
Find the minimum value Ymin and maximum value Ymax of the coordinates.

And as a rough cross point (Xc, Yc), Xc=(Xmin+Xmax)/2 Yc=(Ymin+Ymax)/2 Calculate.

Then, based on the rough cross points (Xc, Yc) obtained by the rough cross point calculation unit 114,
The scanning controller 110 is the motor 111
The cross slit light projector 11 is driven and controlled so that the rough cross point is near the center of gravity coordinates in the projection plane.
Move 2.

Next, when this rough cross point falls within a predetermined area near the center of gravity coordinates, the changeover switch SW is changed to the "1" side and the changeover switch SW is switched to the "2" side.

Here, detecting that the rough cross point has entered a predetermined area near the center of gravity coordinates is performed using, for example, the coordinates of a small area obtained by adding a predetermined value to the center of gravity coordinates,
This can be easily done by comparing the cross points (Xc, Yc) obtained by the rough cross point calculation unit 114. This process is performed, for example, by the separation mechanism 113, the changeover switch SW, etc.
, SW, and SW are controlled by a host central processing unit (not shown).

Next, by switching the changeover switch SW to the "1" side, the skeleton line coordinates of the slit light image obtained by the grayscale image processing section 3 are input to the straight line extraction section 4.

The straight line extraction unit 4 creates a histogram by mapping the skeleton line coordinates onto the Hough plane by Hough transformation based on the input skeleton line coordinates, in the same way as in the process described above.

Based on the histogram obtained in this manner, the inclination of the slit light is detected, and an accurate cross point of the slit light is calculated.

The slit light inclination detecting section 115 shown in FIG. 2 detects the inclination of the slit light.

The process of detecting the inclination of the slit light will be described below.

This method of detecting the inclination of the slit light is described in the 41st
As shown in the figure, each point P1: (X1, Y1) P2: (X2, Y2) P3: (X3, Y3) P4 on the straight line _Y = a _{0 ·} : (X4, Y4) is subjected to Hough transformation and expanded to a new coordinate system (b in the same figure). In the coordinate system, corresponding to each point, b=-X1・a+Y1 b=-X2・a+Y2 This method takes advantage of the fact that b=-X3.a+Y3 b=-X4.a+Y4 and that the straight lines intersect at the point (a ₀ , b ₀ ) in the coordinate system.

FIG. 42 is a detailed circuit diagram of the slit inclination detection section 115.

This slit inclination detection section 115 searches the frequency data in the histogram memory 79 of the straight line extraction section 4 and obtains point coordinate data [θ(max), ρ(max)] where the frequency data is maximum. However, in this embodiment, since there are two target line images corresponding to the cross slit light, the point coordinate data [θ1 (max), ρ1 (max)] [θ1 (max), ρ1 (max)] is used for each line image. θ2(max), ρ2(max)]. The two frequency peak points obtained from the line image corresponding to the cross slit are generally separated with respect to θ,
By adjusting the relative positional relationship between the cross slit light projector 112 that projects the cross slit light and the TV camera 1 that photographs the slit light, the θ1
(max) and θ2(max) is always expressed as 0<θ1(max)<π/2≦θ2(max)≦π …(6) ｜tanθ1(max)｜＜1≦｜tanθ2(max)｜ …(7) It is possible to set the condition as follows.

Therefore, the slit inclination detection section 1 shown in FIG.
15 is a cross slit optical projector 112 and a TV camera 1 so as to satisfy the condition (6) above.
This is the case when .

In FIG. 42, 116 is an address generator that sequentially outputs point coordinate data ρj and θj in the new polar coordinate system, ρj from this address generator 116 is sent to a register 117a, and θj is sent to a register 117b, respectively. The data are sequentially stored in synchronization with clock pulses from a clock generator (not shown).

Further, the point coordinate data θj, ρj from the address generator 116 is transmitted to the histogram memory 79, and from this histogram memory 79, frequency data f(θj, ρj) corresponding to the point coordinates (θj, ρj) are transmitted.
is read out, and the frequency data is stored in the register 117c in synchronization with the clock pulse. 118 is register 11
This comparator compares θj stored in 7b with a predetermined value θt=(π/2) and outputs an H level signal when θj<θt=(π/2).

Further, 119 is a first maximum value detection circuit, 120
is a second maximum value detection circuit, and each maximum value detection circuit 119, 120 has a register 117c.
Register 1 to store again the frequency data stored in
19b, 120b, and each register 119
b, 120b frequency data and the register 117
A comparator 119 that compares the frequency data from c.
a, 120a, and each comparator 119
a, 120a outputs an H level signal when the frequency data in the register 117c exceeds the frequency data in the registers 119b, 120b, and the register 119b outputs an H level signal when the frequency data in the register 117c exceeds the frequency data in the registers 119b, 120b.
The register 120b operates in synchronization with the AND signal from the AND gate G22 of the output signal of the comparator 118, the output signal from the comparator 120a, and the AND gate G22 of the clock pulse. It operates in synchronization with the AND signal. Similar to register 119b, 121a and 121b are registers that operate in synchronization with the output signal of AND gate G23, and registers 121a and 122a each store ρj in register 117a at the timing, and
21b and 122b are designed to store θj in the register 117b at respective timings.

And, 123a and 123b are registers 121a and 121b and registers 122a and 12, respectively.
This is a converter that converts θj and ρj stored in 2b into a and b according to a=-(cosθj/sinθj) b=ρjsinθj, and converts polar coordinate data (θj, ρj) into orthogonal coordinate data (a, b). It is something that converts.

The operation of the configuration described above will be explained.

That is, the address generator 116 sequentially outputs θj and ρj, and converts the frequency data f(θj, ρj) in the histogram memory 79 specified by (θj, ρj) into
In the region θj<π/2, the data is stored in the register 119b via the register 117c, and then stored in the register 119b.
The frequency data of 9b is updated to larger values in sequence.

Further, in the region of π/2≦θj<π, the frequency data f(θj, ρj) is stored in the register 120b via the register 117c, and further in the register 120b.
The frequency data of 0b is updated to successively larger values in the same manner as described above. At this time, register 119
At the timing of updating b, the corresponding ρj and θj are updated to the register 12 via registers 117a and 117b, respectively.
1a and 121b, and is stored in register 12
At the timing of updating 0b, ρj and θj are stored in registers 122a and 122b via registers 117a and 117b in the same manner as described above. When the above processing is performed for all (θj, ρj) (j=1 to N), the frequency data finally stored in the register 117b becomes the maximum value in the region θj<π/2, The frequency data stored in the register 120b has a maximum value in the region of π/2<θj≦π.

Accordingly, the contents of registers 121a and 121b become ρ(max1)θ(max1), which has a maximum frequency in the region θj<π/2, and the contents of registers 122a and 122b become ρ/2≦θj<π ρ(max2), θ where the frequency is maximum in the region
(max2), and the conversion outputs a(max1) and b(max1) of the converter 123a at that time are each _a0 in the linear equation Y= _a0・X+ _b0 , which indicates one skeleton line corresponding to the cross slit light, At the same time when b becomes ₀ , the conversion outputs a(max2) and b(max2) of the converter 123b at that time
are a ₀ ′ and b ₀ ′ in the linear equation Y=a ₀ ′·X+b ₀ ′, which indicates one skeleton line corresponding to the cross slit beam.

Therefore, the a(max1) and a(max2) are
This is the slope of each line image corresponding to the desired cross slit light.

A simultaneous equation is created based on the two linear equations obtained in this way, and its solution is X, Y
is calculated as the cross point of the cross slit light.

The process of forming this simultaneous equation and finding its solution is carried out by the cross point calculation unit 12 of the surface inclination/distance measurement unit 6.
Do it in 4.

Then, based on the cross point (X, Y) obtained by the cross point calculation unit 124, the scanning controller 110 drives and controls the motor 111, so that the cross point (X, Y) is located at the center of gravity coordinates in the projection plane. Cross slit optical projector 112 so that
move.

In the above description, the cross point of the cross slit light is positioned at the center of gravity coordinates of the closed loop region, but the present invention is not limited to this, and the cross point of the cross slit light is positioned at the coordinates of the center of gravity of the closed loop region. It may be.

However, if the center of gravity of the closed loop region is
When the cross slit light is irradiated, there is less possibility that the cross slit light will protrude from the closed loop area.
Furthermore, when determining the position to be irradiated with the cross point of the cross slit light, the center of gravity coordinates are simply used because they are the easiest to calculate.

By executing the processing described above, it is possible to irradiate the slit light within the region of the closed-loop image.

Next, the process of detecting the inclination of the slit light irradiated onto the surface and the cross point of the cross slit light in order to calculate the inclination and coordinates of the surface in (3) will be explained.

FIG. 43 shows a cross slit optical projector 112.
44 is a perspective view showing the spatial arrangement relationship between the TV camera 1 and the TV camera 1; FIG. 44 shows the coordinate relationship during calibration in the spatial arrangement of FIG. 43; and FIG. The coordinates of the image on the photographing surface of the camera 1 are shown, and FIG. 46 is a diagram showing the coordinate relationship when a three-dimensional object has an arbitrary inclination.

In such a configuration, the optical axis of the coordinate system Op-XpYpZp of the cross-slit optical projector 112 is defined as Zp, and the Xp and Yp axes are determined to be orthogonal to this. On the other hand, the coordinate system Oc− of TV camera 1
Regarding XcYcZc, the horizontal axis of the imaging plane is Xc, and the vertical axis is Yc.
and set Zc to be orthogonal to these (see Figure 44).

And the coordinate system Op−XpYpZp is the coordinate system Oc−
Assume that XcYcZc is rotated by α around the Zc axis, β around the Yc axis, and γ around the Xc axis.

It is assumed that this relationship is known by performing calibration at the reference point in advance.

The relationship is expressed by the following formula.

Xp Yp Zp＝H〓(α, β, γ) , −sinβ cosβ sinγ cosβ cosγ …(9) Next, the cross slit optical projector 112 and the TV camera 1 are placed on the object plane π1 with an arbitrary inclination.
Euler angles α _p , β _p , γ _p , α _c , respectively from the reference plane
Referring to FIG. 46, we will derive e〓 and f〓 when the slit beams of S〓 and T〓 are projected by rotating β _c and γ _c .

Here, in order to facilitate the analysis, as shown in Fig. 44, a slit of s is placed on the Xp axis, and a slit of t is placed on the Yp axis.
Consider the slit.

First, the cross slit optical projector 112 and
The coordinate system after rotation of the TV camera is Op−
Let XpYpZp and Oc−XcYcZc.

Op′−Xp′Yp′Zp′ is for Op−XpYpZp, Xp′ Yp′ Zp′=H〓(α _p , β _p , γ _p ) Yc′Zc′ is Xc′ Yc′ Zc′=H〓(α _c , β _c , γ _c )Xc Yc Zc …(11) for Oc − ′(α _c , β _c , γ _c )Xc′ Yc′ Zc′ …(12) Here, the needle “′” of H〓′ is
Indicates the transposed matrix of H〓. Therefore, Op′−
_Xp′Yp′Zp ′ is for _Oc′ − _Xc′Yc′Zc ′. )・H〓
_{′ ( α c , β c} , _{γ c )} , γ)・H〓′(α _c , β _c , γ _c ) = h11 h12 h13 h21 h22 h23 h31 h32 h33 …(14).

On the other hand, the projected image e〓 on the object plane is on the Xp′Zp′ plane of the Op′−Xp′Yp′Zp′ system, so e〓⊥jp′(e〓・jp′)=0 …(15) However, e〓=Xeic′+Yejc′+Zekc′ jp′=h21ic′+h22jc′+h23Kc′…(16) Substituting equation (16) into equation (15), Xeh21+Yeh22+Zeh23=0…(17) Ze that cannot be measured is ∴ Ze=−Xeh ₂₁ +Yeh ₂₂ /h ₂₃ (18) However, h23≠0. Also, the projected image f〓 on the object plane is within the Yp′Zp′ plane of the Op′−Xp′Yp′Zp′ system, so f〓⊥ip′(f〓・ip′)=0 …(18 ) However, f〓=Xfic′+Yfjc′+Zfkc′ ip′=h11ic′+h12jc′+h13Kc′ …(20) Substituting into the above equation, Xfh11+Yfh12+Zfh13=0 (21) Zf that cannot be measured is ∴Zf=−Xfh ₁₁ +Yfh ₁₂ /h ₁₃ …(22) However, h13≠0.

Finally, when deriving the surface vector of π1, the surface vector π〓 is given by the cross product of e〓 and f〓.

Therefore, π〓＝e〓×f〓 ＝ ic jc Kc Xe Ye Ze uic+vjc+wKc…(25) Therefore, the direction cosine of π1 for the Oc′−Xc′Yc′Zc′ system is It is.

In order to calculate the inclination and coordinates of the surface described above, a position/orientation control unit 125 and a surface inclination/distance measurement unit 126 perform processing to detect the inclination of the slit light irradiated onto the surface and the cross point of the cross slit light. The operation will be explained with reference to the flowcharts shown in FIGS. 47 and 48.

In the figure, during calibration, a cross slit optical projector 11 installed at a reference point is used.
Calibration of each coordinate system of 2 and TV camera 1 is performed.

Here, to simplify the analysis, two orthogonal slit beams are projected onto the reference plane π ₀ .

Then, a(max1), which is the slope of each line image obtained by the slit slope detection unit 125, and a
Since (max2) corresponds to the slope of e〓, f〓, the X and Y components, that is, Xe, Ye, Xf, and Yf can be obtained from this. If the relative positional relationship between the TV camera and the cross-shaped slit light projector is calibrated in advance, there is no need to perform calibration again unless the positional relationship is changed.

Next, the case of deriving the inclination of an arbitrary surface will be explained. Cross slit optical projector 112
The TV camera 1 is controlled and directed to the target point.

This is similar to the operation described above. Further, since the cross point of the cross slit light beam is determined by the cross point calculation unit 124, the rotation angle of the cross slit light projector 112 from the reference point can be read from this cross point. Further, the rotation angle of the TV camera can be easily read if the TV camera is rotationally driven by a control signal from a central processing unit (not shown).

In the surface inclination/distance measurement unit 126, α, β, γ,
Find H ₀ from αp, βp, γp, αc, βc, and γc. Next, the lengths of the X and Y components are calculated based on e〓 and f〓 from the slit inclination detection section 125.

Furthermore, Ze and Zf are derived using equations (18) and (22), and u, v, and w are obtained from equations (24) and (25),
Derive the direction cosine of π1 from equation (26).

Further, the distance can be determined from the obtained cross point coordinates.

Below is a method for finding the distance by calculating the center point of the projected cross-slit image when the relative positional relationship between the TV camera system Oc-XcYcZc and the cross-slit optical projector system Op-XpYpZp is known in advance. This will be explained using FIG. 49.

To simplify the explanation, the three-dimensional space will be explained using the XcZc plane and the YcZc plane of TV camera coordinates.

FIG. 49a shows the XcZc plane, and FIG. 49b shows the YcZc plane.

Here, the measurement point, that is, the cross point of the projected image of the cross slit light is P (X0, Y0, Z0), and the position of the cross slit light projector 112 is Q (Xq, Yq,
Zq), and the focal length of the TV camera 1 is F. Also,
On the XcZc plane, the angle between the optical axis Zp and Xc of the cross slit optical projector is θ, and on the YcZc plane, the angle between the optical axis Zp and Yc of the cross slit optical projector is ψ
shall be.

Further, let the image of P on the imaging plane be R, and let its coordinates be (Xs, Ys, -F).

At this time, the straight lines PQ and PR can be expressed as follows on the XcZc plane.

PQ:Z0=(X0−Xq)tanθ+Zq…(26) PR:Z0=(−F/Xs)X0…(27) From equations (26) and (27), the Zq−Xqtanθ/F ₀ +X _s tan θXs (28) Z0=−F/X _s X0 (29) Moreover, the straight lines PQ and PR can be expressed as follows on the YcZc plane.

PQ:Z0=(Y0−Yq)tanψ+Zq…(30) PR:Z0=−F/YsY0…(31) From equations (30) and (31), the Y and Z coordinates of P are: Y0=−Zq−Yqtanψ /F ₀ +YstanψYs...(32) Z0=-F/YsY0...(33) Then, the cross point coordinates (Xc,
Yc) converted to TV camera system coordinates (Xs, Ys)
Find out θ, ψ, Xq, Yq and the focal length F of the TV camera lens from the positional relationship between the TV camera and the cross-shaped slit light projector. 33), by substituting
You can find the distance to an object.

Regarding Z0, either equation (29) or (33) may be used.

(g) Effects of the Invention As explained above, according to the present invention, the inclination and coordinates of the surface are determined for each closed-loop area obtained by photographing an object in three-dimensional space, which makes it possible to recognize three-dimensional objects. The necessary segmentation can be performed at high speed, making it an effective tool as a robot's eye.

[Brief explanation of drawings]

FIG. 1 is an explanatory diagram of an embodiment of an image processing apparatus according to the present invention, FIG. 2 is an explanatory diagram of a grayscale image processing section, and FIG.
Figure 4 is an explanatory diagram of the load coefficient, Figure 5 is an explanatory diagram of the spatial filter calculation section, Figure 6 is an explanatory diagram of the data shift circuit, Figure 7 is an explanatory diagram of the multiplication circuit, and Figure 8 is an explanatory diagram of the addition circuit. An explanatory diagram of the circuit, Fig. 9 is an explanatory diagram of the division circuit, Fig. 10 is an explanatory diagram of the object and its contour extraction state, Fig. 11 is an explanatory diagram of the output when changing the slice level, and Fig. 12 is the outline extraction An explanatory diagram of the operating principle, FIG. 13 is an explanatory diagram of the pole extraction section, FIG. 14 is an explanatory diagram of the data shift circuit, FIG. 15 is an explanatory diagram of the data comparison circuit, FIG. 16 is an explanatory diagram of the threshold value processing section, Figures 17 and 18 are diagrams explaining the effects of contour extraction processing, Figure 19 is a diagram explaining contour data, Figure 20 is a diagram explaining noise removal conditions, and Figure 21 is an explanation of the noise removal section. Fig. 22 is an explanatory diagram of the data shift circuit, Fig. 23 is an explanatory diagram of the logic operation circuit, Fig. 24 is an explanatory diagram of the ρ calculation section, Fig. 25 is an explanatory diagram of the peak point extraction process, and Fig. 26 is an explanatory diagram of the data shift circuit. is a histogram obtained by Hough transformation, FIG. 27 is a clustering calculation section, FIG. 28 is an illustration of an ordering calculation section, FIG. 29 is an illustration of a class classification calculation section, and FIG. 30 is an explanation of an evaluation criterion calculation section. Figure, No. 31
Fig. 32 is an explanatory diagram of the threshold value updating section, Fig. 32 is an explanatory diagram of the processing of the maximum value calculation section, Fig. 33 is a flowchart of the linear equation calculation process, Fig. 34 is an explanatory diagram of the line segment extraction section, and Fig. 35 is an explanatory diagram of the processing of the maximum value calculation section. Processing explanatory diagram of line segment extraction unit, No. 36
38 is a line segment table, FIG. 37 is an operation flowchart of the closed loop processing section, FIG. 39 is an explanatory diagram of slit light projection processing, FIG. 40 is a processing explanatory diagram of the rough cross point calculation section, Fig. 41 is an explanatory diagram of the method of detecting the inclination of the slit light, Fig. 42 is an explanatory diagram of the slit inclination detection unit, Fig. 43 is a perspective view showing the spatial arrangement relationship between the cross-shaped slit light projector and the TV camera,
Figure 44 is a diagram showing the coordinate relationship during calibration, and Figure 45 is a diagram showing the spatial arrangement of Figure 43.
Diagram showing the coordinates of the image on the shooting surface of the TV camera, No. 4
6 is a diagram showing the coordinate relationship when a three-dimensional object has an arbitrary inclination, FIGS. 47 and 48 are operation flowcharts explaining the process for calculating the inclination and coordinates of a surface, and FIG. FIG. 3 is an explanatory diagram of a method for determining the distance to an object from the cross point coordinates of cross slit light. In the figure, 1 is a TV camera, 2 is an A/D converter, 3 is a grayscale image processing section, 4 is a straight line extraction section, 5 is an area extraction section, 6 is a surface inclination/distance measurement section, and 8 is a scanning mechanism control section. , 112 is a cross-slit optical projector, 113 is a separation mechanism, SW, SW,
SW is a changeover switch.

Claims (1)

[Claims] 1. Photographing means for photographing objects in three-dimensional space;
a surface extraction section that extracts a surface constituting the object based on image information obtained by the photographing means; and a surface extraction section that irradiates only a desired surface with slit light based on the surface information obtained by the surface extraction section. an irradiation unit; and a surface information calculation unit that calculates the inclination and three-dimensional spatial coordinates of the surface based on slit light image information obtained by photographing a slit light image irradiated only onto the desired surface. An image processing device consisting of: