JPH0831136B2 - Image processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial field of application] The present invention processes a surface constituting an object in a three-dimensional space by processing images obtained by capturing the same object to be processed at a plurality of different positions. The present invention relates to an image processing method for specifying and obtaining that information.

[Conventional technology]

As a means for obtaining information such as a positional relationship between objects in a three-dimensional space, so-called stereo is performed, in which image processing is performed based on images obtained by imaging the same object from different positions by a plurality of imaging devices. The law is known.

Specifically, an area having the same brightness is obtained from a grayscale image obtained by each of a plurality of image pickup devices (generally, two image pickup devices arranged in parallel at appropriate intervals) based on the difference in brightness. Extract, detect the edge of each extracted region,
Correspondence between both images is performed for each detected edge. Then, for the edges corresponding to each other, information on the distance can be obtained based on the principle of triangulation.

By the way, with the above-mentioned method, distance information can be obtained for the detected edge, but information for each point on the surface of the object cannot be obtained. Therefore, a method for identifying each surface by obtaining and comparing the information of the surface using the information of the edge for which the distance information has been obtained is disclosed in Japanese Patent Application No. Sho 63-226761.
The present invention has been invented by the present inventors.

[Problems to be Solved by the Invention]

In the invention of Japanese Patent Application No. 63-226761, the interpretation of an image is performed by extracting a closed area surrounded by a loop composed of continuous boundary segments obtained from the detected edges as a plane. In other words, the object in the image is recognized. For example, when a hole is formed on the surface, another loop is formed in the loop formed by the segment that surrounds one area. Exists, which causes various inconveniences.

For example, as shown in FIG. 8, a closed loop formed by the segments S11 to S17 forming the boundary line B1 of the outer peripheral contour with respect to the hatched region R and the boundary line H1 of the inner peripheral contour which is the contour of each of the two holes. , S2 composing H2
-S25, S31-S37 It is necessary to estimate the surface of the part surrounded by the closed loop.

In order to perform such a process, it is determined whether or not there is a hole in the boundary line of the outer peripheral contour of all the regions, that is, whether the boundary line of the inner peripheral contour exists, in other words, the closed loop of the segment that surrounds the region. Everything needs to be examined to see if there are more closed loops of segments within it.

Also, as shown in FIG. 9, the hatched area
If there is a hole in R1 and the surface R2 of another hatched object is visible through that hole, then the region R1
Even if the boundary line B2 of the region R2 is present within the boundary line B1 of the outer peripheral contour of, both boundary lines B1 and B2 do not surround the same region, that is, they do not form the same plane. . Therefore, in the example shown in FIG. 9, whether the regions R1 and R2 are surfaces of different objects, or whether an annular groove exists on the surface of the same object, It becomes necessary to distinguish between the two. Such processing becomes more complicated and requires processing time as the image to be processed becomes more complicated.

The present invention has been made in view of such circumstances, and adopts a data structure capable of tracing a mutual relationship among a plurality of boundary lines existing in the same area at the stage of image processing. Thus, an object of the present invention is to provide an image processing method capable of easily identifying a boundary line of a processing target area and estimating a surface thereof.

[Means for solving the problem]

The image processing method according to the present invention obtains common line segment information from each of a plurality of images obtained by capturing images of a recognition target located in a three-dimensional space with a plurality of imaging devices arranged at different positions on the same plane. , An image processing method for recognizing an object by extracting an area surrounded and bounded by a closed boundary line by connecting them, and estimating the extracted area as a plane. If there is a closed boundary line, give a pointer connecting the outer boundary line and the inner boundary line to show that both boundary lines are located in the same area, and then It is characterized by performing estimation.

[Action]

In the image processing method according to the present invention, even when another closed boundary line exists within the closed boundary line that surrounds one area, each boundary line is on the same area. Since it has information indicating whether or not it is located at, it is possible to judge whether or not there is only a hole in one plane, or whether the surface of another object is visible through the hole. Easy to do.

〔Example〕

Hereinafter, the present invention will be described in detail with reference to the drawings showing its embodiments.

FIG. 3 is a block diagram showing the configuration of an image processing apparatus used for carrying out the method of the present invention.

This device includes television camera units 1 and 2 as a plurality of image pickup devices, an image data storage unit 12 that temporarily stores image data that is an output signal thereof, an image preprocessing unit 3,
The correspondence search unit (three-dimensional distance calculation unit) 4, surface estimation unit 5, data output device 6 for outputting final data, and the like.

The television camera units 1 and 2 are the same ITV cameras as the original ITV cameras that image a scene in front and output analog image data of, for example, 512 × 512 pixel sections,
The analog data, which is the output signal, is once stored in the image data storage unit 12, and then each pixel is A / D converted by the image preprocessing unit 3 as digital image data of 256 gradations, for example.

Both revision camera units 1 and 2 are
As shown in the schematic view of the figure, the television camera unit 1
Is arranged on the left side, and 2 is arranged on the right side. Then, by the television camera unit 1 on the left side, the object
The image obtained by capturing 0B is the left image IL, and the image obtained by capturing the object 0B with the right television camera unit 2 is the right image IR.

The image preprocessing unit 3 converts the analog outputs of the television camera units 1 and 2 into digital images as shown in the schematic diagram of FIG. 5 for the left image IL and the right image IR, respectively, as described above. By obtaining the strength and direction of the edge for each point of the image by the first-order differential operator, and performing thinning of the edge, threshold processing, extension processing, etc.,
Obtain occluded edges that encircle each region on the image. Then, the boundary of the area is obtained by tracing the edge by looking at the area on the right side, and the edge is segmented by fitting a straight line or a quadratic curve. A segment image as shown is generated.

For example, in the schematic diagram of FIG. 8 showing an example of obtaining a boundary line, a hatched region R has an outer peripheral boundary line B and hole boundary lines H1 and H2 therein, and each boundary line is S11 ~ S17, S21 ~ S25 and S31 ~
It is represented by the list in S37. The following segment correspondence search is performed on this data structure.

The correspondence search unit 4 associates the corresponding segments from the left and right images IL and IR obtained by the television camera units 1 and 2, respectively. It is possible to obtain a three-dimensional distance relationship based on the principle of triangulation for the segment for which the correspondence relationship has been obtained.

The surface estimation by the surface estimation unit 5 is performed by obtaining a loop connecting the three-dimensional segments for which the correspondence relationship has been obtained by the correspondence search unit 4 as described above. Each loop is independently processed as a component of a different surface, and each loop performs surface reconstruction. Specifically, by reconstructing the surface from each loop in the segment image shown in FIG. 6, the left and right images IL, IR displayed in the surface configuration as shown in the schematic view of FIG. Can be obtained. In this example, each of the left and right images IL and IR is composed of a cross-hatched surface, a hatched surface, and a sprite-coated surface.

However, in this case, for example, as shown in FIG. 8, when there is a hole in the area to be estimated as a surface, a closed loop formed by the segments S11 to S17 of the boundary line B1 of the outer peripheral contour is formed. It is necessary to estimate, as a surface, a portion within the encircled region, other than the region encircled by the both closed loops constituted by the segments S21 to S25 and S31 to S37 of the boundary lines H1 and H2 of the inner peripheral contour. There is.

In order to perform such processing, it is necessary to search the boundary lines included in the outer peripheral boundary line B1, that is, H1 and H2 in this example, from all the boundary lines in the image. However, as shown in FIG. 9, when another different object is seen in the hole, if the boundary line included in the outer peripheral contour boundary line B1 is simply selected, B2 will be selected at the same time as H1. Therefore, the outer contour boundary lines B1 and B2
When the surface is estimated by and, there arises a problem that an erroneous surface that does not actually exist is constructed, or the only surface candidate cannot be determined and the image cannot be interpreted.

Therefore, in such a case, according to the method of the present invention, by adopting a dummy segment as shown by a broken line in FIG. 1, it is possible to avoid constructing an erroneous surface and search space for selecting a boundary line. It is possible to reduce.

The dummy segment is used to function as a pointer for treating a boundary line that surrounds one area and other boundary lines existing in this boundary line as a group. . By adopting this dummy segment, it is possible to consider one boundary line for one area without distinguishing the boundary line of the region between the outer boundary line and the inner boundary line. Specifically, in the example of FIG. 1 in which the dummy segment is applied to the example shown in FIG. 8, the region R includes the segments S11 to S17, the dummy segment D1,
Segments S22 to S24, dummy segment D2, segment S3
2 to S37, S31, dummy segment D3, segments S25, S21, dummy segment D4, and then to segment S11.

Further, in the example of FIG. 2 in which the dummy segment is applied to the example shown in FIG. 9, the region R1 includes the segments S11 to S1.
7, Dummy segment D1, Segments S41 to S47, Dummy segment D2, Dummy segment D2 Returning to segment S11 is treated as a data structure forming a closed loop, and region R2 is treated as forming a different plane.

As described above, such a data structure is an outer peripheral boundary line, that is, an outer boundary line and a hole, when the digital image is divided into regions according to the brightness in the image pre-abbreviation section 3 and the boundary line is obtained. It can be realized by performing a process of setting a pointer between the inner peripheral contour boundary line, that is, the inner boundary line.

〔The invention's effect〕

As described above in detail, according to the present invention, in image processing by the stereo method, when estimating a region surrounded by a closed loop composed of continuous boundary segments from a segment image obtained by edge detection as a surface Even if a plurality of boundary lines exist in one area, information indicating that those boundary lines exist in the same area in advance, that is, a dummy segment (a pointer connecting the inner and outer boundary lines) By adding and grouping, it is possible to narrow the search target space of a group of segments that are surface candidates, reduce the possibility of erroneous estimation, and facilitate surface estimation. When there are many areas in an image, conventionally, it was necessary to check which outside boundary line each area contained within, but according to the present invention, the internal / external relation is a pointer. Since it is indicated by, it is possible to easily identify a large number of areas.

[Brief description of drawings]

FIG. 1 is a schematic view showing an implementation state of the method of the present invention in which a loop of a plurality of segments is grouped by adopting a dummy segment in the example shown in FIG. 8, and FIG. 2 is shown in FIG. FIG. 3 is a schematic diagram showing an implementation state of the method of the present invention in which a loop of a plurality of segments is grouped by adopting a dummy segment as an example, FIG. 3 shows an apparatus configuration used for implementing the image processing method of the present invention. FIG. 4 is a schematic block diagram showing the state of image pickup by the plurality of image pickup devices, and FIG.
FIG. 7 is a schematic diagram showing a digital image obtained by processing an image picked up by an image pickup device. FIG. 6 is a schematic diagram showing a segment image obtained by processing the digital image.
FIGS. 8 and 9 are schematic diagrams showing a surface estimated from a segment image and segments constituting the surface, and FIGS. 8 and 9 are schematic diagrams showing an example of boundary expression by a conventional image processing method.
Fig. 8 shows an example in which two holes are opened in one surface, and Fig. 9 shows an example in which one hole is opened in one surface and the other surface is visible from the hole. Shows. R, R1, R2 ... area, S11 to S17, S21 to S25, S31 to S37, S41 to S
47 …… Segment, IL, IR …… Left and right images, B, B1, B2, H1,
H2 …… Boundary line, 1,2 …… TV camera unit,
3 ... Image preprocessing unit, 4 ... Correspondence search unit, 5 ... Plane estimation unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (56) References Information Processing Society of Japan Research Report 90-CV-68-5, 90 [74] (1990-9-20) P. 1-8

Claims (1)

[Claims] 1. Information of a common line segment is obtained from each of a plurality of images obtained by capturing a recognition target located in a three-dimensional space by a plurality of imaging devices arranged at different positions on the same plane,
An image processing method that recognizes an object by extracting a region surrounded by a closed boundary line that is connected to them and estimating the extracted region as a plane. If a boundary line exists, a pointer connecting the outer boundary line and the inner boundary line is given to show that both boundary lines are located on the same area, and then the plane is estimated. An image processing method comprising: