JP2861014B2 - Object recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial applications]

The present invention relates to an object recognizing device, for example, when a robot grasps an uppermost object from among a large number of irregularly stacked processing objects by a robot, quickly and highly efficiently grasps the uppermost object to be grasped. The present invention relates to an object recognition device capable of performing recognition with high accuracy.

[Prior art]

Conventionally, an object recognizing device of this type recognizes an object by correlating a captured grayscale image with a model grayscale image captured in advance by a model object (Japanese Patent Laid-Open No. 60-48579).
Japanese Patent Application Laid-Open No. Sho 62-269287), an object outline is extracted, and the object is recognized by comparing it with a contour line of a model registered in advance (Japanese Patent Application Laid-Open No. 62-269287). Has been done.

[Problems to be solved by the invention]

However, since the former takes a one-way correlation in the grayscale image, the recognition object is limited to a simple object such as a cylindrical object. There is a problem in that an object that exists and is not suitable for grasping as a whole is recognized as a top-level object. In the latter case, the contour of the detected object extracted on the image is normally cut due to the influence of disturbance such as illumination, so that it is distinguished from the case where the object exists at a lower level and the contour is cut. It does not stick, the recognition accuracy improves when collating with the detailed contours of the model, but the recognition speed decreases.The recognition speed improves when collating with some major contours of the model, but the Even when a part is obstructed by a higher-order object, there is a problem that the object is recognized as the highest-order object. The present invention has been made in order to solve the above-described problems, and has an object to grasp a robot from an object existing in a complicated background or an irregularly stacked object. In other words, it is necessary to efficiently recognize an object that is appropriate for the object, that is, the top object.

[Means for Solving the Problems]

As shown in FIG. 1, the configuration of the invention for solving the above-mentioned problem includes a plurality of configurations for obtaining a contour of a detected object from a two-dimensional image and forming a contour of the detected object from the contour. Object recognition that extracts a line segment, matches the component line segment with the model line segment that forms the contour of the model to be compared, and recognizes an object based on the degree of coincidence between the component line segment and the model line segment. In the apparatus, a main model line segment storage unit that stores information about a main model line segment that forms a main part of a model outline, and a detail that stores information about a detailed model line segment that forms substantially the entire model outline. The model line segment storage unit compares the constituent line segment of the detected object with the main model line segment stored in the main model line segment storage unit, and obtains the position and orientation of the detected object according to the result of the comparison. Signs of choosing a candidate Selecting means, edge image calculating means for obtaining an edge image obtained by binarizing the differential image of the gray scale screen, and a detailed model line segment on the edge image based on the position and orientation of the detected object selected by the candidate selecting means. There is provided object specifying means for comparing the detailed model line segment stored in the storage means with the edge image of the detected object, and specifying the detected object according to the result of the comparison. Preferably, the edge image calculation means obtains a plurality of edge images obtained by binarizing differential images of a plurality of grayscale images having different illumination irradiation directions, and synthesizes the plurality of edge images to generate a synthesized edge image. The object specifying means detects the detailed model line segment stored in the detailed model line storage means on the synthesized edge image based on the position and orientation of the detected object selected by the candidate selecting means. This is to collate with the edge image of the object and specify the detected object according to the result of the collation.

[Action]

In the first stage, the candidate selecting means compares the constituent line segments constituting the outline of the detected object with the main model line segment, and selects an object that matches by a certain amount or more as a detected object candidate. In the second stage, an edge image obtained by binarizing the differential image of the grayscale image is obtained by the edge image calculation means. Since the edge image is an image obtained by binarizing the differential image with a predetermined threshold as a boundary, the edge image has a certain width, unlike the contour of the object, and is hardly interrupted by a disturbance like the contour. Next, the object identifying means compares the edge images of all candidates of the detected object selected in the first stage with the detailed model line segment, and identifies the target object according to the result of the comparison. In the invention of claim 2, in addition to the above, in the second stage,
An edge image is obtained by binarizing differential images of a plurality of grayscale images having different illumination irradiation directions by the edge image calculation means, and a synthesized edge image obtained by synthesizing the edge images is obtained. This edge image is a composite of images obtained by illuminating from different directions, and is an image obtained by binarizing a differential image with a predetermined threshold as a boundary. And it is not interrupted by a disturbance like a contour line. Next, the object identifying means compares the edge images of all candidates of the detected object selected in the first stage with the detailed model line segment, and identifies the target object according to the result of the comparison. As described above, in the first stage, the position and orientation of the detected object are obtained by comparing the constituent line segments of the object with the characteristic main model line segments, so that the calculation time is reduced. In the second stage, based on the position and orientation of the candidate for the detected object selected in the first stage, the final detected object is determined by the degree of coincidence when the detailed model line segment is projected on the edge image. Specified. Since the matching in the second stage is performed only with the candidate already selected in the first stage, and since the position and orientation of the candidate have already been obtained, the detailed model line segment is not displayed on the edge image. The calculation time is short because it is a simple collation of how much is recognized. In addition, since the edge image does not break the edge of the object due to the disturbance, the upper object and the lower object can be reliably determined.

【Example】

Hereinafter, the present invention will be described based on specific examples. In FIG. 2, many workpieces W are mounted on a pallet 5, and a camera 6 for imaging the workpiece W from the upper portion of the pallet 5 is provided. Also, a first illumination device for illuminating the workpiece W uniformly from the upper center is used as the illumination device for the workpiece W.
An illumination device 7, a second illumination device 8 for irradiating the workpiece W from the upper left, and a third illumination device 9 for irradiating the workpiece W from the upper right are provided. These illumination devices 7, 8, 9 are configured to illuminate the workpiece W when commanded by the illumination control circuit 2. The object recognition device A has a configuration in which a central processing unit 1 that performs data processing such as collation and determination, and a video signal obtained by a camera 6 are processed to detect a contour of a detected object and form a contour. An image processing device 3 for extracting line segments and performing data processing such as obtaining a composite edge image; and a storage device 4 for storing data relating to a model and data relating to a detected object.
And an illumination control circuit 2. Further, the image processing device 3 samples an image signal output from the camera 6 to generate grayscale image data in which the grayscale level is digitized, and obtains a brightness gradient by differential operation from the grayscale image data. An edge detection device 32 that generates edge image data representing an edge of an object image, and traces a contour line from the edge image data, extracts a component line constituting the contour line, and relates to the position of the component line. And a line segment extraction device 33 for generating data. The storage device 4 is composed of a RAM or the like, and stores a main model line memory 41 for storing information on the position of the main model line constituting the main part of the contour of the model. A candidate data memory 42 for storing data relating to the position and orientation of a detected object candidate obtained from a set of constituent line segments having a matching degree equal to or greater than a predetermined value, and a detailed model line segment that constitutes substantially the entire contour line of the model A detailed model line segment memory 43 for storing information relating to the position and the like, and a recognition result for comparing the detailed model line segment with an edge image on a memory in the central processing unit corresponding to the selected detected object candidate, and storing the result. And a memory 44. Next, the operation of the present apparatus will be described with reference to the flowcharts of FIGS. 3 and 4 showing the processing procedure of the object recognition apparatus A and the explanatory view of FIG. Usually, the first lighting device 7 is turned on by the lighting control circuit 2, and the video signal obtained by the camera 6 is input to the image input device 31. Then, in the image input device 31, the video signal is sampled and converted into a digital signal to generate a grayscale image. The grayscale image data is input to the edge detection device 32 and differentiated to generate an edge image. The edge image data is input to the line segment extraction device 33,
The contour of the object is extracted by tracking the ridgeline.
The outline is approximated by a broken line or a circle to obtain a line segment image as shown in FIG. Then, in step 100, the central processing unit 1
From the line segment image, a component line composed of a straight line whose radius is substantially equal to the radius of the corresponding circle of the main model line or a straight line whose length is equal to or more than a certain value is extracted, and in step 102, the center position and radius of the circle and the straight line are extracted. Are stored in the area A and the area B of the constituent line segment memory in the central processing unit, respectively. On the other hand, the main model line segment memory 41 stores the position data of the center position of the circle and both end positions of the line segment which are elements of the main model line segment as shown in FIG. 5 (b). In this embodiment, the main model line segment is composed of a circle C1 indicating the outline of a hole formed in the model and straight lines L1, L2, L3 indicating a part of the main ridge line. Next, the process proceeds to step 104 to determine whether or not the area A of the component line segment memory 34 is empty, thereby determining whether or not the component line segment to be compared remains. In the first execution cycle, as long as the component line segment is successfully extracted, the area A is naturally not empty. One circle that is a constituent line segment is selected. Next, at step 108, the area B of the constituent line segment memory
, Three straight lines which are arbitrary constituent line segments are selected. Then, the process proceeds to step 110 to calculate the degree of coincidence for all combinations of the selected constituent line segment and the main model line segment stored in the main model line segment memory 41. The degree of coincidence ρ is a parameter indicating how much the extracted constituent line segment and the main model line segment coincide with each other, including their relative positional relationship. In this embodiment, the degree of coincidence ρ is determined by the following equation. Here, ω _i is the weight given to the main model line segment, and if it matches the main model line segment with a large weight, the degree of coincidence as a whole increases accordingly. ω ₁ is a circle, ω ₂ ~
ω ₄ is the weight of the straight line. Δα _i indicates the degree of coincidence between the i-th model line segment and the constituent line segment to be compared. In the case of a circle, since a circle substantially equal to the radius of the circle of the main model line segment has already been extracted as a constituent line segment, Δα ₁ =
1.0. For a straight line (i ≠ 1), length (m _i) is, i-th main model line m _i the length of,
length (e _j ) is the extracted length of the j-th constituent line segment e _j . Δβ _i indicates the degree of coincidence between the relative positional relationship between the constituent lines and the relative positional relationship between the main model lines. Similarly, in the case of a circle, Δβ ₁ = 1.0. For a straight line (i ≠ 1), when the component line segment e _i exists in the region S, Δβ _i = 1.0 When the component line segment e _i is partially included in the region S, Δβ _i = 0.5 component line segment When e _i is not included in the region S at all, Δβ _i = 0. However, the region S is defined by the following equation. _{S = ∩N (m k, e} k; m i) i ≠ k, all N (m k, e k; m i) are matched one by one endpoint configuration line e _k to the main model line m _k when associating by is determined by the relative positional relationship of the major model line minutes, the presence range of the major model lines m _i. Therefore, the area S is
This is a common area of the possible areas determined from the correspondence between the three main model line segments and the three constituent line segments. Thus, in step 110, the degree of coincidence ρ is calculated. The combination of the main model line segment and the selected constituent line segment corresponds to a circle in this embodiment, and the other three
There are six types of correspondences based on the correspondence relationship between the two straight lines. For each of the combinations, the degree of coincidence is calculated, and the largest set of the degrees of coincidence is selected as a possible combination with the main model line segment. . Then, in the next step 112, the degree of coincidence ρ becomes the threshold T
h is determined, and the degree of coincidence ρ is
If it is larger, in step 118, the position of the detected object is shifted from the center position of the circle, which is one of the extracted constituent line segments, to the straight line of the constituent line segment corresponding to the specific straight line of the main model line segments. The attitude of the detected object is calculated from the inclination of the candidate data, and those values are stored in the candidate data memory 42. And
In step 120, it is determined whether or not the number of the selected detected object candidates is sufficient. If not, the process proceeds to step 122, and the extracted component line segments are converted into component lines in the central processing unit 1. The data is erased from the minute memory, the process returns to step 104, and the next constituent line segment is extracted. On the other hand, in step 112, the coincidence ρ is
If not, the corresponding set of the selected constituent line segment and the main model line segment is returned to region B with an evaluated label. Then, the process proceeds to step 114 to determine whether or not there is another combination extracted from the constituent line segments stored in the region B of the constituent line segment memory in the central processing unit 1. If the combination exists, the process proceeds to step 108, where three other straight lines are extracted from the constituent line segments, and the degree of coincidence ρ is calculated for all corresponding pairs in the same manner as described above. . If there is no other combination of the component lines to be extracted, in step 116, it is determined that the circle of the extracted component line is not a circle corresponding to the model, and the area of the component memory in the central processing unit 1 is determined. A is deleted from A, the process returns to step 104, the next other circle is extracted,
The other three straight lines are selected to generate a corresponding set, and the degree of coincidence is calculated in the same manner. If the number of candidates for the detected object selected in step 120 is sufficient, the selection of the candidates is terminated, and the processing of step 124 and subsequent steps is executed. At this time, the detection object candidate is the fifth
The selection is made as shown in FIG. In step 124, the second illumination device 8 is turned on, the processed object W is illuminated from the upper left, and a video signal is input from the camera 6. In step 126, the gray level image is differentiated and binarized by the edge detection device 32. The obtained edge image is stored in the edge image memory in the central processing unit. Next, in step 128, the third lighting device 9 is turned on, the processing object W is irradiated from the upper right, and a video signal is input from the camera 6,
In step 130, an edge image at the time of upper right illumination obtained by differentiating the grayscale image by the edge detection device 32 and binarizing is obtained. Next, in step 132, the central processing unit 1 calculates the logical sum of the edge image at the time of the upper right illumination and the edge image at the time of the upper left illumination already stored in the edge image memory. A composite edge image as shown in FIG. 4D is stored in the edge image memory. Next, in step 134, the position and orientation data of one detected object candidate stored in the candidate data memory 42 is selected. Next, in step 136, on the synthesized edge image stored in the edge image memory in the central processing unit 1, based on the position and orientation data of the detected object candidate, the data is stored in the detailed model line segment memory 43. When a detailed model line segment as shown in FIG. 5 (e) is projected, the number of pixels γ _k of the overlap between the detailed model line segment and the edge of the k-th detected object candidate selected in the composite edge image
Is calculated. Here, E _i is the number of overlapping pixels in the detailed model line segment e _i , and m is the number of detailed model line segments. Then, in the next step 138, if it is determined that the calculation of the number of overlapping pixels has not been completed for all the detected object candidates stored in the candidate data memory 42, the process returns to step 134, and other detections are performed. The number of overlapping pixels is calculated for the object candidate. When the number of overlapping pixels γ _k is calculated for all the selected detected object candidates, in step 140, the detected object having the largest number of overlapping pixels γ _k is recognized as the top object. In this manner, the top object can be accurately recognized from among a large number of irregularly stacked objects having the same shape as shown in FIG. 5 (f). Also, instead of the number of overlapping pixels γ _k , a detailed model line segment e _i
Number of pixels may be recognized as E _i / V _i a ≧ T satisfying for all detailed model line detection object overlaps maximum object the topmost object in proportion among the candidates when a V _i a. Here, T is an allowable value of the overlapping ratio of each detailed model line.

【The invention's effect】

The present invention relates to a candidate selecting unit for comparing a constituent line segment of a detected object with a main model line segment and selecting a candidate for a detected object in accordance with a result of the comparison, and an edge image obtained by binarizing a differential image of a grayscale image. And an object specifying means for comparing the edge image of the detected object selected by the candidate selecting means with the detailed model line segment on the edge image, and specifying the detected object according to the result of the comparison. That the candidate selection time is reduced, the matching method with the detailed model line is simplified and the matching time is shortened, and the matching is performed using an edge image that is not interrupted by disturbance. Therefore, there is a remarkable effect that the recognition accuracy of the upper object and the lower object is improved. According to the second aspect of the present invention, a plurality of edge images obtained by binarizing differential images of a plurality of grayscale images having different illumination irradiation directions are obtained, and the plurality of edge images are synthesized to obtain a synthesized edge image. On the combined edge image, the detailed model line segment is compared with the edge image of the detected object, and the detected object is specified according to the result of the comparison. Therefore, since the combined edge image is obtained from a plurality of grayscale images having different illumination directions, the uppermost detection object can be detected with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an outline of the present invention;
FIG. 2 is a block diagram showing the configuration of an object recognition apparatus according to a specific embodiment of the present invention, FIGS. 3 and 4 are flowcharts showing the operation of the apparatus, and FIG. FIG. DESCRIPTION OF SYMBOLS 1 ... Central processing unit, 2 ... Lighting control circuit 3 ... Image processing device, 4 ... Storage device 6 ... Camera, 7 ... First lighting device 8 ... Second lighting device, 9 ... Third Lighting equipment

Continuation of front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G06T 7/00

Claims (2)

(57) [Claims] 1. A contour line of a detected object is obtained from a two-dimensional image,
A plurality of component lines constituting the contour of the detected object are extracted from the contour, and the component lines are compared with the model line constituting the contour of the model to be compared. An object recognition device for recognizing an object based on a degree of coincidence between the model line segment and the model line segment. Detailed model line segment storage means for storing information about detailed model line segments constituting substantially the entire contour line of the model; constituent line segments of the detected object and main model line segments stored in the main model line segment storage means Selecting means for determining the position and orientation of the detected object in accordance with the matching result and selecting a candidate for the detected object; and edge image calculating means for obtaining an edge image obtained by binarizing the differential image of the gray scale screen When On the edge image, a detailed model line segment stored in the detailed model line segment storage unit is compared with an edge image of the detected object based on the position and orientation of the detected object selected by the candidate selection unit. And an object specifying means for specifying a detected object in accordance with the result of the comparison. 2. An edge image calculating means according to claim 1, wherein said differential image of a plurality of grayscale images having different illumination directions is obtained.
Calculating a plurality of edged image values, synthesizing the plurality of edge images to obtain a combined edge image, wherein the object identifying unit detects, on the combined edge image, a detection selected by the candidate selecting unit. Based on the position and orientation of the object, the detailed model line segment stored in the detailed model line segment storage unit is collated with the edge image of the detected object, and the detected object is identified according to the collation result. The object recognition device according to claim 1, wherein: