JP2864945B2 - Industrial robot device with visual detection function - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an industrial robot apparatus with a visual detection function in which an industrial robot operates by certifying an article based on an image of a photographing apparatus.

[0002]

2. Description of the Related Art FIGS.
FIG. 5 is a diagram showing a conventional industrial robot device with a visual detection function similar to that shown in Japanese Patent Application Publication No. 9014, FIG. 5 is a conceptual overall configuration diagram, and FIG. FIG. 7 is a perspective view showing the arrangement of articles conveyed by the apparatus of FIG. 5, and FIG. 8 is a plan view of FIG. In the figure, (1) is an industrial robot having a control device (2), (3)
Is an article made of, for example, a cardboard box, which is gripped and transported by the industrial robot (1), and is a line (4) that bisects the upper surface
Are formed on the upper surface. Reference numeral (5) denotes a photographing device installed directly above the article (3) and connected to the control device (2) of the industrial robot (1) via the image processing device (6).

[0003] A conventional industrial robot apparatus with a visual detection function is configured as described above, and an article (3) is photographed from above.
The position of the article (3) is recognized by the image processing device (6) from the image taken by (5), and the article (3) is recognized by the industrial robot (1).
Is gripped and transported.

[0004] The image processing apparatus (6) extracts the article (3) by any of the following methods and identifies its position. That is, position recognition 1. The image input as shown in FIG.
The product (3) is extracted by binarization as shown in FIG. 1. Location certification The image input as shown in FIG. 6 (a) is compared with the pattern shown in FIG. 6 (c) of the article (3) inputted in advance for each pixel, and the overall similarity is tested. Certify the position of (3). Position recognition3. Paying attention to the corners or sides shown in FIG. 6D of the input image as shown in FIG.
The contour extraction shown in (e) is performed and the position of the article (3) is recognized.

[0005]

The above-mentioned conventional industrial robot apparatus with a visual detection function has the following problems since the position of the article (3) is recognized by the above method. That is, the position recognition 1. If the goods
When the difference between the brightness of (3) and the surrounding brightness is small, or when the articles (3) overlap each other, the article (3)
Extraction becomes difficult. In addition, the above-mentioned position recognition 2. In the case of (1), it is difficult to identify the article (3) itself when the article (3) is rotationally displaced. In addition, the above-mentioned position recognition 3. In case of goods (3)
When the position of the item (3) is greatly displaced, it becomes difficult to identify the article (3) itself.

When two or more articles (3) are lined up as shown in FIG. 7, when the outline of the image of the photographing device (5) is extracted, it becomes as shown in FIG. Although there is a need to certify two articles (3) as shown in FIG. 8 (b), there is a problem that erroneous recognition may occur as shown in FIG. 8 (c).

The present invention has been made to solve such a problem, and an object of the present invention is to provide an industrial robot apparatus having a visual detection function capable of locating a position regardless of the state of an article.

[0008]

In an industrial robot apparatus with a visual detection function according to the first aspect of the present invention, an industrial robot arranged opposite to an article and a photographing apparatus for photographing the article from above. One line segment is extracted from the device and the image outline of the article by the output of the photographing device as a first line segment, and the first line segment, the second line segment parallel to the first line segment, and the first line segment are extracted.
A third line segment orthogonal to the line segment and a first line segment separated from the third line segment;
Recognizing a fourth line segment orthogonal to the line segment, a quadrangle formed by the first line segment, the second line segment, the third line segment, and the fourth line segment and a predetermined quadrangle previously input are both used. And an image processing apparatus that outputs the position of the article when the two coincide with each other, and a control apparatus that instructs the industrial robot to convey the article based on the output of the image processing apparatus.

According to a second aspect of the present invention, there is provided an industrial robot apparatus with a visual detection function, comprising: an industrial robot arranged opposite to an article having a bisector formed on an upper surface; And a line segment are extracted from an image contour of an article output from the photographing device to extract quadrangles surrounded by the line segments. A half of the squares is selected, and a half of the square group is selected from the group of half of the squares, and a half of the square group of the half of the selected square group is two. In the case of two pieces, two pieces are recognized as one article and the position is output. When the half square is four or more pieces, two pieces from the end are recognized as one article respectively and the position is output. An image processing device that outputs Controller and is provided for instructing the conveyance of the article to industrial robots by the output of the processing unit.

According to a third aspect of the present invention, there is provided an industrial robot apparatus with a visual detection function, comprising: an industrial robot arranged opposite to an article having a bisector formed on an upper surface; And a line segment are extracted from an image contour of an article output from the photographing device to extract quadrangles surrounded by the line segments. A quadrilateral half of the quadrilateral and a quadrilateral equivalent to the predetermined quadrilateral are selected and selected.
Equivalent squares are counted as two half squares from the group of squares, and the even number of half squares is focused on a group of squares continuous to a predetermined square shape. If the number of squares is two, the two are recognized as one article and the position is output. If the number of half squares is four or more, two pieces from the end are each regarded as one article. An image processing device that certifies and outputs the position is provided, and a control device that instructs the industrial robot to convey articles by an output of the image processing device is provided.

[0011]

According to the first aspect of the present invention, the position of the article is determined by comparing the image contour of the article with a predetermined quadrangle, and a robot operation is instructed to the industrial robot.

In the invention according to the second aspect of the present invention, a half of a predetermined quadrangle with respect to a predetermined quadrangle is focused on from an image contour of an article having a bisector formed on an upper surface. The position of the article is recognized based on the arrangement of the half squares, and the robot operation is commanded to the industrial robot.

[0013] In the third aspect of the present invention configured as described above, a half of the predetermined quadrangle is focused on from the image contour of the article in which the bisector is generated on the upper surface. The position of the article is recognized based on the arrangement of the half squares, and the robot operation is commanded to the industrial robot. In addition, from the image contour of an article having a bisector on the upper surface which is equivalent to an article having a bisector on the upper face and focusing on a half of a predetermined quadrangle from the image contour, two half-rectangles are obtained. Counting, the predetermined 4 in the article in which the bisector was generated on the upper surface
The position of the article is determined based on the arrangement status of all the half rectangles including the half rectangle with respect to the square, and the robot operation is commanded to the industrial robot.

[0014]

【Example】

Embodiment 1 FIG. 1 and 2 show an embodiment of the present invention. FIG. 1 is a flowchart for explaining the operation of the image processing apparatus, and FIG. 2 is a view for explaining the image processing situation in the image processing apparatus of FIG. 1 and 2 are the same as those in FIG. 5 described above. In the figure, an article (3) is photographed from above by a photographing device (5) and input to an image processing device (6) to generate a screen shown in FIG. 2 (a). Hereinafter, the procedure of image recognition in the image processing apparatus will be described with reference to the flowchart shown in FIG.

That is, FIG. 2A shows a screen before image processing, and a contour is extracted from this image in step (101) to extract a line segment as shown in FIG. 2B. Next, the process proceeds to step (102), and a first line segment (7) is set as shown in FIG. And step (103)
Then, as shown in FIG. 2 (d), a line segment substantially parallel to the first line segment (7) is found and a second line segment (8) is set, and a line segment orthogonal to the first line segment (7) is set. Is found, and a third line segment (9) is set. Further, a line segment that is separated from the third line segment (9) and is orthogonal to the first line segment (7) is found, and a fourth line segment (10) is set.

Next, the process proceeds to step (104), and as shown in FIG. 2 (e), the first segment (7), the second segment (8), the third segment (9) and the fourth segment (10) ) Are extended, and the rectangle (11) surrounded by these straight lines is noticed.

Then, the process proceeds to step (105) and the square (1
Compare the vertical and horizontal lengths of 1) with the previously input plane and horizontal lengths of the article (3), that is, the vertical and horizontal lengths of a predetermined quadrangle. (106)
If not, go to step (107).

In step (106), the position of the article (3) is calculated, the certification is completed, and the result is input to the control device (2). Also step
In (107), another second line segment, third line segment, and fourth line segment corresponding to the first line segment (7) are searched.
Returning to 3), if it is not found and there are no second, third and fourth line segments to be searched, step (108)
Proceed to.

If the operation of searching for all the second, third and fourth line segments corresponding to the other first line segments has not been completed in step (108), the flow returns to step (102). If it has been completed, it is determined that there is no article (3).

Thus, the article (3) can be extracted even when the difference between the brightness of the article (3) and the brightness around the article is small or the articles (3) overlap each other. When the article (3) itself can be recognized even when the (3) is rotationally displaced, and when the plane of the article (3) is rectangular and the position of the planar end is largely shifted. The article (3) itself can be certified. Therefore, the position of the article can be recognized, and the abnormal operation of the industrial robot (1) due to the inability to be recognized can be eliminated, and the work efficiency of the industrial robot device with the visual detection function can be improved.

Embodiment 2 FIG. FIG. 3 is a diagram showing another embodiment of the present invention. FIG. 3 is a flowchart for explaining the operation of the image processing apparatus. The other parts of FIG. 3 are configured in the same manner as FIG. Hereinafter, the procedure of image recognition in the image processing apparatus will be described with reference to the flowchart shown in FIG.

That is, in step (201), one plane of the article (3) stacked and arranged near the industrial robot (1) is photographed by the photographing device (5), and the process proceeds to step (202). In step (202), the outline of the image photographed is extracted, line segments are extracted, and rectangles surrounded by those line segments are extracted. The length and width of the rectangle and the shape of the article (3) input in advance are extracted. Based on the vertical and horizontal lengths of the plane, that is, the vertical and horizontal lengths of the predetermined quadrangle, a quadrangle substantially half the plane of the article (3) is selected.

Next, proceeding to step (203), if selection of a quadrangle that is almost half of the plane of all articles (3) based on the vertical and horizontal lengths of the predetermined quadrangle is not completed, the process returns to step (202). If completed, the process proceeds to step (204). Step (20
If there is no substantially half square group in which the even half of the squares in the half group of the predetermined quadrangle selected in 4) are arranged to form the predetermined square, Certification of 3) is completed. In addition, in the quadrangle group of approximately half of the predetermined quadrangle selected in step (204), an approximately half of the quadrangle group in which an even number of the approximately half of the quadrangles are arranged to form the predetermined quadrangle is If there is, go to step (205) paying attention to it.

Then, in the group of substantially half rectangles, which is noticed in step (205), if there are two substantially half rectangles, it is recognized as one article (3), and the position of the article (3) is calculated. After the certification is completed, input is made to the control device (2). In the case where there are four or more half-half squares in the group of half-half squares that have been noted, each of the two from the end is identified as an article (3), and the position of the article (3) is calculated and authorized. Is completed and input to the control device (2).

Thus, even when articles (3) having a line formed at the center of the upper surface such as a cardboard box are lined up,
It is possible to certify the article (3) itself by preventing erroneous certification. Therefore, the abnormal operation of the industrial robot (1) due to the inability to recognize the position of the article can be eliminated, and the work efficiency of the industrial robot device with the visual detection function can be improved.

Embodiment 3 FIG. FIG. 4 is a view showing another embodiment of the present invention. FIG. 4 is a flowchart for explaining the operation of the image processing apparatus. The other parts of FIG. 4 are configured in the same manner as FIG. Hereinafter, the procedure of image recognition in the image processing apparatus will be described with reference to the flowchart shown in FIG.

That is, in step (301), a one-step plane of the article (3) stacked and arranged near the industrial robot (1) is photographed by the photographing device (5), and the process proceeds to step (302). In step (302), the contour of the image photographed is extracted, line segments are extracted, and a rectangle surrounded by those line segments is extracted, and the length and width of the rectangle and the previously input object (3) are extracted. A square which is approximately half the plane of the article (3) based on the plane length and width, that is, the height and width of the predetermined rectangle;
Select an equivalent quadrangle for almost one article (3).

Next, the process proceeds to step (303), in which a quadrangle that is substantially half of the plane of all the articles (3) based on the vertical and horizontal lengths of the predetermined quadrangle, and an equivalent quadrangle of substantially one article (3) If the selection of the polygon is not completed, the process returns to step (302), and if completed, the process proceeds to step (304). In a group of substantially half of the predetermined quadrangles selected in step (304) and a group of substantially equivalent quadrangles, an even number of the substantially half of the quadrangles are arranged to form a predetermined half. If there is no square group, the certification of the article (3) ends. In step (304), substantially equivalent quadrangles are treated as two half-quadrangles.

Further, in the group of substantially half of the predetermined quadrangles selected in step (304) and the group of substantially equivalent quadrangles, an even number of the approximately half of the predetermined squares are arranged side by side to form a predetermined square. If there is almost half of the quadrangle group that has formed, the process proceeds to step (305), noting that. Then, in the substantially half square group focused in step (305), the substantially half squares for one predetermined square are 2n-th and 2n + 1-th (n
= 1, 2, 3,) when step (3
Return to 04).

In the group of substantially half squares noted in step (305), if there are two substantially half squares, the process proceeds to step (306), and the two are combined into one article (3).
The position of the article (3) is calculated and input to the control device (2). Further, when the half square group in the half square group of interest has four or more half squares, two pieces from the end are respectively recognized as the article (3), the position of the article (3) is calculated, and the control device ( Input to 2). Then, the process returns to step (304).

Thus, when the articles (3) having a line formed at the center of the upper surface are partially arranged like the corrugated cardboard box, the articles (3) are arranged side by side. Even if products (3) with lines formed at the center of the cardboard are lined up and tape is stuck on the center of the upper surface of a part of those cardboard boxes, the center line can not be seen,
(3) The certification of itself becomes possible. Therefore, regardless of the condition of the article, the normal position can be recognized, the abnormal operation of the industrial robot (1) due to the inability to be recognized can be eliminated, and the working efficiency of the industrial robot device with the visual detection function can be improved. .

[0032]

According to the first aspect of the present invention, as described above, an industrial robot arranged to face an article, a photographing apparatus for photographing an article from above, and an article by an output of the photographing apparatus A single line segment is extracted from the image contour of the first line segment to be a first line segment, the first line segment, a second line segment parallel to the first line segment, a third line segment orthogonal to the first line segment, and the like. Recognize the fourth line segments apart from the third line segment and orthogonal to the first line segment, respectively,
The quadrangle formed by the first line segment, the second line segment, the third line segment, and the fourth line segment is compared with a predetermined quadrilateral input in advance. An image processing device for outputting a position and a control device for instructing an industrial robot to convey an article based on an output of the image processing device are provided.

As a result, the position of the article is identified by comparing the image contour of the article with a predetermined quadrangle, and a robot operation is instructed to the industrial robot. Therefore, the position can be recognized regardless of the state of the article, and the abnormal operation of the industrial robot due to the poor recognition can be eliminated, thereby improving the working efficiency of the industrial robot device with the visual detection function.

As described above, according to the second aspect of the present invention, an industrial robot arranged opposite to an article having a bisector formed on its upper surface, and a photographing apparatus for photographing an article from above. A line segment is extracted from an image contour of an article output by the apparatus and the photographing device, and quadrangles surrounded by the line segments are extracted, and a quadrangle which is half of a predetermined quadrangle previously input from these quadrangles is extracted. From the half square group, and pays attention to the half square group that is continuous to the predetermined square shape from the half square group. If the half square group in the noted half square group has two half squares, An image processing apparatus that certifies a piece as one article and outputs its position, and when four or more half squares are four or more, two pieces from the end are each recognized as one article and outputs the position. And the output of this image processing device It is provided with a control device for commanding a conveyance of the article to bot.

With this arrangement, a half of the predetermined quadrangle is focused on from the image contour of the article in which the bisector is generated on the upper surface, and the position of the article is recognized based on the arrangement state of the half quadrangle. Robot operation is instructed to the robot. Therefore, even when articles having bisectors formed on the upper surface are lined up, the position can be properly recognized, and the abnormal operation of the industrial robot due to the poor recognition can be eliminated, and the industrial robot apparatus with the visual detection function can be eliminated. This has the effect of improving work efficiency.

As described above, according to the third aspect of the present invention, an industrial robot disposed opposite to an article having a bisector formed on its upper surface, and a photographing apparatus for photographing an article from above. A line segment is extracted from an image contour of an article output by the apparatus and the photographing device, and quadrangles surrounded by the line segments are extracted, and a quadrangle which is half of a predetermined quadrangle previously input from these quadrangles is extracted. And a quadrilateral equivalent to the predetermined quadrilateral is selected, and the same quadrilateral is counted as two half quadrilaterals from the selected quadrilateral group, and an even number of the half quadrilaterals becomes the predetermined quadrilateral shape. Attention is paid to the continuous quadrilateral group, and when two half quadrilaterals are two in these quadrilateral groups, the two are recognized as one article and the position thereof is output, and the half quadrilateral is replaced by four. If there are more than two, two from the end are each one article An image processing apparatus for outputting the position with a constant, is provided with a control device for commanding a conveyance of the article to industrial robots by the output of the image processing apparatus.

With this arrangement, a half of a predetermined quadrangle is noticed from the image contour of the article in which the bisector is generated on the upper surface, and the position of the article is determined based on the arrangement state of the half quadrangle. Robot operation is instructed to the robot. Also, focusing on a predetermined quadrangle from the image contour of an article having an equal bisector on the upper surface and having no bisector on the upper face, counting the two quadrants as two half halves, Half 4 for a given quadrangle in the article for which the bisector was generated
The position of the article is recognized based on the arrangement status of all half squares including the square, and a robot operation is instructed to the industrial robot. Therefore, even when an article having a bisector formed on the upper surface and an article not having a bisector formed on the upper surface are arranged side by side, the position can be properly recognized, and abnormal operation of the industrial robot due to poor recognition is eliminated. This has the effect of improving the work efficiency of the industrial robot device with the visual detection function.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a first embodiment of the present invention, and is a flowchart illustrating an operation of an image processing apparatus.

FIG. 2 is a view for explaining an image processing situation in the image processing apparatus of FIG. 1;

FIG. 3 is a view showing a second embodiment of the present invention, and is a flowchart for explaining the operation of the image processing apparatus;

FIG. 4 is a view showing a third embodiment of the present invention, and is a flowchart for explaining the operation of the image processing apparatus;

FIG. 5 is a conceptual overall configuration diagram showing a conventional industrial robot apparatus with a visual detection function.

FIG. 6 is a view for explaining an image processing situation by the apparatus of FIG. 5;

FIG. 7 is a perspective view showing an arrangement state of articles conveyed by the apparatus of FIG. 5;

FIG. 8 is a plan view of FIG. 7;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Industrial robot 2 Control device 3 Article 5 Imaging device 6 Image processing device 7 1st line segment 8 2nd line segment 9 3rd line segment 10 4th line segment 11 Quadrangle

──────────────────────────────────────────────────続 き Continuation of front page (56) References JP-A-2-144774 (JP, A) JP-A-62-222108 (JP, A) JP-A-61-17906 (JP, A) JP-A-3-144 228589 (JP, A) JP-A-3-202290 (JP, A) JP-A-2-145277 (JP, A) JP-A-62-165206 (JP, A) JP-A-5-127723 (JP, A) (58) Field surveyed (Int. Cl. ⁶ , DB name) G01B 11/00 G01T 1/00 B25J 13/08

Claims (3)

(57) [Claims] 1. A and industrial robots, which are arranged to face the article, first extracts the imaging device for shooting the article, the one line segment from the image contour of the article by the output of the imaging device The first line, the second line parallel to the first line, the third line orthogonal to the first line, and the third line
Recognize a fourth line segment that is separated from the line segment and is orthogonal to the first line segment, and preliminarily inputs a quadrangle formed by the first line segment, the second line segment, the third line segment, and the fourth line segment. Predetermined 4
An image processing apparatus that compares the two sides of the square and outputs the position of the article when the two coincide with each other, and a control apparatus that instructs the industrial robot to convey the article by the output of the image processing apparatus. Industrial robot with built-in visual detection function. Wherein the industrial robot bisector is arranged opposite to the generated article on the upper surface, the imaging device for shooting the article, the line from the image contour of the article by the output of the imaging device To extract a quadrangle surrounded by those line segments, and from these quadrangles, a predetermined
A half of the quadrangle is selected, and from the half quadrangle group, the half quadrangle group that is continuous with the predetermined quadrangle shape is noted. If there are two squares, the two are identified as one of the above-mentioned articles and their positions are output, and the half of the four squares is output.
If there are four or more squares, an image processing device that recognizes each two from the end as one of the articles and outputs the position of the article, and instructs the industrial robot to transport the articles by the output of the image processing apparatus Industrial robot device with a visual detection function, comprising: 3. A industrial robot bisector on the upper surface is arranged opposite to the generated article, the imaging device for shooting the article, the line from the image contour of the article by the output of the imaging device To extract a quadrangle surrounded by those line segments, and from these quadrangles, a predetermined
A quadrangle equivalent to a half of the quadrangle and a quadrangle equivalent to the predetermined quadrangle are selected, and the quadrangle equivalent to the above quadrangle is counted from the selected quadrangle group as two quadrangles of the half, and the quadrangle of the half is selected. Attention should be paid to the above-mentioned quadrangular group in which even numbers continue in the above-mentioned predetermined quadrangular shape.
If the number of squares is two, the two are identified as one of the articles and the position is output. If the number of the half squares is four or more, two of each from the end are each one of the articles. An industrial robot device with a visual detection function, comprising: an image processing device that certifies and outputs the position thereof; and a control device that instructs the industrial robot to carry the article by the output of the image processing device.