JP5458885B2 - Object detection method, object detection apparatus, and robot system - Google Patents

Description

  The present invention relates to an object detection method and an object detection apparatus that determine the three-dimensional position and orientation of an object, and a robot system using the same.

When a gripping operation is performed using a robot, it is necessary to detect the object and detect the position and orientation. Conventionally, a method of detecting an object by performing pattern matching using a two-dimensional grayscale image from a CCD camera has been used as an object detection method.
However, this method is effective when all the objects have the same posture, but when the objects are in a random posture with respect to the CCD camera, such as in a stacked state, the position and posture of the object are changed. It was difficult to detect.
For this reason, conventionally, before detecting an object, it was necessary to separately provide a step of taking out and aligning the object with a parts feeder.
Also, since a parts feeder is required for a single type of object, if there are many types of objects, the cost and effort required to install the parts feeder will increase, and the robot's gripping operation will be automated. It was an obstacle.

In order to solve this problem, in order to detect an object in an arbitrary posture, a method of pattern matching by measuring a three-dimensional shape of an object with a range sensor (distance sensor) has been proposed.
For example, a plurality of feature points on the distance data image are detected, and the positional relationship between the feature points on the shape model of the target object in which the positional relationship of each feature point is stored in advance is most closely matched. Object detection can be performed by selecting a feature point group (see Patent Document 1).
A method using both distance data and a two-dimensional gray image has also been proposed. Roughly identify the area where the object exists from the grayscale image, determine the position and orientation of the object's inertial principal axis in 3D space from the distance data image measured from the same viewpoint as the grayscale image, and around this inertial principal axis This is to detect the degree of coincidence with the shape model of the object stored in advance when the object is virtually rotated (see Patent Document 2).

JP 2001-143073 (page 6, FIG. 1) JP-A-7-287756 (page 13, FIG. 1)

However, in the object detection method described in Patent Document 1, it is necessary to compare and compare three or more feature points. Therefore, an enormous amount of calculation is required, and it takes time for the arithmetic processing.
There is also a problem that a large storage area is required for one model.

Moreover, in the method of Patent Document 2, it is difficult to detect an object from a grayscale image in a state where the objects are piled up (stacked in a random posture), and there is a problem that the system lacks robustness. .
Moreover, the method of checking the degree of coincidence when the object is virtually rotated around the inertial spindle has a problem that it takes a long processing time.
The present invention has been made in view of such problems, and provides an object detection method, an object detection device, and a robot system equipped with the object detection method that can accurately detect the position and orientation of an object with a small amount of calculation. For the purpose.

In order to solve the above problem, the present invention is as follows.
The present invention (Claim 1) is an object detection method for detecting the position and orientation of an object in a three-dimensional space, and distance data for obtaining distance data by measuring the distance from each position on the object. An acquisition step; a plane / edge extraction step for extracting a plane portion and an edge portion of the object in the distance data; a plane selection step for selecting an arbitrary plane from the plurality of extracted plane portions; An edge image creating step for creating an edge image that is projected perpendicularly from the edge portion with respect to a plane and expressed by shading corresponding to a distance from the edge portion to the selected plane; A point on the edge image by comparing the edge image with a reference edge image calculated in advance and stored as a database with the object at a known position. A voting step for associating each point on the reference edge image having a gray value with each other and sequentially voting the position difference in the position space to obtain the most voting place, and rotating the reference edge image And a position / orientation calculation step for obtaining the position and orientation of the object from the angle of the reference edge image most voted and the position in the position space. .
The present invention (Claim 2) is the object detection method according to Claim 1, wherein the plane / edge extraction step calculates an average curvature based on the distance data obtained in the distance data acquisition step; And a step of extracting, as the plane portion, a region obtained by collecting places where the absolute value of the average curvature is smaller than a predetermined value.
The present invention (Claim 3) is the object detection method according to Claim 1, wherein, in the plane selection step, the plane having the largest area is preferentially selected from the plane portions extracted in the plane / edge extraction step. It is characterized by selection.
The present invention (Claim 4) is an object detection method for detecting the position and orientation of an object in a three-dimensional space, and distance data for obtaining distance data by measuring a distance from each position on the object. For the selected plane, an acquisition step, a plane extraction step for extracting a plane portion of the object in the distance data, a plane selection step for selecting an arbitrary plane from the extracted plurality of plane portions, A distance image creating step for projecting the distance data vertically to create a distance image representing the distance between the object and the selected plane in shades, and a minute image in the distance image created in the distance image creating step A normal angle image is created by calculating a normal direction of a local virtual surface for each region, and creating a normal angle image by expressing the angle between the calculated normal direction and the plane in shades. Comparing the step and the reference distance image calculated in advance with an object at a known position and stored as a database with the distance image created in the distance image creation step, and corresponding points having the same gray value, The distance image voting step for sequentially voting the difference in position to the position space, the reference normal angle image calculated with the object at a known position in advance and stored as a database, and the normal angle image creating step A normal angle image voting step for comparing each of the points having the same gray value with each other by comparing the normal angle image and sequentially voting a difference in the position to the position space, the distance image voting step, and the normal angle As a result of the image voting step, a voting step for obtaining the most voting place in the position space, and the voting step as the reference distance image and the reference method A position / posture calculation step for obtaining the position and posture of the object from the angle of the reference distance image where the most votes are gathered and the position in the position space; It is characterized by that.
The present invention (Claim 5) is the object detection method according to Claim 4, wherein the plane extraction step calculates an average curvature based on the distance data obtained in the distance data acquisition step; And a step of extracting, as the plane portion, a region obtained by collecting places where the absolute value of the average curvature is smaller than a predetermined value.
According to the present invention (Claim 6), in the object detection method according to Claim 4, in the plane selection step, a plane having the largest area is preferentially selected from the plane portions extracted in the plane extraction step. It is characterized by that.
The present invention (Claim 7) is an object detection device for detecting the position and orientation of an object in a three-dimensional original space, and measures distance from each position on the object to acquire distance data. A distance sensor; and a calculation device that performs a calculation based on a measurement value of the distance sensor, wherein the calculation device calculates a plane portion and an edge portion of the object in the distance data measured by the distance sensor. A plane / edge extraction unit for extraction, a plane selection unit for selecting an arbitrary plane from the extracted plurality of plane parts, and a projection perpendicular to the selected plane from the edge part to surround the edge part the crack region, calculated with respect to the edge image generating unit configured to generate an edge image representing the edge portion in shading which corresponds to the distance to the selected plane, the object in the already known positions, The edge image database storing the quasi-reference edge image, the edge image created by the edge image creation unit and the reference edge image stored in the reference edge image database are compared, and the edge image database A voting unit that associates each point on the reference edge image having the same gray value as the point and sequentially votes the position difference on the position space to obtain the most voting place, and rotates the reference edge image And an object detection unit that obtains the position and orientation of the object from the angle of the reference edge image where the most votes are collected and the position in the position space. It is said.
The present invention (Claim 8) is the object detection device according to Claim 7, wherein the plane / edge extraction unit calculates an average curvature based on the distance data obtained by the distance sensor, and the average curvature is calculated. A region obtained by gathering locations where the absolute value of the value is smaller than a predetermined value is extracted as the planar portion.
The present invention (Claim 9) is the object detection device according to Claim 7, wherein the plane selection unit gives priority to the one having the largest area among the plane portions extracted by the plane / edge extraction unit. It is characterized by selection.
The present invention (Claim 10) is an object detection device that detects a position and orientation of a target object in a three-dimensional space, and measures a distance from each position on the target object to acquire distance data. A sensor and a calculation device that performs a calculation based on a measurement value of the distance sensor, the calculation device extracting a plane portion of the object in the distance data, and a plurality of extracted A plane selection unit that selects an arbitrary plane from the plane portion of the image, and the distance data is projected perpendicularly to the selected plane to express the distance between the object and the selected plane in light and shade. A distance image creating unit for creating a distance image; and calculating a normal direction of a local virtual surface for each minute region in the distance image created in the distance image creating step, and calculating the normal direction and the plane the angle formed by the A normal angle image creating unit that creates a normal angle image by expressing the density, a reference image database that stores in advance a reference distance image that is calculated with respect to the object at a known position and is used as a reference; A reference normal angle image database that stores a reference normal angle image that is calculated with respect to the target at a known position and stores a reference normal angle image, and the distance image created by the reference distance image and the distance image creation unit. The distance image voting unit that sequentially associates points having the same gray value and voting the difference of the positions sequentially into the position space, and the method created by the reference normal angle image and the normal image creation unit A normal angle image voting unit for comparing points with the same gray value by comparing with the line angle image and sequentially voting the position difference to the position space, the distance image voting unit, and the normal angle image For each vote in the voting department As a result, the voting unit for obtaining the most voting part in the position space and the operation for obtaining the part by the voting part are sequentially repeated by rotating the reference distance image and the reference normal angle image, thereby obtaining the most voting gathering. And a position / posture calculation unit that obtains the position and posture of the object from the angle of the reference distance image and the position in the position space.
The present invention (Claim 11) is the object detection apparatus according to Claim 10, wherein the plane extraction unit calculates an average curvature based on the distance data obtained by the distance sensor, and calculates the absolute value of the average curvature. A region obtained by gathering locations where the values are smaller than a predetermined value is extracted as the planar portion.
The present invention (Claim 12) is the object detection device according to Claim 10, wherein the plane selection unit preferentially selects the plane portion extracted from the plane extraction unit with the largest area. It is characterized by that.
The invention of the present application (Claim 13) is a robot system for gripping an object, a transport robot that grips and transports the object, and the object detection device according to any one of claims 7 to 12. And a robot control device that controls a movement trajectory of the transfer robot relative to the object based on an output of the object detection device.
The invention of the present application (Claim 14) is the robot system according to Claim 13, further comprising a work stocker that stores the object in a random posture, and the distance sensor of the object detection device is configured for the entire area of the work stocker. The distance data is acquired.
The invention of the present application (Claim 15) is the robot system according to Claim 13, further comprising a work stocker for storing the object in a random posture, wherein the distance sensor of the object detection device The distance data is obtained for a predetermined partial area set in advance.

According to the present invention, when detecting the position and orientation of an object, only the translation and rotation components on the plane of the edge image and the reference edge image obtained from the object to be detected need to be obtained, and the amount of calculation is reduced. Since the number is small, the position and orientation of the object can be detected earlier when an arithmetic device having the same performance is used.
Furthermore, in order to detect the highest part of the vote in the position space, even if a part of the object is hidden from the distance sensor or the distance data is affected by disturbance etc., the object can be stably detected. Can be detected.
In addition, when detecting the position and orientation of the object, only the translation and rotation components on the plane of the distance image and normal angle image obtained from the object to be output, the reference distance image and the reference normal angle image are obtained. The matching process can be performed at high speed. Furthermore, because the method of searching for the highest part of the vote in the position space is taken, even if a part of the object is hidden or other objects are reflected, the vote is gathered and the object is detected stably. Is possible.
Further, by applying the object detection device to the robot system, when the robot system performs a gripping operation of the gripping target, the object detection device can quickly and accurately determine the position and orientation of the gripping target. Therefore, a robot system with high work efficiency can be obtained.

It is a flowchart which shows the process sequence concerning 1st Embodiment of this invention. It is a figure which shows the whole structure of the robot system concerning 1st, 2nd embodiment of this invention. It is a block diagram of an object detection device showing an embodiment of the present invention. It is a figure which shows the production | generation procedure of the edge image concerning 1st Embodiment of this invention. It is a figure which shows an example of the edge image concerning 1st Embodiment of this invention. (A)-(C) are figures which show the aspect of the vote to the position space concerning 1st Embodiment of this invention. It is a flowchart which shows the process sequence concerning 2nd Embodiment of this invention. It is a schematic diagram for demonstrating the production | generation aspect of the edge image concerning 2nd Embodiment of this invention. (A) is a figure which shows the distance image concerning 2nd Embodiment of this invention, (B) is a figure which shows the normal-line angle image concerning 2nd Embodiment of this invention.

  Hereinafter, the best specific example for carrying out the method of the present invention will be described with reference to the drawings.

First, a schematic configuration of a robot system using an object detection device according to an embodiment of the present invention will be described.
[First Embodiment]
As shown in FIG. 2, the robot system 100 includes an object detection arithmetic device (arithmetic device) 50, a robot control device 70, a work transfer robot 81, a gripping device 82, a distance sensor 83, and a work stocker 84. The distance sensor 83, the calculation device 50, the robot control device 70, and the workpiece transfer robot 81 are connected so as to be able to communicate with each other.
The work stocker 84 is a box-shaped container containing a work W as a detection target (target). A large number of workpieces W are stored in a pile in the workpiece stocker 84 in a random posture (three-dimensional direction taken by the workpieces W).
The distance sensor 83 is disposed above the work stocker 84 and is set to detect distance data to be described later in the entire area of the work stocker 84.
Here, the workpiece transfer robot 81 is configured by a vertical articulated manipulator, and a gripping device 82 is attached to the tip of the workpiece transfer robot 81. The gripping device 82 may be any device as long as it can grip (hold) the workpiece W, such as a gripper device that grips the workpiece W or an adsorption device using air or electromagnetic force.
The robot control device 70 is configured by a computer, and is configured to control each unit device of the robot system 100.
The functional configuration of the arithmetic device 50 will be described later.
FIG. 3 shows an example of a block diagram of an object detection apparatus that detects the position and orientation of an object from distance data. Hereinafter, the block of the object detection apparatus will be described with reference to FIG.
The arithmetic device 50 is composed of one or more computers, and includes functional blocks 51 to 55. The plane / edge extraction unit 51 extracts a plane portion and an edge portion of the workpiece W from distance data obtained by measuring the workpiece W in a three-dimensional space. The plane selection unit 52 arbitrarily selects and calculates one plane from a plurality of plane portions extracted by the plane / edge extraction unit 51. The gray edge image creation unit 53 performs a process of expressing the distance from each edge part to the calculated plane as a gray level at a position obtained by projecting each edge part perpendicularly to the plane calculated by the plane selection unit 52. The reference edge image database 54 stores gray-scale edge data proportional to the distance calculated in advance with the workpiece W at a known position. The position / posture calculation unit 55 of the work W performs a process of calculating the position and the posture by calculating the distance of the work W by comparing the gray edge created by the gray edge image creation unit 53 with the data of the reference edge image database. Do. Specifically, each point on the reference edge image having the same gray value as that on the gray edge image is associated with each other, and the position difference is sequentially voted in the position space to obtain the most voted part.
Thereafter, the reference edge image is rotated and the voting is repeated in the same manner, and the position and the posture of the workpiece W are obtained by obtaining the angle of the reference edge image having the most votes and the position in the position space. ing.
The result calculated by the position / orientation calculation unit 55 in this manner is displayed on the display unit 60 to allow the operator to perform visual confirmation work or the like, or to transmit to the robot control device 70 for information for the next work. It is supposed to be.

FIG. 1 is a flowchart showing a procedure of an object detection method according to the first embodiment of the present invention.
In order to perform the object detection, it is necessary to store in advance the information (reference edge image) of the object to be the workpiece W to be detected in the arithmetic device 50. The procedure will be described below.
First, in step S101, distance data is acquired using the distance sensor 83 for the workpiece W placed in a predetermined posture at a predetermined position set in advance.
In this embodiment, the distance data refers to positional information on the plane (x axis, y axis) and information on the distance (z axis) between the workpiece W corresponding to each pixel and the distance sensor 83 on the plane. In pixel units.
The distance sensor 83 that acquires the distance data obtains a corresponding point between each image obtained by capturing the workpiece W with a device (three-dimensional scanner) that scans the workpiece W with laser light to obtain a three-dimensional shape, or a plurality of cameras. It is composed of a device (compound eye stereo view) that obtains the three-dimensional information of the workpiece W from the principle of triangulation. In addition, as long as the distance of all the parts of the workpiece | work W of a detection can be detected, you may use the distance sensor of such a system.
In step S102, a plane portion and an edge portion are extracted from the distance data. Whether or not the shape obtained from the distance data is a plane can be determined by obtaining a curvature from the surrounding distance data. A typical example of obtaining the curvature is an average curvature, and the average curvature H of the plane portion can be obtained by Expression (1).

なお、ｆは距離データ（ｘ、ｙ、ｚ）を表わしている。また、ｆｘ、ｆｙ、ｆｘｘ、ｆｙｙ、ｆｘｙは、それぞれ関数ｆに対して添字ｘ、ｙに関しての微分を表している。
平均曲率Ｈが０（Ｈ＝０）のときは、抽出された平面部分は平面（平坦）である。また、平均曲率Ｈが正（Ｈ＞０）のときには、抽出された平面部分が凸曲面状であり、平均曲率Ｈが負（Ｈ＜０）のときには、抽出された平面部分が凹曲面状となる。
ここでは、式（１）で算出された平均曲率Ｈの絶対値がある程度小さい箇所（予め設定された閾値Ｍよりも平均曲率の絶対値が小さい箇所（Ｍ＞｜Ｈ｜））を集合させて（グループ化して）、領域分割を行うことにより平面とみなせる（近似できる）領域を抽出することができる。
また、距離データからエッジ部分を抽出するには、隣接した距離データとの位置の差が大きい箇所を求めることでエッジ部分を抽出することが出来る。
次に、ステップＳ１０３ではステップＳ１０２で抽出された平面部分から任意の平面を選択し、平面方程式を算出する。ステップＳ１０３において平面を選択する際には、平面領域の大きさが大きいものを優先して選択する。
領域の大きさが大きい平面を優先して選択することにより、物体検出の精度をより向上させることができる。この選択により、ワークストッカ８４内のワークＷから１つのワークが選択されることとなる。
平面は一般に式（２）で表すことができる。
ｚ＝ａｘ＋ｂｙ＋ｃ ・・・・（２）
選択された平面領域の各距離データの座標を
（ｘｉ、ｙｉ、ｚｉ） ｉ＝１、…、ｎとすると、
式（３）で表すことができる。

Note that f represents distance data (x, y, z). Further, fx, fy, fx, fyy, and fxy represent differentiations with respect to the subscripts x and y with respect to the function f, respectively.
When the average curvature H is 0 (H = 0), the extracted plane portion is a plane (flat). When the average curvature H is positive (H> 0), the extracted plane portion is a convex curved surface, and when the average curvature H is negative (H <0), the extracted plane portion is a concave curved surface. Become.
Here, locations where the absolute value of the average curvature H calculated by the equation (1) is small to some extent (locations where the absolute value of the average curvature is smaller than a preset threshold M (M> | H |)) are gathered. By dividing the area (by grouping), an area that can be regarded as a plane (approximable) can be extracted.
Further, in order to extract the edge portion from the distance data, the edge portion can be extracted by obtaining a portion having a large position difference from the adjacent distance data.
Next, in step S103, an arbitrary plane is selected from the plane portions extracted in step S102, and a plane equation is calculated. When selecting a plane in step S103, a plane having a large size is selected with priority.
By preferentially selecting a plane having a large area size, the accuracy of object detection can be further improved. By this selection, one work is selected from the works W in the work stocker 84.
The plane can be generally represented by formula (2).
z = ax + by + c (2)
If the coordinates of each distance data of the selected plane area are (xi, yi, zi) i = 1,.
It can be expressed by equation (3).

この式をＲ＝ＰＱとおくと、式（４）となり、平面方程式の係数ａ、ｂ、ｃを求めることができる。

When this equation is set to R = PQ, equation (4) is obtained, and the coefficients a, b, and c of the plane equation can be obtained.

次に、ステップＳ１０４ではステップＳ１０３で算出された平面にエッジ部分を垂直に投影してエッジ画像を生成する。

Next, in step S104, the edge portion is vertically projected on the plane calculated in step S103 to generate an edge image.

FIG. 4 shows an edge image generation procedure.
The plane portions 2a to 2c and the edge portions 3a to 3c of the workpiece W are extracted by the procedure up to step S102. In step S103, the plane 2 having the largest area is selected from the plane portions 2a to 2c, and the plane 4 is calculated. The edge portions 3 a to 3 c are projected perpendicularly on the plane 4. At this time, a distance from a point on the edge portions 2a and 2c to a point on the edge portions 5a and 5c projected onto the plane is obtained, and this is converted into a gray value to generate an edge image.

FIG. 5 shows an example of the edge image.
The gray value is determined in proportion to the distance from the edge portion to the plane. Since the edge portion 3b is on the plane 4, it is displayed in the middle of a light color and black as represented by 5b on the edge image. Since the edge portion 3a is on the workpiece W side with respect to the plane, the projected edge portion 5a is displayed in a color darker than 5b. On the contrary, since the edge portion 3c is on the space side with respect to the plane, the projected edge portion 5c is displayed in a lighter color than 5b. However, in the example of the distance image in FIG. 5, a portion that is not an edge portion is displayed in white. An edge image is created by these steps.
In step S105, the created edge image is registered in the database as a reference edge image. At this time, the equation of the plane 4 needs to be stored together with the edge image.

Next, a procedure for detecting the position and orientation of the workpiece W actually loaded on the workpiece stocker 84 will be described. The processing from step S101 to S104 is the same as the above-described reference edge image acquisition procedure.
As shown in FIG. 1, in step S101, the distance sensor 83 acquires distance data (actual scene) for the entire work sticker 84 (distance data acquisition step). In step S102, a plane portion and an edge portion are extracted from the distance data (plane / edge extraction step).
In step S103, an equation representing the selected plane is calculated by selecting one plane portion from the plane portions extracted in step S102 (plane selection step). Note that when selecting the plane portion, the one having a large area of the plane portion or the one at the top is selected.
In this way, it is possible to select a work W with less interference by obstacles such as other work W or a plane of the work W stacked on the upper side, and when the object is detected and gripped by the robot arm 81 The workpiece W that can be easily gripped can be selected, and the accuracy of gripping the workpiece W by the robot arm 81 is improved.
In step S104, an edge image is projected perpendicularly to the calculated plane to create an edge image (edge image creation step).
Next, in step S106, voting is performed in the position space in comparison with a reference edge image registered in advance as a model.

FIG. 6 shows a voting procedure for the position space. FIG. 6A shows a reference edge image, and FIG. 6B shows an edge image created for an actual scene.
The point P0 (x0, y0) on the edge of the reference edge image registered as a model is associated with the point P1 (x1, y1) on an arbitrary edge of the edge image created in the actual scene. At this time, only those with close gray values at each point are associated with each other, and the difference between the positions is voted on the position space (x1-x0, y1-y0) shown in FIG. 6C. This voting is repeated for points on the edge of the edge image of the actual scene (voting step).
Next, in step S107, the most concentrated part (peak) of the vote is searched from the position space corresponding to the angle of each reference image, and the angle of the reference image and the position where the vote is concentrated are obtained.
In step S108, the reference edge image is rotated by a predetermined angle to execute the processes in steps S106 and S107, and the voting is repeated to another position space each time.
The process of step S108 is repeatedly executed until the cumulative rotation amount of the reference edge image reaches a predetermined angle (here, 360 degrees).
Since the equations of the plane of the workpiece W placed at a known position and the plane of the workpiece W during actual work are obtained in step S103, the posture of the plane portion of the workpiece W during actual work is specified. it can. Further, by obtaining the most concentrated angle and position of voting, the position and rotation amount of the workpiece W on the plane can be known, so that the position and posture of the workpiece W in the three-dimensional space can be calculated. (Step S109)
Steps S108 and S109 correspond to a position / posture calculation step.

Since the object detection apparatus and the robot system according to the first embodiment of the present invention are configured as described above, as a method for obtaining the position and orientation of the workpiece W, first, the plane portion and the edge of the workpiece W are extracted, A plane is calculated by arbitrarily selecting from a plurality of extracted plane parts, and an edge image is created by expressing the distance from the edge to the plane in shades at the position where the edge is projected perpendicularly to the plane. Are compared with the reference edge image calculated by the workpiece W at the position and stored as a database, and each point on the reference edge image having the same gray value as the point on the edge image is associated with each other, and the difference between the positions is assigned to the position space Voting sequentially to find the most voting location, rotate the reference edge image in the same manner and repeat the voting in turn, and determine the angle of the most voting reference edge image and the position in the position space. As a result, even if there are variations in the shape of the detection workpiece W or a plurality of detection workpieces W are randomly stacked, it is possible to ensure good robustness against such a state and to detect the detection workpiece more accurately. The position and posture of W can be obtained.
Furthermore, the amount of calculation (calculation amount) by the calculation device 50 can be reduced, and the position and orientation of the detection target can be detected satisfactorily even if the cost of the computer constituting the calculation device 50 is suppressed. .
In addition, since the object detection device can determine the position and orientation of the object at high speed and accurately, the robot system can be applied to a robot system that grips and conveys the work W that is the object. Thus, the robot system 1 with high work efficiency can be obtained.
In addition, the distance sensor 83 detects the distance data for the entire upper area of the work stocker 84, and detects the position and orientation of the work W based on the obtained distance data. Therefore, the distance data is stored in the work stocker 84. One of the workpieces W gripped by the robot system 1 is selected from the plurality of workpieces W, the position and posture of the workpiece W are detected, and the end effector 82 of the robot arm 81 is appropriately approached to the workpiece W. The workpiece W can be gripped satisfactorily.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. The second embodiment is different from the first embodiment only in the shape of the workpiece W and the procedure of the object detection arithmetic device 50, and the description of the parts configured similarly to the first embodiment will be omitted.
As shown in FIG. 7, the planar portions 12 a to 12 c of the workpiece W are extracted by the procedure from Step T <b> 100 to Step T <b> 102. In addition, about the procedure of step T100-T102, it is the same as that of step S100-S102 of 1st Embodiment.
Then, Step T103 is executed to select 12c having the largest area from the plane portions 12a to 12c, and the plane 13 is calculated. The calculation procedure of the plane 13 is the same as step S103 in the first embodiment.
Then, as shown in FIG. 8, in step T104, the distance data is projected perpendicularly onto the plane 13, and at this time, the distance to the plane is obtained from each distance data of the work W, and this is converted into a gray value to obtain a distance image. Is generated.
FIG. 9A is a diagram showing a distance image according to the present embodiment. Here, the gray value on the distance image is determined in proportion to the distance from each pixel of the distance data to the plane 13.
That is, the projected projection plane portion 14a is displayed in a pale color because the distance from the plane portion 12a to the plane 13 is long, and the projection plane portion 14c is displayed in a dark color because the distance to the plane 13 is short.
Further, the projection plane portion 14b is inclined with respect to the plane 13, and the distance to the plane 13 changes. Therefore, the projection plane portion 14b is displayed by changing from a dark color to a light color according to the distance to the plane 13. Therefore, the flat surface portion 14b gradually changes from a dark color to a light color according to the inclination of the flat surface portion 12b from the flat surface portion 14a to the flat surface portion 14C side.
In step T105, a local virtual surface is generated for each minute region in the distance image generated in step T104, its normal direction is calculated, and an angle formed with the normal direction of the plane portion 13 is calculated. Then, the gray value is determined in proportion to the angle formed with the normal direction of the plane portion 13, and a normal angle image is generated. In order to easily obtain the normal angle, the distance image can be differentiated in the X direction and the Y direction, and the intensity of the change can be obtained to substitute for the normal angle.
FIG. 9B shows an example of a normal angle image. Since the projected plane portions 14a and 14c form a plane parallel to the plane 13, the distance image is dark because there is no change in density and the angle to the normal direction of the plane 13 is almost 0 degrees (degree). Displayed in color. Since the projected plane part 14b is inclined with respect to the plane 13, it is displayed in a lighter color corresponding to the inclination than the plane parts 14a and 14c on the normal angle image.
In step T106, the created distance image and normal angle image are registered in the database as a reference distance image and a reference normal angle image, respectively. At this time, the equations of the plane 13 are associated with each other and stored in the database.
Next, a procedure for detecting the workpiece W in the workpiece stocker 84 will be described.
The distance sensor 83 extracts a plane part from the distance data obtained by executing the above steps T101 to T103 on the work W in the work stocker 84, and selects an arbitrary plane part from the plurality of extracted plane parts. A plane is selected by selection (distance data acquisition step, plane extraction step, plane selection step).
When selecting the plane part, the one having a large area of the plane part or the one on the top (that is, closer to the distance sensor 83) is selected with priority. This is effective when the workpiece W is gripped by a robot after detecting an object that can select a workpiece W with little overlap between the workpieces W or a plane on the workpiece W stacked on the upper side.
In step T104, the distance data is projected vertically on the calculated plane to create a distance image.
In step T105, an angle formed between each calculated point image and the normal direction of the plane 13 is obtained, and a normal angle image is generated by plotting the angle as a gray value.
Next, Step T107 is executed. In step T107, the reference distance image registered in the database in advance and the distance image calculated for the detection target (that is, the workpiece W in the workpiece stocker 83) are compared, and the position space is voted.
Next, Step T108 is executed. In step T108, the reference normal angle image registered in the database in advance and the normal angle image calculated with respect to the detection target (that is, the work W in the work stocker 83) are compared, and the position space is voted.
The voting procedure will be described. A point P0 (x0, y0) on the edge of the reference distance image registered in the database is associated with an arbitrary point P1 (x1, y1) in the distance image. That is, only the gray value of each pixel on the distance image and the gray value on the normal angle image are made to correspond to each other, and the difference in position is voted on the position space (x1-x0, y1-y0). Do (plot). This voting is repeated for each point of the gray image of the actual scene.
In step T109, the reference distance image and the reference normal angle image are rotated every predetermined angle.
Then, the processing of steps T107 and T108 is executed every time step T109 is executed, and voting is repeatedly performed to another position space each time.
Note that the process of step T109 is repeatedly executed until the cumulative rotation amount of the reference distance image and the reference normal angle image reaches a predetermined angle (360 degrees in this case).
Next, in step 110, the location where voting is most concentrated in steps T107 to T109 is searched from the position space corresponding to the angle of each reference image, and the angle of the reference image corresponding to the position space and the concentration of voting. Find the position where you were.
Since the respective equations of the plane of the workpiece W placed at a known position and the plane of the workpiece W during actual work are obtained in step T103, the posture of the plane portion of the workpiece W during actual work is obtained. be able to.
Further, since the position and rotation amount of the workpiece W on the plane can be known by obtaining the most concentrated angle and position of the vote, the position and posture of the workpiece W in the three-dimensional space are obtained (step 110). .
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and can be appropriately modified and applied without departing from the gist of the present invention.
In the first embodiment and the second embodiment described above, the distance data in the entire area of the work stocker 84 is detected by the distance sensor 83. However, a partial area of the upper surface of the work stocker 84 is calculated as an arithmetic unit. The position and orientation of the object may be detected based on the distance data obtained by detecting the distance data for only a partial area of the work stocker 84 in advance.
By configuring in this way, the amount of calculation can be reduced as compared with the case of using the distance data for the entire area of the work stocker 84.

2a to 2c, 12a to 12c Plane portions 3a to 3c Edge portions 4, 13 Plane portions 5a to 5c, 14a to 14c Edge portions 50 projected onto the plane Object detection device 51 Plane portion / edge extraction portion 52 Plane selection portion 53 Contrast edge Image creation unit 54 Reference edge image database 55 Position / attitude calculation unit 60 Display unit 70 Robot control device 81 Robot arm 82 End effector 83 Distance sensor 84 Stocker 100 Robot system W Workpiece (object)

Claims (15)

An object detection method for detecting the position and orientation of an object in a three-dimensional space,
A distance data acquisition step for obtaining distance data by measuring a distance from each position on the object;
A plane / edge extraction step of extracting a plane portion and an edge portion of the object in the distance data;
A plane selection step of selecting an arbitrary plane from the extracted plurality of plane portions;
An edge image that creates an edge image in which the area that is projected perpendicularly from the edge portion to the selected plane and surrounded by the edge portion is represented by shading corresponding to the distance from the edge portion to the selected plane. Creation steps,
Each of the points on the reference edge image having the same gray value as the point on the edge image by comparing the edge image with the reference edge image calculated in advance and stored as a database in the object at a known position A voting step for sequentially voting the difference in the position in the position space to find the most voting place,
A position / posture calculation step of obtaining the position and posture of the object from the angle of the reference edge image most voted and the position in the position space, sequentially repeating the voting step by rotating the reference edge image; The object detection method characterized by having. The plane / edge extraction step includes:
Calculating an average curvature based on the distance data obtained in the distance data obtaining step;
The object detection method according to claim 1, further comprising: extracting, as the planar portion, a region obtained by collecting places where the absolute value of the average curvature is smaller than a predetermined value. 2. The object detection method according to claim 1, wherein, in the plane selection step, a plane having the largest area is preferentially selected from the plane portions extracted in the plane / edge extraction step. An object detection method for detecting the position and orientation of an object in a three-dimensional space,
A distance data acquisition step for obtaining distance data by measuring a distance from each position on the object;
A plane extraction step of extracting a plane portion of the object in the distance data;
A plane selection step of selecting an arbitrary plane from the extracted plurality of plane portions;
A distance image creating step for projecting the distance data perpendicularly to the selected plane to create a distance image representing the distance between the object and the selected plane in shades;
A normal direction of a local virtual surface is calculated for each minute region in the distance image created in the distance image creation step, and an angle formed by the calculated normal direction and the plane is expressed by shading. A normal angle image creating step for creating a line angle image;
A reference distance image calculated with an object at a known position in advance and stored as a database is compared with the distance image created in the distance image creation step, and points having the same gray value are associated with each other. A distance image voting step for sequentially voting the difference to the position space;
A point having the same gray value is obtained by comparing the reference normal angle image calculated in advance with an object at a known position and stored as a database with the normal angle image created in the normal angle image creation step. Normal angle image voting step for sequentially voting to the position space, associating the difference between the positions;
The distance image voting step and the normal angle image voting result of step, and voting determining a location to gather the most votes in the position space,
The repeated voting step sequentially by rotating the reference range image and the reference normal angle images, the position and orientation of the object from the angle of the most votes of gathered the reference range image and a position on the position space And a position / orientation calculation step to be obtained. The plane extraction step includes:
Calculating an average curvature based on the distance data obtained in the distance data obtaining step;
5. The object detection method according to claim 4, further comprising the step of extracting, as the planar portion, an area obtained by collecting places where the absolute value of the average curvature is smaller than a predetermined value. 5. The object detection method according to claim 4, wherein, in the plane selection step, a plane having the largest area is preferentially selected from the plane portions extracted in the plane extraction step. An object detection device for detecting a position and orientation of a target object in a three-dimensional original space,
A distance sensor for measuring distance from each position on the object and obtaining distance data;
An arithmetic device that performs an operation based on a measurement value of the distance sensor, and the arithmetic device,
A plane / edge extraction unit for extracting a plane part and an edge part of the object in the distance data measured by the distance sensor;
A plane selection unit for selecting an arbitrary plane from the extracted plurality of plane portions;
An edge image that creates an edge image in which the area that is projected perpendicularly from the edge portion to the selected plane and surrounded by the edge portion is represented by shading corresponding to the distance from the edge portion to the selected plane. The creation department;
An edge image database that is calculated for the object in a known position in advance and stores a reference edge image as a reference;
The point on the reference edge image having the same gray value as the point on the edge image by comparing the edge image created by the edge image creation unit with the reference edge image stored in the reference edge image database A voting unit that finds the most voting part by sequentially voting the difference in position on the position space,
An object detection unit for rotating the reference edge image and sequentially repeating the operation of the voting unit, and obtaining the position and orientation of the object from the angle of the reference edge image that is most voted and the position in the position space. An object detection device characterized by comprising: The plane / edge extraction unit includes:
An average curvature is calculated based on the distance data obtained by the distance sensor , and an area obtained by collecting places where the absolute value of the average curvature is smaller than a predetermined value is extracted as the planar portion. The object detection apparatus according to claim 7, wherein the object detection apparatus is characterized. The object detection device according to claim 7, wherein the plane selection unit preferentially selects a plane having the largest area from the plane portions extracted by the plane / edge extraction unit. An object detection device for detecting a position and orientation of a target object in a three-dimensional space,
A distance sensor for measuring distance from each position on the object and obtaining distance data;
An arithmetic device that performs an operation based on a measurement value of the distance sensor, and the arithmetic device,
A plane extraction unit for extracting a plane portion of the object in the distance data;
A plane selection unit for selecting an arbitrary plane from the extracted plurality of plane portions;
A distance image creating unit for projecting the distance data perpendicularly to the selected plane to create a distance image representing the distance between the object and the selected plane in shades;
A normal direction of a local virtual surface is calculated for each minute region in the distance image created in the distance image creation step, and an angle formed by the calculated normal direction and the plane is expressed by shading. A normal angle image creation unit for creating a line angle image;
A reference image database that is calculated in advance for the object at a known position and stores a reference distance image as a reference;
A reference normal angle image database that is calculated in advance for the object at a known position and stores a reference normal angle image serving as a reference;
A distance image voting unit that compares the reference distance image and the distance image created by the distance image creation unit and associates each point with the same gray value, and sequentially votes the position difference in the position space;
A method in which the reference normal angle image and the normal angle image created by the normal image creation unit are compared, points having the same gray value are associated with each other, and a difference between the positions is sequentially voted in the position space A line angle image voting section;
As a result of each vote of the distance image voting unit and the normal angle image voting unit, a voting unit that obtains the most voting place in the position space;
The operation for obtaining the location in the voting unit is sequentially repeated by rotating the reference distance image and the reference normal angle image, and the object is determined based on the angle of the reference distance image most voted and the position in the position space. An object detection apparatus comprising: a position / posture calculation unit that obtains the position and posture of an object. The plane extraction unit
An average curvature is calculated based on the distance data obtained by the distance sensor , and an area obtained by collecting places where the absolute value of the average curvature is smaller than a predetermined value is extracted as the planar portion. The object detection apparatus according to claim 10, wherein the object detection apparatus is characterized. The object detection device according to claim 10, wherein the plane selection unit preferentially selects a plane having the largest area from the plane portions extracted by the plane extraction unit. A robot system for gripping an object,
A transport robot for gripping and transporting the object;
The object detection device according to any one of claims 7 to 12,
And a robot control device that controls a movement trajectory of the transfer robot relative to the object based on an output of the object detection device. A work stocker for storing the object in a random posture;
The robot system according to claim 13, wherein the distance sensor of the object detection device acquires the distance data for the entire area of the work stocker. A work stocker for storing the object in a random posture;
14. The robot system according to claim 13, wherein the distance sensor of the object detection device acquires the distance data for a predetermined partial area set in advance among all areas of the work stocker.

Images