JP7264247B2 - Information processing device and information processing method - Google Patents

Description

The present invention relates to an information processing device and an information processing method.

Conventionally, in inspection and picking in FA (Factory Automation), as a method of recognizing (detecting) an object from an image, a model of the object based on 3D CAD data is used to recognize the position and orientation of the object, and then the gripping position by the robot hand. A technique for estimating is proposed. If the object is a rigid body such as a bolt or nut, the shape of the object is fixed. Therefore, the position and orientation of the object loaded in bulk in a container are recognized by the object model based on 3D CAD data, and then the robot hand grasps it. Perform position estimation.

On the other hand, a technique has also been proposed for recognizing a grippable position of an object from three-dimensional measurement data of the object and the shape of the robot hand without using such an object model. Such a technique is effective even when the object is a rigid body such as a bolt or nut, but it is not suitable for a non-rigid body such as a flexible cable or an irregularly shaped object such as a liquid detergent package. It is also effective for an object for which a model of the object based on 3D CAD data cannot be used, as in the case of .

As a technique for recognizing a grippable position of an object from three-dimensional measurement data of the object, there is a technique described in Patent Document 1, for example. Here, a three-dimensional point set composed of adjacent points is created from the three-dimensional point group, and each set is based on a predetermined size criterion, a predetermined area criterion, and a predetermined length criterion. We are extracting a set of possible 3D points.

Further, as a technique for recognizing a grippable position of an object from the three-dimensional measurement data of the object and the shape of the robot hand, there is a technique described in Patent Document 2, for example. Here, grippable features are extracted from the three-dimensional measurement data of the object according to the shape of the robot hand. Then, this feature is matched with the shape model of the robot hand, and a matching score is given according to the degree of matching. determines the ranking. For a suction-type robot hand, surfaces larger than a certain area are extracted as features.

However, as described above, for a suction-type robot hand, a standard such as a predetermined area is a criterion for determining the admissibility or superiority of suction-holding. When recognizing the grippable position of the object from the data and the shape of the robot hand, it is necessary for the user to adjust the threshold parameters for determining whether or not the predetermined criteria are met. In addition, such parameter adjustment requires knowledge and experience in gripping bulk objects, and the parameters need to be adjusted each time the object to be gripped changes, which also imposes a burden on the user.

JP 2017-090317 A Japanese Patent Application Laid-Open No. 2006-275608

SUMMARY OF THE INVENTION It is an object of the present invention to provide a technology capable of reducing the user's burden when recognizing a position where a target object can be sucked and gripped.

The present invention for solving the above problems is
a parameter setting unit for setting a parameter for evaluating an evaluation index, which is an index for evaluating a position of sucking and holding a target object, based on the size of a suction pad provided on a suction hand that sucks and holds the target object;
a planar region extraction unit that extracts a planar region based on three-dimensional measurement data obtained by three-dimensionally measuring the target object;
a suction-holding likelihood calculation unit that calculates a suction-holding likelihood indicating ease of suction-holding based on an evaluation result of the evaluation index for the extracted planar region;
a suction grip position/orientation calculation unit that calculates a position/orientation of the suction hand with respect to the suction grip position on the target object;
The information processing apparatus further comprises a grasping priority order setting unit that sets a priority order of the sucking and grasping positions based on the sucking and grasping likelihood and the grasping position and orientation of the sucking hand.

A parameter for evaluating an evaluation index often depends on the area of the planar region of the target object. For this reason, assuming that the area of such a planar region has a relationship similar to the size of the suction pad, in the present invention, the parameter for evaluating the evaluation index used for calculating the suction-holding likelihood is set to the target object is set based on the size of the suction pad provided on the suction hand that suctions and grips. The size of the suction pad can be determined appropriately even by a user with little knowledge or experience regarding grasping position recognition by actually measuring the size of the suction pad used for suction grasping or by referring to the specifications. can be set. The suction grip likelihood is calculated by evaluating the evaluation index using the set parameters for the planar area extracted from the three-dimensional measurement data of the target object. Therefore, according to the present invention, it is possible to reduce the user's burden when recognizing the position where the target object can be sucked and gripped.

Moreover, in the present invention,
The parameter based on the size of the suction pad may be a value determined based on the diameter of the suction pad.

According to this, when the suction pad is circular, a parameter that appropriately reflects the size of the suction pad can be set according to the diameter of the suction pad.

Further, in the present invention, the parameter based on the size of the suction pad may be a value determined based on the length of the short side of a rectangle that circumscribes the suction pad.

According to this, when the suction pad has a shape other than a circle, such as an oval, or when there are a plurality of suction pads, the size of the suction pad can be appropriately determined according to the length of the short side of the rectangle circumscribing the suction pad. You can set the parameters that are reflected in the

Further, in the present invention, the evaluation index may include a distance from the center of gravity of the plane area.

The closer the suction gripping position is to the center of gravity of the flat area, the less the deviation due to the weight of the target object is and the easier it is to suck and grip. Ease of use can be appropriately evaluated.

Further, in the present invention, the evaluation index may include a distance from the non-planar area that is not the planar area.

Even in cases where it is difficult to evaluate the easiness of suction and grip appropriately based on the distance from the center of gravity of the flat area, such as when the target object is ring-shaped with a hole, the evaluation index is based on the distance from the non-flat area. By including it, it is possible to appropriately evaluate the easiness of suction gripping.

Moreover, in the present invention, the evaluation index may include the area of the planar region.

The larger the planar area of the target object, the simpler the shape is likely to be, and the easier it is to suck and grip. Also, when a plurality of target objects are piled up in bulk, if the area of the planar region is large, it is unlikely that other target objects will be superimposed on top of each other. Therefore, if the evaluation index includes the area of the plane region, it is possible to appropriately evaluate the easiness of suction gripping.

In addition, the present invention
a step of setting a parameter for evaluating an evaluation index, which is an index for evaluating a position of sucking and holding a target object, based on the size of a suction pad provided on a suction hand that sucks and holds the target object;
a step of extracting a planar region based on three-dimensional measurement data obtained by three-dimensionally measuring a scene including a plurality of target objects;
a step of calculating a suction-holding likelihood indicating ease of suction-holding based on the evaluation result of the evaluation index for the extracted planar region;
calculating the position and orientation of the suction hand with respect to the suction gripping position on the target object;
setting a priority order of the suction gripping positions based on the suction gripping likelihood and the gripping position and orientation of the suction hand;
It is an information processing method including

A parameter for evaluating an evaluation index often depends on the area of the planar region of the target object. For this reason, assuming that the area of such a planar region has a relationship similar to the size of the suction pad, in the present invention, the parameter for evaluating the evaluation index used for calculating the suction-holding likelihood is set to the target object is set based on the size of the suction pad provided on the suction hand that suctions and grips. The size of the suction pad can be determined appropriately even by a user with little knowledge or experience regarding grasping position recognition by actually measuring the size of the suction pad used for suction grasping or by referring to the specifications. can be set. The suction grip likelihood is calculated by evaluating the evaluation index using the set parameters for the planar area extracted from the three-dimensional measurement data of the target object. Therefore, according to the present invention, it is possible to reduce the user's burden when recognizing the position where the target object can be sucked and gripped.

ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to reduce a user's burden at the time of recognizing the position which can be suction-grasped of a target object.

FIG. 1 is a diagram showing an example of the overall configuration of a gripping position recognition device. FIG. 2 is a functional block diagram of the information processing device. FIG. 3 is a flowchart showing an example of grip recognition processing. FIG. 4 is a schematic diagram of a suction pad. FIG. 5 is a diagram showing a tray on which target objects are piled in bulk. FIG. 6 is a diagram showing an example of segment extraction. FIG. 7 is a flowchart showing an example of an adsorption likelihood calculation process. 8A and 8B are diagrams showing the distance from the center of gravity of the segment at the suction gripping position, and FIG. 8C is a diagram showing an example of the suction gripping position. FIG. 9A is a schematic diagram showing the relationship between pixels and segments, and FIG. 9B is a schematic diagram showing an example of classification of suction gripping positions according to the distance of the center of gravity of the segment. FIGS. 10A and 10B are diagrams showing the distance between the suction gripping position and the non-flat area, and FIG. 10C is a diagram showing an example of the suction gripping position. 11A, 11B and 11C are explanatory diagrams of the advantages of classification according to the distance from the non-planar area of the suction grip position. FIG. 12 is a diagram showing an example of classification of suction gripping positions according to the distance to the non-flat area. 13A and 13B are schematic diagrams showing areas of segments, and FIG. 13C is a diagram showing an example of suction gripping positions. FIGS. A, B, and C are diagrams showing examples of classification of suction gripping positions according to the area of segments. FIG. 15 is a diagram showing a calculation example of the suction-holding likelihood. FIG. 16 is a diagram schematically showing calculation of the gripping position and orientation of the suction hand. 17A and 17B are diagrams showing examples of calculation of the priority of gripping position candidates of the suction hand. FIG. 18 is a diagram showing an example of the priority order of grip position candidates of the suction hand. 19A, 19B and 19C are diagrams showing other forms of suction pads. 20A and 20B are diagrams showing parameters for other forms of suction pads.

[Example of application]
Hereinafter, application examples of the present invention will be described with reference to the drawings.
The present invention is applied to an information processing device 21 included in a gripping position recognition device 2 whose overall configuration is shown in FIG. In the gripping position recognition device 2, the sensor unit 20 performs three-dimensional measurement of the plurality of target objects 29 bulk-stacked on the tray 28, and the information processing device 21 performs three-dimensional measurement of the target objects 29 based on the three-dimensional measurement data of the plurality of target objects 29. According to the suction gripping likelihood indicating the easiness of suction gripping, the suction gripping positions are prioritized and output to the PLC 25 . The PLC 25 controls the robot hand 27 based on this priority order, and sucks and grips the target object 29 with the suction pad 26 .
The information processing device 21 comprehensively calculates the suction gripping likelihood when the target object 29 is sucked and gripped by the robot hand 27 having the suction pad 26 attached thereto as described above, from a plurality of suction gripping indices. Here, the suction-gripping likelihood is an index indicating the ease with which a target object can be suction-grasped, and is generally called suction-gripping possibility. The easiness of sucking and gripping a target object means the degree of possibility that the target object can be properly gripped by the suction pad when the target object is sucked by the suction hand having the suction pad. That is, there is a high possibility that the suction pad can reliably suck and hold the target object in a manner necessary for a desired action of the suction hand, such as transporting the target object to another place. Conversely, the ease with which the target object can be sucked and gripped means the low possibility of failure in sucking and gripping the target object. For example, the target object cannot be lifted because the suction pad does not properly adsorb the target object, or even though the target object was once adsorbed by the suction pad, it interferes with other target objects or obstacles and causes it to fall off. Also, there is a low possibility that the target object will be unbalanced and dropped, resulting in failure in suction gripping.

In the present invention, focusing on the fact that parameter adjustment for evaluation by the suction gripping index often depends on the area of the planar region of the target object, the area of the planar region of the target object is determined by the size of the suction pad 26. We assumed a similar relationship. Based on this assumption, the diameter, which is an example of the size of the suction pad 26, is set by the user's designation or the like, thereby automatically setting the optimum parameters. Then, the parameters set in this manner are used to evaluate a plurality of suction-grip indices. Further, the suction-holding likelihood is calculated based on the plurality of suction-holding indices. As a result, it is possible to reduce the user's burden of parameter adjustment, calculate the suction-grip likelihood suitable for the size of the target object, and determine the optimum suction-gripping position for suction.

As for the size of the suction pad 26, when it is circular like a bellows type suction pad 261 (see FIG. 4) or a standard type suction pad 262 (see FIG. 19A), the diameter is increased as shown in FIG. can be adopted. However, not only the diameter but also the size of the suction pad 26 can be reflected in accordance with the shape of the suction pad 26, and a value that can be easily obtained and set by the user by actual measurement or the like can be appropriately adopted for parameter setting. For example, in the case of an oblong suction pad 263 (see FIG. 19B), the length of the short side of a rectangle circumscribing the oval that is the outer shape of the suction pad 263 can be adopted (see FIG. 20A), When the suction pad 264 has a plurality of suction pads (see FIG. 19C), the length of the short side of the rectangle circumscribing the plurality of suction pads can be adopted (see FIG. 20B). In these cases, the length of the circumscribing rectangle other than the short side may be a value that reflects the size of the suction pad 26, and other lengths such as the long side and the diagonal line may be used.

<Embodiment>
(Overall Configuration of Holding Position Recognition Device)
A gripping position recognition device according to an embodiment of the present invention will be described with reference to FIG.

The gripping position recognition device 2 is installed in a production line that assembles and processes articles, and recognizes the object 29 loaded on the tray 28 based on the data taken in from the sensor unit 20 and the data on the size of the suction pad 26 . This is a system for recognizing the sucked gripping position/orientation of the robot hand 27 . Objects to be recognized (hereinafter also referred to as “target objects”) 29 are randomly stacked on the tray 28 . Hereinafter, the robot hand 27 with the suction pad 26 attached thereto is referred to as a "suction hand".

The gripped position recognition device 2 is roughly composed of a sensor unit 20 and an information processing device 21 . A wired or wireless connection is established between the sensor unit 20 and the information processing device 21 , and the output of the sensor unit 20 is taken into the information processing device 21 . The information processing device 21 is a device that performs various processes using data captured from the sensor unit 20 . The processing of the information processing device 21 may include, for example, distance measurement (distance measurement), three-dimensional shape recognition, object recognition, scene recognition, and the like. The recognition result of the gripping position recognition device 2 is output to a PLC (Programmable Logic Controller) 25, a display 22, or the like, for example. The recognition result is used for controlling the robot hand 27, for example.

(sensor unit)
The sensor unit 20 has at least a camera for taking an optical image of the target object 29 . Furthermore, the sensor unit 20 may include components (sensors, illumination devices, light projection devices, etc.) necessary for three-dimensional measurement of the target object 29 . For example, when the depth distance is measured by stereo matching (also called stereo vision, stereo camera method, etc.), the sensor unit 20 is provided with a plurality of cameras. In the case of active stereo, the sensor unit 20 is further provided with a light projection device for projecting structured light onto the target object 29 in a random dot pattern. When three-dimensional measurement is performed by the space-encoded pattern projection method, the sensor unit 20 is provided with a light projection device for projecting pattern light and a camera. In addition, any method such as a photometric stereo method, a TOF (time of flight) method, a phase shift method, or the like may be used as long as it is a method capable of acquiring three-dimensional information of the target object 29 .

(Information processing device)
The information processing device 21 is configured by a computer including, for example, a CPU (processor), RAM (memory), nonvolatile storage device (hard disk, SSD, etc.), input device, output device, and the like. In this case, the CPU develops a program stored in the nonvolatile storage device in the RAM and executes the program, thereby realizing various configurations described later. However, the configuration of the information processing device 21 is not limited to this, and all or part of the configuration described later may be realized by a dedicated circuit such as FPGA or ASIC, or by cloud computing or distributed computing. You may

FIG. 2 is a functional block diagram showing the configuration of the information processing device 21. As shown in FIG. The information processing device 21 calculates a suction gripping likelihood representing the easiness of suction gripping of the target object from the three-dimensional measurement data of the target object by the sensor unit 20 and the data related to the size of the suction pad 26. The priority of the gripping position of the target object is set based on the suction gripping likelihood. The information processing device 21 has a parameter setting unit 211 , a segment extraction unit 212 , a suction-grip likelihood calculation unit 213 , a grip position/orientation calculation unit 214 , and a grip priority order setting unit 215 . Here, the segment extractor 212 corresponds to the "plane area extractor" of the present invention.

(Grip position recognition processing)
An example of gripping position recognition processing, which is an information processing method by the information processing device 21, will be described with reference to the flowchart of FIG.

In step S101, the three-dimensional measurement data of the target object based on the image three-dimensionally measured by the sensor unit 20 is obtained (step S101). Three-dimensional measurement data is data composed of a plurality of points, each of which has three-dimensional information. The three-dimensional data may be of any format, for example, data in which each point has a three-dimensional coordinate value in the camera coordinate system, or a depth value (depth Data (distance image) in a format associated with distance information) may also be used.

ここで、ｆは焦点距離（カメラパラメータ）であり、Ｚはカメラ（光学中心）からの距離である。Ｚは画像の各画素により値が変化するが、全てのセグメントに対して同じ閾値パラメータを使用するために、Ｚを画像内の最小距離値に設定する。In step S<b>102 , the parameter setting unit 211 calculates the parameters of the suction grip index. Here, the suction grip index corresponds to the "evaluation index" in the present invention. FIG. 4 shows the general shape of a bellows-type suction pad 261 provided in the suction hand 27 . The diameter of the suction pad 261, that is, the diameter of the substantially circular suction portion 2611 at the tip of the suction pad 261 is φ [mm]. Here, the size of the suction pad 261 is reflected by the diameter. For the diameter of the suction portion 2611, the user may input the measured value of the suction pad 261 used for suction, or the 3D CAD data of the suction pad 26 stored in the predetermined area 215 of the nonvolatile storage device may be used. You may use the data of models, such as. Also, the diameter of the suction portion 2611 may be obtained from three-dimensional measurement data obtained by imaging the suction pad 261 attached to the suction hand 27 by the sensor unit 20 . Then, the parameter setting section 211 converts the diameter φ [mm] of the adsorption section 2611 into the size D [pixel] on the image using the camera parameters. Conversion from φ [mm] to D [pixel] is performed by the following formula.
[Number 1]

where f is the focal length (camera parameter) and Z is the distance from the camera (optical center). Z varies in value for each pixel in the image, but to use the same threshold parameter for all segments, set Z to the minimum distance value in the image.

In step S103, the segment extraction unit 212 calculates an object plane region (segment) from the three-dimensional measurement data of the target object. Here, normal maps around the X-axis and the Y-axis are calculated with respect to the X-axis and the Y-axis (parallel to the paper surface) in the camera coordinate system from the range image of the target object. Then, the variation of the normal map is calculated, and a region with a small variation is extracted as a plane region (segment). As shown in FIG. 5, an example will be described in which packages 291 of confectionery, etc., which are irregular-shaped target objects, are bulk-stacked on the tray 30. FIG. A plurality of target objects bulk-stacked on the tray 30 in this way are three-dimensionally measured by the sensor unit 20 . Then, a distance image is acquired as three-dimensional measurement data. A normal map is created from the distance image, and the variation of the normal around the X-axis and Y-axis is calculated with respect to the X-axis and Y-axis parallel to the paper surface. When the target objects are piled in bulk in the manner shown in FIG. A region Ar1 corresponding to is calculated as a region in which the variation of the normal around the X-axis and around the Y-axis is small in the normal map. In addition, an area Ar2 corresponding to the side surfaces of the two target objects 292 and 293, which intersect the X-axis and the Y-axis at an angle close to perpendicular to each other and are arranged such that the package sides face upward (toward the side of the paper), is In the normal map, it is calculated as an area where the normals around the X-axis and around the Y-axis vary greatly. FIG. 6 schematically shows plane regions (segments) extracted in this way, which are regions with small variations in the normal map (for reference, the visible outline of the package is also shown). Areas R1 to R8 hatched in FIG. 6 are planar areas.
Note that the method of extracting a planar region is not limited to the method using the normal map as described above. For example, a method of grouping adjacent point groups in a three-dimensional point group, or a method of detecting edges in directions perpendicular to the X-axis and Y-axis using distance values as thresholds can be employed.

Next, the processing from step S105 to step S112 is repeated for each segment (step S104, step S113).

(Suction grip likelihood calculation processing)
In step S105, the suction-grip likelihood calculation unit 213 calculates the suction-grip likelihood for each pixel in the segment from a plurality of suction-grip indices. An example of suction-holding likelihood calculation processing will be described with reference to FIG. 7 .

In step S201, the suction-grip-likelihood calculation unit 213 classifies the suction-grip indices into A, B, and C in order of ease of grasping, according to the distance to the center of gravity of the segment. 8A and 8B are diagrams schematically showing the distance from the center of gravity of the segment. In FIG. 8A, the suction grip position Sp11 is close to the center of gravity C11 of the segment Sg11. In contrast, in FIG. 8B, the suction grip position Sp12 is far from the center of gravity C11 of the segment Sg11. The closer the suction gripping position is to the center of gravity of the segment, the easier it is to grip because there is less deviation due to the weight of the target object. For example, as shown in FIG. 8C, an area Ar3 surrounded by a dotted line circle of a confectionery package 291 bulk-stacked on a tray is close to the center of gravity of the segment, and is a position suitable for suction gripping. is far from the centroid of the segment and is not suitable for suction gripping.

なお、Ｋ_ｃ ^（Ａ），Ｋ_ｃ ^（Ｂ）は定数であり、ユーザではなく、開発者又は評価者が予め実験的に決定する。この例では、Ｋ_ｃ ^（Ａ）＝Ｋ_ｃ ^（Ｂ）＝１．０とする。A specific description will be given of a method of classifying the suction grip index into A, B, and C in order of ease of gripping according to the distance to the center of gravity of the segment.
Let _distc be the distance to the segment center of gravity, and _thc ^(A) and _thc ^(B) be the threshold parameters of the index based on the distance to the segment center of gravity.
At this time, the suction grip index is classified into A, B, and C according to the distance to the center of gravity of the segment according to the following formula.
[Number 2]

Note that K _c ^(A) and K _c ^(B) are constants, which are experimentally determined in advance by the developer or evaluator, not by the user. In this example, K _c ^(A) = K _c ^(B) = 1.0.

An example of classification according to the method described above is shown in FIGS. 9A and 9B. 9A and 9B are diagrams schematically showing the relationship between pixels and segments, where one square corresponds to one pixel. In FIG. 9A, the shaded pixels are the regions extracted as the segment Sg21, and the cross indicates the position of the center of gravity C21 of the segment Sg21. Also, the non-planar region Np21 is indicated by white pixels. FIG. 9B shows the classification result when D=3 [pixel] for the segment Sg21 shown in FIG. 9A . With the center of gravity C21 of the segment Sg21 as the center, the inner circle Cr12A corresponds to the threshold parameter thc ₍ ^A) , and the outer circle Cr21B corresponds to _{the threshold parameter thc} ( ^B) . Pixels included in the region Sg21A of the segment are classified as A because the distance from the center of gravity C21 is less than the threshold parameter th _c ^(A) . Pixels included in the region Sg21B of the segment are classified _as B because the distance from the center of gravity C21 is greater than or equal to the threshold parameter thc ^(A) _and smaller than thc ^(B) . Pixels included in the region Sg21C of the segment _are classified as C because the distance from the center of gravity C21 is equal to or greater than the threshold parameter thc ^(B) . In this way, using the threshold parameter, the suction-grip indices can be classified into A, B, and C in order of ease of grasping according to the distance to the center of gravity of the segment.

In step S<b>202 , the suction-grip likelihood calculation unit 213 classifies the suction-grip indices into A, B, and C in order of ease of grasping, according to the distance from the non-flat area. 10A and 10B are diagrams schematically showing the distance from the non-planar area. In FIG. 10A, the suction gripping position Sp13 in the segment Sg12 is far from the non-planar region Np12. On the other hand, in FIG. 10B, the suction gripping position Sp14 is close to the non-planar region Np12. The farther the suction gripping position is from the non-planar region, the less the risk of gripping near the boundary of the object. For example, as shown in FIG. 10C, an area Ar 3 surrounded by a dotted line circle of a confectionery package 291 bulk-stacked on a tray is far from the non-flat area, and is a position suitable for suction gripping. A region Ar 4 surrounded by a dotted line circle 292 is a position not suitable for suction gripping because the distance from the non-flat region is short. The difference between the suction grip index based on the distance from the non-flat area and the suction grip index based on the distance from the centroid of the segment will be described for a ring-shaped target object 293 with a hole in the center as shown in FIG. 11A. For such a target object 293, the center of gravity C12 of the segment will be positioned at the center of the hole. Therefore, if the suction-grip index is based on the distance from the center of gravity of the segment, the suction-grip index is higher than the area Ar5 of the hole opened in the center, as shown in FIG. 11B. On the other hand, if the suction gripping index is based on the distance from the non-flat area, the suction gripping index is low in the hole region, and is the center of the thickness in the radial direction, and the suction of the ring-shaped region Ar6 on the target object. The grasping index becomes higher. Thus, for a ring-shaped target object with a hole, the suction gripping index based on the distance from the non-flat area is effective.

なお、Ｋ_ｐ ^（Ａ），Ｋ_ｐ ^（Ｂ）は定数であり、ユーザではなく、開発者又は評価者が予め実験的に決定する。この例では、Ｋ_ｐ ^（Ａ）＝１．５、Ｋ_ｐ ^（Ｂ）＝１．０とする。A method of classifying the suction grip index into A, B, and C in order of ease of gripping according to the distance from the non-flat area will be specifically described.
Let dist _p be the distance from the non-flat area, and th _p ^(A) and th _p ^(B) be the threshold parameters of the index based on the distance from the non-flat area.
At this time, the suction grip index is classified into A, B, and C according to the distance from the non-flat area according to the following formula.
[Number 3]

Note that K _p ^(A) and K _p ^(B) are constants, which are experimentally determined in advance by the developer or evaluator, not by the user. In this example, K _p ^(A) =1.5 and K _p ^(B) =1.0.

An example of classification according to the method described above is shown in FIG. The segment shape is the same as in FIG. 9A. FIG. 12 is also a diagram schematically showing the relationship between pixels and segments, and one square corresponds to one pixel. Again, the classification result when D=3 [pixels] is shown. Pixels included in the segment Sg22A are classified as A because the distance from the non-planar region Np21 is greater than the threshold parameter th _p ^(A) . The pixels included in the segment Sg22B are classified _as B because the distance from the non-planar region Np21 is greater than the threshold parameter thp ^(B) _and less than or equal to thp ^(A) . Then, since the distance from the non-planar region Np21 of the pixels included in the segment Sg22C is equal to or less than the threshold parameter th _p ^(B) , the segment is classified as C. In this way, using the threshold parameter, the suction-grip index can be classified into A, B, and C in order of ease of gripping according to the distance from the non-flat area.

In step S<b>203 , the suction-grip likelihood calculation unit 213 classifies the suction-grip indices into A, B, and C in order of ease of grasping according to the segment area. 13A and 13B are diagrams schematically showing the areas of segments. In FIG. 13A, the area of segment Sg13 is large, and in FIG. 13B, the area of segment Sg14 is small. The larger the area of the segment, the more likely that the target object has a simpler shape and is easier to grasp. In addition, when the area of the segment is large in the target object in bulk, it is less likely that other target objects are superimposed on it. On the other hand, if the area of the segment is small, the shape of the target object is likely to be complicated, making it difficult to grasp. In addition, if the area of the segment is small in the target object in bulk, there is a high possibility that other objects overlap. For example, as shown in FIG. 13C, an area Ar5 surrounded by a dotted line of a confectionery package 291 bulk-stacked on a tray is a position suitable for suction gripping because the area of the segment is large. On the other hand, in FIG. 13C, a region Ar6 surrounded by a dotted line of a confectionery package 294 in which a plurality of packages are overlaid and only the end of the back face is exposed is not suitable for suction gripping because the area of the segment is small. position.

なお、Ｋ_ｓ ^（Ａ），Ｋ_ｓ ^（Ｂ）は定数であり、ユーザではなく、開発者又は評価者が予め実験的に決定する。この例では、Ｋ_ｓ ^（Ａ）＝６．０、Ｋ_ｓ ^（Ｂ）＝３．０とする。A specific description will be given of a method of classifying the suction grip index into A, B, and C in order of ease of gripping according to the area of the segment.
Let size be the area of the segment, and th _s ^(A) and th _s ^(B) be the threshold parameters of the index based on the area of the segment.
At this time, the suction grip index is classified into A, B, and C according to the area of the segment according to the following formula.
[Number 4]

Note that K _s ^(A) and K _s ^(B) are constants, which are experimentally determined in advance by the developer or evaluator, not by the user. In this example, K _s ^(A) =6.0 and K _s ^(B) =3.0.

Examples of classification according to the method described above are shown in FIGS. 14A, 14B, and 14C. Segment Sg23A shown in FIG. 14A is classified as A because the area of the segment is greater than the threshold parameter th _s ^(A) . The segment Sg23 shown in FIG. 14B is classified as B because the area of the segment is greater than the threshold parameter th _s ^(B) and equal to or less than th _s ^(A) . Segment Sg23C shown in FIG. 14C is classified as C because the area of the segment is equal to or less than the threshold parameter th _s ^(B) . In this way, using the threshold parameter, the suction-grip index can be classified into A, B, and C in order of ease of gripping according to the area of the segment.

In step S204, the suction-grip likelihood is calculated by integrating the plurality of suction-grip indices described in steps S201 to S203. Here, the suction-gripping likelihood is calculated by integrating the three suction-gripping indices, but the number of suction-gripping indices is not limited to this. Further, the method for calculating the suction-grip index described above is not limited to the method described above.
FIG. 15 shows a calculation example of the suction-holding likelihood. Here, according to the classification result of the suction-grasping index, the suction-grasping likelihood is classified into seven stages of A, B1 to B3, and C1 to C3 in order from easier to grasp to harder to grasp.

That is, when the number of A in the suction-grip index satisfies the condition of 3 (the number of B and C is 0), the suction-grip likelihood is classified as A. For example, when all of the distance to the segment center of gravity, the distance from the non-flat region, and the segment area are classified as A, the suction-grip likelihood is classified as A.

When the suction-grip index satisfies the condition that the number of A is 2 and the number of B is 1 (the number of C is 0), the suction-grip likelihood is classified as B1. For example, when both the distance to the segment center of gravity and the distance from the non-flat area are classified as A, and the segment area is classified as B, the suction-grip likelihood is classified as B1. If the suction-grip index satisfies the condition that the number of A's is 1 and the number of B's is 2 (the number of C's is 0), the suction-grip likelihood is classified as B2. For example, when the distance to the segment center of gravity is A, and the distance to the non-planar region and the segment area are B, the suction gripping likelihood is classified as B2. When the condition that the number of B in the suction-grip index is 3 (the number of A and C is 0) is satisfied, the suction-grip likelihood is classified as B3. For example, when all of the distance to the segment center of gravity, the distance from the non-flat region, and the segment area are classified as B, the suction-grip likelihood is classified as B3.

When the condition that the number of Cs in the suction-grip index is 1 (the number of A or B is 2) is satisfied, the suction-grip likelihood is classified as C1. For example, when the distance to the segment center of gravity is B, the distance to the non-flat area is A, and the segment area is C, the suction gripping likelihood is classified as C1. When the number of Cs in the suction-grip index satisfies the condition of 2 (the number of A or B is 1), the suction-grip likelihood is classified as C2. For example, when the distance to the segment center of gravity is A, and both the distance to the non-planar region and the segment area are C, the suction-grip likelihood is classified as C2. When the condition that the number of C in the suction-grip index is 3 (the number of A and B is 0) is satisfied, the suction-grip likelihood is classified as C3. For example, when all of the distance to the segment center of gravity, the distance from the non-flat area, and the segment area are classified as C, the suction-grip likelihood is classified as C3.

Returning to the flowchart of FIG. 3, the description of the gripping position recognition processing is continued.
In step S106, the suction-grip likelihood calculation unit 213 rearranges the pixels in the segment according to the suction-grip likelihood calculated in step S105. The rearrangement of each pixel in the segment according to the suction-grip likelihood may be performed by the grip position/orientation calculation unit 214 based on the suction-grip likelihood calculated by the suction-grip likelihood calculation unit 213 .

Next, the processing of steps S108 and S109 is repeated for each pixel in the segment, starting from the pixel with the highest suction-holding likelihood (steps S107 and S112).

In step S108, the gripping position/orientation calculation unit 214 calculates the gripping position/orientation of the suction hand 27 for each pixel. As schematically shown in FIG. 16, a translation vector can be obtained from the gripping position coordinates (x, y) of the suction pad 261 and the distance z between the target object 29 and the camera <sensor unit> 20, and the gripping position coordinates ( A rotation matrix can be calculated from the x-direction and y-direction angles of the normal vector Nv in x, y). The position and orientation of the suction hand 27 at the suction gripping position are calculated from these translation vectors and rotation matrix. Then, the approach angle of the suction hand 27 along the direction perpendicular to the surface of the target object 29 is calculated by using the angle of the normal vector Nv at the suction grip position of the target object 29 .

In step S109, it is determined whether or not the grasping position and orientation of the suction hand 27 calculated in step S108 interfere with an obstacle or a three-dimensional point group. A well-known method can be adopted for interference determination, so detailed description is omitted.

If it is determined in step S109 that there is no interference, the process advances to step S111 to register the gripping position/orientation as a representative gripping position within the segment in a predetermined area of the nonvolatile storage device.
If it is determined in step S109 that there is interference, the processing of steps S108 and S109 is performed for the next pixel with the highest suction-holding likelihood (steps S107 and S110).

In step S111, the processing of steps S108 and S109 is completed for the pixels in the segment to be processed, and the representative grip position is registered.

When the processing from step S104 to step S112 is completed for the segment extracted in step S103, the process proceeds to step S113.

In step S113, the gripping priority setting unit 215 calculates the gripping priority of the registered gripping positions. That is, gripping position candidates are detected from each segment, and the priority among these gripping position candidates is calculated. Here, for example, as schematically shown in FIGS. 17A and 17B, using an index such as the distance value Z from the sensor unit to the segment of the target object 29, the distance value from the sensor unit to the segment is small ( 17A) has a higher priority, and a larger distance value from the sensor unit to the segment (FIG. 17B) has a lower priority. As shown in FIG. 18, in the example in which the confectionery packages 29 shown in FIG. 5 are stacked on the tray in bulk, the priority is lower in the order of gripping position candidates P1, P2, P3, P4, P5, P6, P7, and P8. Then, the priority of each gripping position candidate is calculated.
The method of prioritizing gripping positions is not limited to this. For example, the priority may be given according to the distance from the non-flat area, the distance from the center of gravity of the segment, the variation in the normal to the suction gripping position, the area of the segment, the approach angle of the suction hand to the target object, and the like.

The information on the gripping priority of the gripping positions calculated by the gripping priority setting unit 215 is output to the PLC 25 together with the information on the gripping position and orientation. is done.

(Advantages of this embodiment)
In the configuration and processing described above, when the user designates the diameter of the suction pad to be used, the threshold parameter in suction-holding likelihood calculation, which indicates the ease of suction-holding, is automatically calculated from the diameter of the suction pad. As a result, the burden of parameter adjustment on the user can be reduced .

(Modification)
In the above-described embodiment, the suction-grip index discretely classifies the easiness of grasping into three stages of A, B, and C, but the calculation method of the suction-grip index is not limited to this, and continuously changes. may be defined by a function that For example, each suction-grip index can be defined as a continuous function as follows. Also here, the pad diameter is D [pixel], and the pad radius is R [pixel].
Let _distc be the distance to the segment center of gravity, and let _Kc be a constant determined experimentally in advance by the designer or evaluator (in this example, _Kc = 3.0).
At this time, the suction gripping index f _c (dist _c ) based on the distance to the center of gravity of the segment is calculated by the following formula.
[Number 5]

Let dist _p be the distance from the non-flat area, and K _p be a constant experimentally determined in advance by the designer or evaluator (in this example, K _p =6.0).
At this time, the suction-grip index f _p (dist _p ) based on the distance from the non-flat area is calculated by the following formula.
[Number 6]

Let the area of the segment be size, and the constants determined experimentally in advance by the designer or evaluator are K _s ^(α) and K _s ^(β) (in this example, K _s ^(α) = 1.05, K _s ^(β) = 3.0).
At this time, the suction-grip index fs(size) based on the segment area is calculated by the following formula.
[Number 7]

The suction-grip index function described above is based on a sigmoid function, and outputs a numerical value in the range of 0.0 to 1.0.
[Number 8]

ここで、各ｎ乗根の数値は、ｎ_ｃ＞０、ｎ_ｐ＞０、ｎ_ｓ＞０で設定し、影響度を抑制したい指標にはｎを大きな値に設定する。The function of the suction grip index is not limited to the one described above, and can be output as an arbitrary function. etc. may be applied to correct within the range of 0.0 to 1.0.
Furthermore, the degree of influence of each suction-grip index may be suppressed by the n-th root of ( _nc , _np , _ns ). For example, the suction-holding likelihood g may be calculated by the following formula.
[Number 9]

Here, the numerical value of each n-th root is set such that _nc > 0, n _p > 0, and _ns > 0, and n is set to a large value for an index whose degree of influence is desired to be suppressed.

Further, in the above-described embodiment, the suction pad 261 is a circular bellows type suction pad, but the type and number of the suction pads 261 are not limited to this. As shown in FIG. 19A, a standard type suction pad 262 may be used. Further, an oblong type suction pad 263 may be used as shown in FIG. 19B, or a multi-pad type suction pad 264 may be used as shown in FIG. 19C, and the number of multi-pad type suction pads is not limited to four. In such an oval type suction pad 263, as shown in FIG. 20A, the length of the short side of the circumscribing rectangle Rt1 of the suction portion 2631 is regarded as the diameter φ of the suction pad 263, and the suction grip likelihood parameter is automatically calculated. can do. As shown in FIG. 20B, the length of the short side of a circumscribing rectangle Rt2 circumscribing the multi-pad type suction units 2641, 2641, 2641, 2641 is regarded as the diameter of the suction pad 264, and similarly the suction gripping likelihood parameters can be automatically calculated.

In the above-described embodiment, so-called modelless gripping position recognition that does not use a model such as 3D CAD data of the target object 29 has been described. Parameter settings can be applied.

In order to allow comparison between the constituent elements of the present invention and the configurations of the embodiments, the constituent elements of the present invention will be described below with reference numerals in the drawings.
<Invention 1>
Evaluate an evaluation index for evaluating a suction gripping position of a target object (29) based on the size of a suction pad (26) provided on a suction hand (27) for suction gripping the target object (29). a parameter setting unit (211) for setting parameters for
a planar area extraction unit (212) for extracting a planar area based on three-dimensional measurement data obtained by three-dimensionally measuring the target object (29);
a suction-holding likelihood calculation unit (213) that calculates a suction-holding likelihood indicating ease of suction-holding based on the evaluation result of the evaluation index for the extracted planar region;
a suction gripping position/orientation calculation unit (214) that calculates the position/posture of the suction hand with respect to the suction gripping position on the target object;
An information processing apparatus comprising: a grasping priority order setting unit (215) for setting a priority order of the sucking and grasping positions based on the sucking and grasping likelihood and the grasping position/orientation of the sucking hand (27). (21).

21: Information processing device 26: Suction pad 27: Suction hand 211: Parameter setting unit 212: Segment extraction unit 213: Suction grip likelihood calculation unit 214: Suction grip position/orientation calculation unit 215: Grip priority setting units 261, 262, 263, 264: suction pads

Claims (7)

a parameter setting unit that sets a threshold parameter based on the size of a suction pad provided on a suction hand that suction-holds a target object;
a planar region extraction unit that extracts a planar region based on three-dimensional measurement data obtained by three-dimensionally measuring the target object;
For the extracted planar region, the suction-grip index, which is an evaluation index of the suction-gripping position with respect to the target object, is classified based on the threshold parameter, and based on the classification results for the plurality of suction-gripping indices, the suction-gripping is performed. a suction-holding-likelihood calculation unit that calculates a suction-holding-likelihood that indicates ease of
a suction gripping position/posture calculation unit that calculates a gripping position/posture of the suction hand with respect to the suction gripping position on the target object in relation to the calculated suction gripping likelihood ;
An information processing apparatus, comprising: a gripping priority order setting unit that sets a priority order of the gripping positions and orientations for a plurality of the target objects . 2. The information processing apparatus according to claim 1, wherein the threshold parameter based on the size of the suction pad is a value determined based on the diameter of the suction pad. 2. The information processing apparatus according to claim 1, wherein said threshold parameter based on the size of said suction pad is a value determined based on the length of a short side of a rectangle circumscribing said suction pad. 4. The information processing apparatus according to claim 1, wherein said evaluation index includes a distance from a center of gravity of said plane area. 5. The information processing apparatus according to any one of claims 1 to 4, wherein the evaluation index includes a distance from the non-planar area that is not the planar area. 6. The information processing apparatus according to any one of claims 1 to 5, wherein the evaluation index includes an area of the planar region. setting a threshold parameter based on the size of a suction pad provided on a suction hand that suction-holds a target object;
a step of extracting a planar region based on three-dimensional measurement data obtained by three-dimensionally measuring a scene including a plurality of target objects;
For the extracted planar region, the suction-grip index, which is an evaluation index of the suction-gripping position with respect to the target object, is classified based on the threshold parameter, and based on the classification results for the plurality of suction-gripping indices, the suction-gripping is performed. a step of calculating an adsorption-grip likelihood indicating ease of
calculating a gripping position and orientation of the suction hand with respect to the suction gripping position on the target object in relation to the calculated suction- gripping likelihood;
setting a priority order of the gripping positions and orientations for the plurality of target objects ;
Information processing method including.

Images