JP7779277B2 - Picking method and picking device - Google Patents

Description

The present invention relates to a picking method and a picking device for picking bulk materials.

For example, in the plate refining process, test pieces are cut out from the plate to inspect the quality of the product. The cut test pieces are temporarily stored in bulk on buckets, etc., and when a certain number of test pieces are collected, they are transported to the inspection site. Conventionally, the bulk test pieces are manually picked and aligned at the inspection site.
However, picking and arranging these bulk-stacked test specimens requires heavy physical labor, and this work is fraught with dangers, such as the risk of falling due to poor footing and pinching of fingers or other parts of the body by the heavy test specimens.

One method for automatically picking bulk-stacked components is described in Non-Patent Document 1, for example. In Non-Patent Document 1, the surface shape of bulk-stacked components of a single shape is measured using a 3D shape measurement sensor. Next, Non-Patent Document 1 recognizes the position and orientation of individual components by comparing the measured surface shape with a 3D CAD model of the component prepared in advance. Then, Non-Patent Document 1 calculates a gripping point from the recognized information, and the component is picked using the robot's gripping unit (end effector).

Patent Document 1 also discloses a method for recognizing the position and orientation of an object. In this method, feature points are detected on a range image and then matched with feature points of a model prepared in advance. By performing this matching process, Patent Document 1 recognizes the position and orientation of the object.

IHI Technical Report Vol. 48 No. 1 "Development of a Bin Picking System Using 3D Object Recognition Technology" (March 2008)

Japanese Patent Application Laid-Open No. 2001-143073

However, the method in Non-Patent Document 1 targets components with a single, predetermined shape and automatically generates gripping points by matching measured 3D point cloud data with a shape model. For this reason, it is not suitable as a method for automatically picking test pieces with diverse and undefined dimensions.

Furthermore, in the method of Patent Document 1, when detecting feature points on a range image, edge extraction processing is performed using a Gaussian Laplacian filter, and then the edge portions are calculated. The intersections of each edge are then considered to be feature points of the object. However, edge extraction processing using a Laplacian filter or the like is highly dependent on the image capture conditions. For this reason, there is a concern that the method of Patent Document 1 may not be able to extract edges reliably. Furthermore, with the method of Patent Document 1, edge extraction is difficult due to differences in hue, for example, when components of the same color overlap. Furthermore, the method of Patent Document 1 detects the target object by comparing the extracted feature points with a pre-prepared object model. For this reason, the method of Patent Document 1 requires a model of the object to be picked. Therefore, the method of Patent Document 1 is not suitable as a method for picking objects whose dimensions are not yet determined.

The present invention was developed with the above points in mind, and one of its objectives is to accurately estimate the position and orientation of individual bulk-stacked components, even if the dimensions of some of the components are unknown. Another objective of the present invention is to enable accurate picking of components through this estimation.

To solve this problem, one aspect of the present invention is a picking method for picking bulk materials, in which a 3D measurement sensor is used to measure the area where the materials are placed from above to obtain a 3D point cloud, and a computer processes the detected feature values from the obtained 3D point cloud to detect surface shape features that define the surface of the materials.The computer processes the detected feature values to estimate at least the position of the materials' position and orientation, and grasps the materials based on the estimation.

According to this aspect of the present invention, a three-dimensional point cloud of the surface of a bulk pile of components is acquired, and feature quantities defining characteristic regions of the component's surface are detected from the three-dimensional point cloud. Therefore, according to this aspect of the present invention, even if the dimensions of some of the bulk piled components are unknown, the position and orientation of each component can be estimated individually. As a result, according to this aspect of the present invention, components can be picked with high accuracy even if the dimensions of the components have not been determined.

1 is a schematic diagram illustrating a picking device according to an embodiment of the present invention. FIG. 10 is a schematic diagram illustrating another picking device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the configuration of a calculation unit. 1A and 1B are diagrams illustrating a method for obtaining three-dimensional point cloud data and feature amounts (feature regions). FIG. 10 is a diagram illustrating an example of a method for obtaining a feature amount (feature region). FIG. 10 is a diagram illustrating an example of a hypothesized model formula. FIG. 10 is a diagram illustrating an inlier. FIG. 10 is a diagram illustrating an example of clustering. FIG. 2 is a diagram illustrating the configuration of a grip control unit. FIG. 1 is a schematic diagram illustrating the configuration of an apparatus according to an embodiment. FIG. 2 is a top view of bulk-stacked steel plates in an embodiment. FIG. 10 is a diagram showing the detected steel plate together with its center of gravity.

Next, an embodiment of the present invention will be described with reference to the drawings.
The picking device of this embodiment automatically picks up multiple bulk-stacked components (blocks) one by one using a gripping unit (end effector), and then transports the components to another location or re-stacks the components.

In this embodiment, a plate-shaped test piece, particularly a metal plate material such as steel, aluminum, or titanium, is used as an example of the member. That is, a plate-shaped member is used as an example of the member. Also, the member is exemplified as a plate-shaped member with a flat upper surface. However, it is assumed that the dimensions of each of the bulk-stacked members are different. For example, the surface shape (contour shape) of the upper surface of the member is not the same. Of course, the present invention is also applicable to members of the same shape.
Furthermore, "bulk stacked" refers to components being placed on a placement surface in a state where the orientation of each component is random and there is a possibility that the components may overlap each other.
The surface on which the members are placed may be flat or may have spaces in part, such as a lattice pattern.

(composition)
As shown in Fig. 1, the picking device of this embodiment includes a conveying device 3 equipped with a gripping unit 4 (end effector), a three-dimensional measurement sensor 2, a calculation unit 10, and a gripping control unit 11. Although Fig. 1 illustrates an example in which a plurality of members 1 are stacked in bulk on a floor surface 20, the members 1 may also be stacked in bulk on a pallet or the like.

<Conveyance device 3>
The transport device 3 is a device that grips the member 1 with a gripping portion 4 and moves (transports) the member 1. In Fig. 1, the transport device 3 is illustrated as being composed of a vertical articulated robot arm.

[Gripping part 4]
1, the gripper 4 is detachably attached to the tip of the robot arm. In this embodiment, the gripper 4 can be attached and detached to the tip of the arm with a single touch, such as by rotating it. In other words, the gripper 4 can be automatically changed by controlling the transport device 3.
The gripping unit 4 is, for example, a suction gripper, a magnetic gripper, a gripper having a workpiece such as a plurality of claws, etc. The gripping unit 4 may be a known end effector capable of gripping the target member 1.
1, reference numeral 6 indicates an example of a spare gripping unit. The spare gripping unit 6 is a gripping unit that changes depending on the mass of the member 1, and is placed within the movable range of the arm.
In this embodiment, the gripping unit 4 pre-installed on the transport device 3 is a suction gripper, and the spare gripping unit 6 is a gripping gripper having claws.

[Thickness measurement sensor 5]
Furthermore, a plate thickness measurement sensor 5 is attached to the gripping portion 4 of this embodiment. The plate thickness measurement sensor 5 is a known plate thickness measurement sensor such as an ultrasonic sensor, and is a sensor that measures the plate thickness, which is the dimension in the vertical direction of the member 1. The plate thickness measurement sensor 5 may be provided separately from the conveying device 3.
2 is an example in which the main body of the conveying device 3 is composed of a gate-shaped palletizer 7. The gripping unit 4 is detachably attached to the tip of the tool part of the palletizer 7, and the gripping unit 4 moves along the palletizer 7.

<3D survey sensor 2>
The three-dimensional measurement sensor 2 measures from above an area where a plurality of components 1 are placed (referred to as a placement area). The three-dimensional measurement sensor 2 is a sensor for acquiring a three-dimensional point cloud of the surfaces (mainly the upper surfaces) of the bulk-stacked components 1 as viewed from above. The three-dimensional measurement sensor 2 can be a known three-dimensional measurement sensor such as a stereo camera, a light-section method, or an optical time-of-flight method.
1 illustrates a case where the three-dimensional surveying sensor 2 is placed above the placement area and supported by a base (not shown). Also, FIG. 2 illustrates a case where the three-dimensional surveying sensor 2 is supported by a tool unit of a palletizer 7, and moves synchronously with the movement of the tool unit.

<Calculation unit 10>
3, the calculation unit 10 includes a three-dimensional point cloud data acquisition unit 10A, a feature amount detection unit 10B, and a member identification unit 10C. The calculation unit 10 and the grip control unit 11 are processed by a computer and stored as programs in a storage unit, and each program is called to execute processing.

<Three-dimensional point cloud data acquisition unit 10A>
The three-dimensional point cloud data acquisition unit 10A stores in a storage unit three-dimensional point cloud data obtained when the placement area is surveyed from above based on the signal from the three-dimensional surveying sensor 2.
The interval between the surveyed points may be set according to the target member 1. For example, information on each point is acquired at 1 mm intervals. Each point has three-dimensional position information (x, y, z) relative to a predetermined origin coordinate. The three-dimensional point cloud data is a collection of points having such three-dimensional position information.

In this embodiment, a three-dimensional point cloud of the surface of two or more bulk-stacked components 1 as seen from above is acquired, as shown in Fig. 4(a). Fig. 4(b) is a schematic diagram of the acquired three-dimensional point cloud information 8(P).
The surveyed three-dimensional point cloud data also includes point cloud data of the placement surface, such as the floor surface 20. Therefore, point cloud data whose height (z coordinate) is equal to or less than a predetermined value is deleted to obtain a set of three-dimensional point cloud data of only the member 1. The predetermined value is, for example, a value obtained by adding a safety margin (for example, 2 mm) to the height position (z coordinate) of the placement surface.

<Feature Amount Detection Unit 10B>
The feature amount detection unit 10B detects, from the stored three-dimensional point cloud data, a feature area of the surface shape that defines the surface of the component 1. Here, since the measurement is performed from above, the top surface of the component 1 is the surface that defines the component 1, but if the component 1 is tilted, the point cloud of the side surface of the component 1 is also measured.
In this embodiment, a flat plate-shaped member 1 is exemplified. Therefore, the surface shape defining the surface of the member 1 is treated as a plane, and feature amounts defining the feature regions of the surface shape are detected.
Even if the surface of the member 1 is flat, it has a certain degree of roughness depending on the material of the member 1. A flat surface with irregularities due to the roughness is called a plane.

[Feature detection method]
In this embodiment, the shape of the surface to be defined is set as a plane, and therefore the feature amount is detected using the plane equation "ax+by+cz+d=0."
First, the coefficients a, b, and c are adjusted as variables from the stored three-dimensional point group data to find a set of points Pi that matches the equation (model formula) of an arbitrary plane.
4(c) and 4(d) show an example in which a planar region 9 (Pi), which is an example of a feature region, is calculated from the three-dimensional point cloud information 8 (P) using an algorithm for detecting planar portions. The point cloud Pi that constitutes this planar region 9 (Pi) constitutes the feature quantity that defines the plane. Note that a known algorithm such as the RANSAC algorithm may be used as the algorithm for detecting planar portions.

An example of processing using the RANSAC algorithm will be described with reference to FIG.
First, in step S1, several points are randomly extracted from the three-dimensional point cloud data as shown in FIG. 6, and a model formula is assumed in which the extracted several points are included.
The model formula is a formula that defines the surface shape. For example, a model formula such as y = ax + b for a straight line or ax + by + cz + d = 0 for a flat surface is assumed. Note that a, b, c, and d in the formula are variables that define the model formula.

Next, in step S2, as shown in Fig. 7, data points whose distance from the assumed model formula is equal to or less than a threshold value are determined to be inliers (data that are not outliers) of the assumed model formula, taking into consideration that the surface of the member 1 is not an ideal flat surface.
Next, in step S3, it is determined whether the number of points belonging to the assumed model formula is equal to or greater than a preset threshold. If the condition is satisfied, the process proceeds to step S4. On the other hand, if the condition is not satisfied, that is, if the number of points belonging to the assumed model formula is small, the process proceeds to step S1, where the assumed model formula is changed, and the processes of steps S1 and S2 are repeated.

Here, if the assumed model formula is correct, there should be a large number of inliers. In other words, if the assumed model formula has a small number of points, it may not be a model formula that defines the surface of component 1. The threshold value is set, for example, based on the length of the smallest side of the top surface of the target component 1 and the distance between the measured points.

In step S4, the assumed model formula (planar region) is determined to be correct, and clustering is performed on the set Pi of points belonging to the model formula. An example of clustering is shown in FIG.
In this embodiment, a set of points whose inter-point distance is smaller than a preset threshold is defined as one cluster. Figure 4(d) shows an example of three clusters, A, B, and C.
Next, in step S5, the cluster (set) with the largest number of points among the plurality of clusters A, B, and C is recognized as the feature amount of the plane Pi, plane defined by the assumed model formula.

For example, the planar region 9 (Pi) of points contained in the calculated plane portion is reclassified using a clustering algorithm to remove noise and eliminate sets of points extracted from regions where components 1 intersect. This calculates a set 10 (Pi, plane) of points containing only components 1 that match the calculated plane. In Figure 4(d), the group of points contained in set A becomes the feature quantity that defines the plane defined by the assumed model formula. In other words, the calculated set 10 (Pi, plane) becomes information on the feature region that defines the surface of one component 1.

Here, the DBSCAN algorithm used for clustering in this embodiment will be described in detail. Specifically, the DBSCAN algorithm is one of the techniques for dividing a point cloud into sets A, B, and C of neighboring points, as shown in Fig. 8(a). Clustering of the point cloud is performed by repeating the following steps (1) to (3) as shown in Fig. 8(b) until the cluster no longer expands or until a specified number of times (Fig. 8(c)).
(1) First, an arbitrary point is selected, and points within a radius r are searched for.
(2) If there are a certain number of points or more within the radius r, it is recognized as a cluster.
(3) For all points recognized as clusters, the cluster is expanded by searching for points within the radius r again in step (1).
In this embodiment, DBSCAN is used as the clustering algorithm, but k-means, HDBSCAN, etc. may also be used.

Next, in step S6, the point group of the obtained set 10 (Pi, plane) is deleted from the stored three-dimensional point group data.
Next, in step S7, if the number of stored three-dimensional points is greater than a preset threshold, the process proceeds to step S1, where a process is executed to calculate a feature area defining the surface of the other member 1.
On the other hand, if it is determined that the stored three-dimensional point cloud is equal to or smaller than the preset threshold, the process ends.
By repeating the above-described process, it is possible to detect the characteristic regions and characteristic quantities of the surface shape that defines the surface of one or more members 1 from the acquired three-dimensional point cloud.

<Component Identification Unit 10C>
The component identification unit 10C obtains identification information for identifying (estimating) at least the position of at least one component 1, based on the characteristic region detected by the feature detection unit 10B. It is preferable to obtain identification information for both the position and the orientation. However, if the mounting portion of the gripping unit 4, which is made of a suction gripper, is swingably attached to the tip of the robot arm, it is possible to grip the component 1 with only the position information.
Furthermore, since the information on the characteristic region (feature amount) is information that defines the surface of the component 1, it is possible to estimate the position and orientation (tilt, etc.) of the target surface.
The information of the characteristic region includes information on the top surface and information on the side surface of the component 1. Information with a tilt equal to or smaller than a predetermined value may be determined as information on the top surface of the component 1, and information with a tilt equal to or larger than the predetermined value may be determined as information on the side surface.

<Gripping control unit 11>
The grip control unit 11 moves the gripping unit 4 based on the identification information obtained by the member identification unit 10C to grip the member 1 to be picked, and then transports the gripped member 1 to a predetermined position.
9, the grip control unit 11 of this embodiment includes a control unit main body 11A, a center of gravity detection unit 11B, a mass estimation unit 11C, and a grip unit selection unit 11D. The control unit main body 11A executes basic control of the operation of the transport device 3.

Here, if there are multiple components 1 stacked in bulk, information on the features (feature areas) of the multiple components 1 is obtained. From this feature information, the component 1 to be grasped and picked is selected. There are no particular limitations on the method for selecting components 1. For example, based on the feature information, priority is given to selecting components that meet certain conditions, such as components with a relatively high height, components with a low inclination of the top surface (close to horizontal), and components with a top surface shape that is close to rectangular.

In this embodiment, the gripping unit 4 is a suction gripper, so the gripping control unit 11 moves the gripping unit 4 to the top surface of the member 1 to be picked. Next, the gripping unit 4 sucks and grips the top surface of the target member 1, and then controls the transportation of the gripped member 1. This control can be performed using a known control method.

[Center of gravity detection unit 11B]
The grip control unit 11 of this embodiment includes a center-of-gravity detection unit 11B. The center-of-gravity detection unit 11B estimates the position and shape of the top surface of the member 1 from the detected feature amounts, and identifies (estimates) the position of the center of gravity of the member 1 as viewed from above.
The center of gravity is detected by calculating the center of gravity position in the planar region of the target member 1 and the normal vector of the plane based on specific information that specifies (estimates) the top surface position and orientation of the member 1. Here, the center of gravity position in the planar region is determined by the arithmetic mean value of each component of the coordinates x, y, and z for all points belonging to the calculated plane.

The grip control unit 11 then moves the gripping unit 4 to the upper surface of the target member 1 so that the center position of the grip (suction cup) by the gripping unit 4 coincides with or approaches the specified (estimated) position of the center of gravity, and grips the member 1. As a result, the gripping unit 4 can grip the member 1 more stably.
This method of grasping by identifying the center of gravity is particularly effective for the gripping unit 4 of the suction type or magnet type.

[Mass estimation unit 11C]
The grip control unit 11 of this embodiment includes a mass estimation unit 11C. The mass estimation unit 11C estimates the mass of the member 1 from the preset density of the member 1, the measured thickness, and the estimated shape of the upper surface.
The thickness is measured by a plate thickness measuring sensor 5 that measures the thickness, which is the dimension in the vertical direction, of the gripped member 1. The density of the member 1 is stored in advance in a storage unit.

The grip control unit 11 of this embodiment then calculates, for example, the center of gravity position in the planar region of the workpiece 1 and the normal vector of the plane using the center of gravity detection unit 11B. Thereafter, the control unit main body 11A moves the gripping unit 4 of the conveying device 3 to the center of gravity position of the planar region along the normal vector of the plane. Then, the thickness measurement sensor 5 attached to the gripping unit 4 measures the thickness of the workpiece 1. Thereafter, the mass estimating unit 11C estimates the mass of the workpiece 1 based on the thickness measured by the thickness measurement sensor 5, the area of the planar region measured by the three-dimensional survey sensor 2, and the density stored in advance.
In addition, if only the members 1 have a plate thickness within a predetermined range, there is no need to measure the plate thickness.

[Gripping part selection part 11D]
The grip control unit 11 of this embodiment includes a gripping unit selector 11D. The gripping unit selector 11D changes (selects) the gripping method depending on the mass estimated by the mass estimator 11C.
In this embodiment, the gripping unit selecting unit 11D continues picking of the component 1 by the gripping unit 4 consisting of the suction gripper when the mass of the component 1 estimated by the mass estimating unit 11C is less than a specified value.

On the other hand, the gripper selector 11D of this embodiment temporarily suspends the picking process when the mass of the component 1 is equal to or greater than a specified value. Then, the tip of the arm robot is moved to an area where a spare gripper 6 is placed, and a tool change is performed to a spare gripper 6 (such as a gripping gripper) that has a larger weight capacity than the gripper 4 consisting of a suction gripper. This operation may be performed by an operator.
Here, the selection of the gripping method is exemplified by changing the gripping part itself, but it may also be possible to change the gripping position using the same gripping part 4, etc.
It is preferable that, once picking of one component 1 is completed, the process of acquiring the feature amount (feature area) that defines the component 1 is executed again, because the picking of the component 1 may change the positions of the remaining components 1.

(Operations etc.)
By using the above device, it becomes possible to automatically pick out the bulk stacked members 1 individually.
According to this embodiment, a 3D surveying sensor 2 (3D shape measuring sensor) acquires 3D point cloud information about the surfaces of the bulk-stacked components 1 as viewed from above. Then, planar regions on the surfaces of the components 1, which are composed of a combination of planes, are identified from the 3D point cloud information. As a result, in this embodiment, the position and orientation of each component 1 can be estimated with high accuracy without detecting the edges of each component 1.

Furthermore, by calculating the center of gravity position and the normal vector of the plane for the top surface of the member 1, the gripper 4 can be attached to the center of gravity position of the top surface of the member 1 or its vicinity along the normal vector of the top surface. This allows for more stable and accurate picking of the bulk-stacked members 1.
Furthermore, by having the plate thickness measurement sensor 5, the thickness of the component 1 can be measured and the mass can be estimated when picking. This makes it possible to determine whether the component 1 can be picked using the current gripping method, and to change the tool to an appropriate gripping unit 4 depending on the mass of the component 1. As a result, in this embodiment, components 1 such as bulk steel plates can be picked stably.

(Modification)
(1) In the above embodiment, the component 1 to be picked is a plate component, but it is not limited to a plate component and may be a pipe or a spherical component.
If the component 1 is a pipe or a spherical component, the feature region of the component 1 detected from the 3D point cloud is a curved region rather than a flat region. The curved region can be calculated by changing the equation used in the detection algorithm to a model formula that represents the target curved surface.
If the component 1 is a mixture of plate-like components and pipes, the process is performed as follows: First, the feature values of the planar regions are calculated using the plane calculation equation. Then, the feature values of the curved regions are calculated using the curved surface calculation equation.

(2) In the above embodiment, the gripping center of the gripping unit 4 is attached to the center of gravity, and the gripping unit 4 grips the upper surface of the member 1 to pick it.
Depending on the type of gripper 4 (end effector), the component 1 may be picked by gripping the outer periphery of the component 1 or by inserting the claws of the gripper 4 under the component 1. In this case, it is not necessarily necessary to calculate the center of gravity of the component 1.
However, even in this case, the member 1 can be more stably grasped by controlling the gripping center to approach the center of gravity of the member 1 when gripped.

(3) In addition, in the above embodiment, the mass of the component 1 is estimated, and the type of gripper 4 (hand gripper) is selected based on the estimated mass. However, picking may be performed using only grippers 4 with a large weight capacity. In this case, it is not necessarily necessary to measure the thickness of the component 1.

(others)
The present disclosure may also have the following configuration.
(1) A picking method for picking bulk materials, comprising:
A 3D measurement sensor measures the area where the components are placed from above to obtain a 3D point cloud.
Detecting surface shape features that define the surface of the component from the acquired three-dimensional point cloud through computational processing by a computer;
estimating at least the position of the position and orientation of the member based on the detected feature amount through computational processing by a computer;
The component is grasped based on the above estimation.
Picking method.

(2) The feature amount is detected by regarding the surface shape that defines the surface of the member as a plane.
(3) estimating the position and shape of the top surface of at least one member from the detected feature amount, and estimating the position of the center of gravity of the member as viewed from above from the estimation;
The member is grasped by bringing the center of the grasp into agreement with or close to the position of the estimated center of gravity.
(4) Measure the thickness, which is the vertical dimension of the member to be gripped;
Estimating the mass of the member from the measured thickness, the density of the member, and the estimated shape of the upper surface;
The method of grasping the member is selected according to the estimated mass.

(5) A picking device that picks bulk materials with a gripper,
a three-dimensional measurement sensor that measures the area where the member is placed from above and acquires a three-dimensional point cloud;
a feature amount detection unit that detects feature amounts of a surface shape that defines the surface of the component from the three-dimensional point cloud acquired by the three-dimensional measurement sensor;
a member identifying unit that obtains identification information that identifies at least the position of the member out of the position and orientation of the member based on the feature amount detected by the feature amount detecting unit;
a gripping control unit that moves the gripping unit and controls the gripping of the member by the gripping unit based on the obtained specific information;
A picking device comprising:

(6) The feature amount detection unit detects the feature amount by treating the surface shape that defines the surface of the member as a plane.
(7) The component identification unit estimates the position and shape of the top surface of at least one component from the detected feature amount, and identifies the position of the center of gravity of the component as viewed from above from the estimated information;
The grip control section adjusts the position of the grip section so that the center of gripping by the grip section coincides with or approaches the position of the identified center of gravity, and executes gripping of the member by the grip section.

Next, an example based on this embodiment will be described.
FIG. 10 shows a schematic diagram of the picking device used in the examples.
In this embodiment, as shown in Fig. 10, a three-dimensional measurement sensor 2 that measures the shape of the component 1 by an optical cutting method is mounted on the tool part of a three-axis Cartesian robot (transport device 3). The three-dimensional measurement sensor 2 is scanned in the xy direction to obtain three-dimensional point cloud information of the surface (mainly the upper surface) of the steel plate as the bulk component 1.

The bulk-stacked members 1 were made of steel plates having thicknesses of approximately 10 mm to 25 mm and 50 mm to 500 mm. Steel plates with varying sizes were also bulk-stacked.
FIG. 11 shows the state of the steel plates stacked in bulk in this example.
FIG. 12 shows the results of classifying the measured three-dimensional point cloud information by plane detection and dividing it into sections for each detected steel plate.
The detected center of gravity positions of the upper surfaces of the steel plates are indicated by black dots in Fig. 12. As can be seen from Fig. 12, it was found that the classification of each steel plate and the detection of the center of gravity positions were generally successful for the upper surfaces of the individual steel plates.

REFERENCE SIGNS LIST 1 MEMBER 2 3D MEASUREMENT SENSOR 3 CONVEYOR 4 GRIP PART 5 PLATE THICKNESS MEASUREMENT SENSOR 6 AUXILIARY GRIP PART 10 COMPUTING PART 10A 3D POINT CLOUD DATA ACQUISITION PART 10B CHARACTERISTICS DETECTION PART 10C MEMBER IDENTIFICATION PART 11 GRIP CONTROL PART 11A CONTROL PART BODY 11B CENTER OF GRAVITY DETECTION PART 11C MASS ESTIMATION PART 11D GRIP PART SELECTION PART

Claims (4)

A picking method for picking bulk materials, comprising:
A 3D measurement sensor measures the area where the components are placed from above to obtain a 3D point cloud.
Detecting surface shape features that define the surface of the component from the acquired three-dimensional point cloud through computational processing by a computer;
estimating at least the position of the position and orientation of the member based on the detected feature amount through computational processing by a computer;
The component is grasped based on the above estimation ,
detecting the feature amount by using a surface shape defining the surface of the member as a plane;
estimating the position and shape of the top surface of at least one member from the detected feature amount;
Measure the thickness, which is the vertical dimension of the member to be gripped;
Estimating the mass of the member from the measured thickness, the density of the member, and the estimated shape of the upper surface;
Select a method of grasping the part depending on the estimated mass.
Picking method. From the estimation of the position and shape of the upper surface of the member , the position of the center of gravity of the member as viewed from above is estimated;
The center of the grip is aligned with or close to the estimated center of gravity, and the member is gripped.
A picking method according to claim 1 . A picking device that picks bulk materials with a gripper,
a three-dimensional measurement sensor that measures the area where the member is placed from above and acquires a three-dimensional point cloud;
a feature amount detection unit that detects feature amounts of a surface shape that defines the surface of the component from the three-dimensional point cloud acquired by the three-dimensional measurement sensor;
a member identifying unit that obtains identification information that identifies at least the position of the member out of the position and orientation of the member based on the feature amount detected by the feature amount detecting unit;
a gripping control unit that moves the gripping unit and controls the gripping of the member by the gripping unit based on the obtained specific information;
a plate thickness measurement sensor that measures the thickness, which is the vertical dimension of the member to be gripped;
Equipped with
the feature amount detection unit detects the feature amount by treating a surface shape that defines the surface of the member as a plane;
the component identification unit estimates a position and a shape of an upper surface of at least one component from the detected feature amount;
The grip control unit includes a mass estimation unit that estimates a mass of the member from the measured thickness, the density of the member, and the estimated shape of the upper surface, and a gripping unit selection unit that selects a gripping method in accordance with the mass estimated by the mass estimation unit.
Picking device. the member identification unit identifies the position of the center of gravity of the member as viewed from above based on information on the estimated position and shape of the upper surface of the member ;
the grip control unit adjusts the position of the grip unit so that the center of gripping by the grip unit coincides with or approaches the position of the identified center of gravity, and executes gripping of the member by the grip unit.
The picking device according to claim 3 .