JP7373700B2 - Image processing device, bin picking system, image processing method, image processing program, control method and control program - Google Patents

Description

The present invention relates to an image processing device, a bin picking system, an image processing method, and an image processing program.

For example, food that is packed into lunch boxes that are produced on a line at a factory is often supplied in bulk. Here, in the case of standardized industrial products, it is possible to recognize a single operation target by comparing it with a three-dimensional model of the product, etc., but since food products are generally amorphous, there is no standard for comparison. Difficult to configure.

Conventionally, a method for detecting fried chicken, which is an example of an irregularly shaped food, has been proposed (for example, Non-Patent Document 1). This technology uses VCCS (Voxel Cloud Connectivity Segmentation) and graph cuts to segment the area of a single individual from a bulk pile of fried chicken.

Japanese Patent Application Publication No. 2016-045088

Nobuaki Fukuchi, Koichi Hashimoto, "Object detection in bulk loading scene without using shape model," 18th System Integration Division Conference, pp. 632-637, 2017. J. Papon, A. Abramov, M. Schoeler and F. Wrgtter, "Voxel Cloud Connectivity Segmentation - Supervoxels for Point Clouds," IEEE Conference on Computer Vision and Pattern Recognition, pp. 2027-2034, 2013. Y. Y. Boykov and M. P. Jolly, "Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images," Proc. of IEEE International Conference on Computer Vision, Vol. 1, pp. 105-112, 2001. E.J. Newman, Mark and Girvan, Michelle, "Finding and Evaluating Community Structure in Networks," Physical review. E, Statistical, nonlinear, and soft matter physics, Vol. 69, pp. 026113, 2004. Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte and Etienne Lefebvre, "Fast unfolding of communities in large networks," Journal of Statistical Mechanics: Theory and Experiment, Vol. 2008, No. 10, P. 10008, 2008. Shota Hirama, Yukiko Nakagawa, Noriaki Nakagawa, Ryuichi Ueda, "Manipulation of food using force-controlled manipulators - Detection of irregularly shaped fried foods using deep learning," 17th Society of Instrument and Control Engineers System Integration Division Conference, pp. 2478 -2481, 2016.

For example, the method using VCCS and graph cuts requires many parameters such as the surface area of fried chicken to be detected and a threshold value. Here, for example, when used in a factory, if there are many parameters, there is a risk that it will not be able to cope with different environments.

An object of the present invention is to improve the versatility of a method for detecting an individual from a pile of objects.

The image processing device includes a point cloud information acquisition unit that acquires point cloud data from a distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk; The object detection unit performs clustering on a node group generated by modeling the group data, and specifies a region where one object exists.

According to a method using clustering, it is possible to identify a region where one object exists even in an irregularly shaped work. Furthermore, compared to a method using graph cuts, for example, this method does not require many parameters depending on the type of work, and can improve versatility.

Further, clustering may be performed using a community detection method that classifies nodes so as to maximize modularity, which is a predetermined evaluation function, based on distances between nodes. Specifically, clustering can be performed using a so-called community detection method in a network.

Further, the clustering may be performed using the Louvain method. In this way, the time required for processing can be shortened by repeating local optimization and aggregation.

Further, a bin picking system according to another aspect includes the above-described image processing device, a distance image generation device, and a manipulator that moves a target object present in the area specified by the target detection unit.

In this way, it is possible to realize a system that identifies irregularly shaped workpieces stacked in bulk and moves them using a manipulator.

Furthermore, one object may be an object located on the surface of a plurality of objects stacked in bulk and not hidden behind other objects with the manipulator as a reference. In this way, it is possible to specify one object that can be moved without causing the pile of bulk materials to collapse as much as possible.

Note that the contents described in the means for solving the problems can be combined as much as possible without departing from the problems and technical idea of the present invention. Further, the content of the means for solving the problem can be provided as a device such as a computer, a system including a plurality of devices, a method executed by a computer, or a program executed by a computer. Note that a recording medium holding the program may be provided.

It is possible to improve the versatility of the method of detecting an individual from a pile of objects.

FIG. 1 is a schematic side view showing an example of a bin picking system. FIG. 1 is a functional block diagram showing an example of the configuration of a bin picking system. FIG. 3 is a processing flow diagram showing an example of position information display processing. FIG. 3 is a diagram showing an example of a node group. FIG. 3 is a diagram showing an example of a node group in which edges are set. FIG. 3 is a diagram showing an example of a group of nodes classified into clusters. FIG. 3 is a diagram illustrating an example of a group of nodes to be operated. 5 is a perspective view showing an example of a group of nodes belonging to a workpiece 5 to be operated, a three-dimensional solid inscribed therein, and a center of gravity. FIG. FIG. 3 is a diagram showing an example of a node group classified by soft clustering.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<Device configuration>
FIG. 1 is a schematic side view showing an example of a bin picking system according to this embodiment. The bin picking system 1 in FIG. 1 includes an RGB-D camera 2, a manipulator 3, and a control device 4. Moreover, the bin picking system 1 picks up one workpiece 5 from a pile formed by bulk workpieces 5 such as fried chicken, and moves it to a container (not shown) such as a lunch box, for example. Serve.

The RGB-D camera 2 is a distance image generation device that includes a solid-state image sensor using CMOS or the like, and a depth sensor using an infrared projector, sensor, etc., and generates color (RGB) images and point cloud data. Outputs . A color image is raster data that includes a plurality of pixels that have information indicating a color relative to a position within the image. The point cloud data includes a plurality of points having information indicating the distance (depth) from the camera to the surface of the object to be imaged and the color of the object to be imaged.

The manipulator 3 is a robot arm with degrees of freedom in multiple axes. Manipulator 3 is controlled by signals received from control device 4 . Further, the manipulator 3 includes a gripper (two-finger gripper) 31 at the tip of the arm, and can grip and move an object. Moreover, the manipulator 3 may further include a sensor that detects whether or not it is gripping an object.

The control device 4 is a computer that controls the operation of the manipulator 3 based on information obtained from the RGB-D camera 2. The control device 4 is a general PC (Personal Computer).
It may be a server device, or it may be a microcontroller having an interface with the RGB-D camera 2 and the manipulator 3. Further, the control device 4 may be configured to be communicably connected to the RGB-D camera 2 and the manipulator 3 via a network (not shown).

<Functional configuration>
FIG. 2 is a functional block diagram showing an example of the configuration of the bin picking system. The control device 4 is a computer and includes an input/output I/F (Interface) 41, a storage device 42, a processor 43, and a bus or signal line 44.

The input/output I/F 41 includes an interface for communicably connecting to the RGB-D camera 2 according to a predetermined standard, and an interface for communicably connecting to the manipulator 3 according to a predetermined standard.

The storage device 42 includes main storage devices such as RAM (Random Access Memory) and ROM (Read Only Memory), and auxiliary storage devices (secondary storage devices) such as HDD (Hard-disk Drive), SSD (Solid State Drive), and flash memory. equipment). The main storage device temporarily stores programs read by the processor, image data exchanged with other devices, and secures a work area for the processor. The auxiliary storage device stores programs and the like executed by the processor.

The processor 43 is an arithmetic processing device such as a CPU (Central Processing Unit), and performs each process according to the present embodiment by executing a program. In the example of FIG. 2, functional blocks are shown within the processor 43. The processor 43 includes a point cloud information acquisition section 431, a graph generation section 432, an object detection section 433, and a robot control section 434. Note that the control device 4 functions as an image processing device that processes point cloud data acquired by the RGB-D camera 2 using the point cloud information acquisition unit 431, graph generation unit 432, and object detection unit 433.

The point cloud information acquisition unit 431 acquires point cloud data output by the RGB-D camera 2 via the input/output I/F 41 and stores it in the storage device 42.

The graph generation unit 432 generates nodes in a three-dimensional space using voxels based on the color, distance, brightness, etc. of point cloud data. Further, the graph generation unit 432 uses a predetermined node/edge search method to connect nodes by edges, for example, when the distance between nodes is within a predetermined range.

The target detection unit 433 performs clustering on each node. Clustering can be performed using, for example, the Louvain method. That is, the target detection unit selects the node pair with the largest difference in modularity between adjacent nodes, and integrates the selected node pair into the same cluster until the modularity no longer changes. Repeat the process of consolidating into one node. Furthermore, the object detection unit 433 identifies, among the plurality of clusters, a cluster located at the top of the bulk pile, which is not hidden by other objects with the manipulator 3 as a reference, as a work to be operated.

For example, the robot control unit 434 determines the center of gravity of the point group forming the identified cluster, determines the position of the target to be gripped by the gripper 31 of the manipulator 3, and adjusts the size at which the gripper 31 is opened so that the point group is accommodated. Determine. Further, the robot control unit 434 outputs a signal to the manipulator 3 via the input/output I/F 41 to operate the manipulator 3 to grasp the target workpiece and move it to a predetermined location.

The above components are connected via a bus or signal line 44.

<Picking process>
FIG. 3 is a processing flow diagram showing an example of the picking processing method according to the present embodiment. Note that S1 to S4, which will be described later, can be said to be an image processing method for processing point cloud data acquired by the RGB-D camera 2.

The point cloud information acquisition unit 431 continuously acquires point cloud data generated by the RGB-D camera 2 via the input/output I/F 41 (FIG. 3: S1). In this step, point cloud data of an imaging target including a plurality of works 5 and containers such as bats in which these are placed is acquired.

The graph generation unit 432 uses the point group data acquired in S1 to generate a graph including nodes arranged in a three-dimensional space and edges connecting the nodes (S2). In this step, the graph generation unit 432 randomly (without bias) selects a plurality of points from the point cloud data with sufficient spatial resolution, and generates corresponding voxels. Further, voxels for which no other voxels exist within a predetermined range or voxels whose color (hue, saturation, brightness) greatly deviates from the standard are deleted. In this way, it is possible to remove noise, generate a node group that models the imaging target, and reduce the amount of data to be processed.

FIG. 4 is a diagram showing an example of a generated node group (voxel). In the example of FIG. 4, each voxel is schematically represented by a square. Further, FIG. 4 is a plan view as seen from the RGB-D camera 2 located above the object to be imaged, and each voxel is assumed to have a depth (height) value.

Further, the graph generation unit 432 uses a predetermined node/edge search method such as a KD-tree to set edges between nodes. For example, an edge is set when the distance between nodes is within a predetermined range. Note that the search method is not particularly limited.

FIG. 5 is a diagram showing an example of a node group to which edges are set. FIG. 5 is also a plan view of the imaging target viewed from above. Further, in the example of FIG. 5, solid line edges connecting nodes whose distances are within a predetermined range are added to the node group shown in FIG. 4.

After that, the target detection unit 433 performs clustering of nodes (FIG. 3: S3). Although the clustering method is not particularly limited, so-called community detection in a network can be used. For example, boundaries may be set at positions where the difference in modularity becomes large, such as the Newman method or the Louvain method, and nodes may be clustered such that the modularity within the cluster (community) is high. In other words, based on modularity, which is a predetermined evaluation formula, if the connection between nodes is determined to be dense, two nodes are classified into the same cluster, and if the connection is determined to be sparse, the two nodes are classified into the same cluster. Two nodes can be classified into different clusters.

なお、ｍはグラフ内のすべてのエッジの重みの総和である。Ａ_ｉｊは、ノードｉとノードｊとの間のエッジの重みである。重みは、例えばノード間の距離が近いほど高くなるように設定する。ｋ_ｉ、ｋ_ｊは、それぞれのノードｉ、ノードｊの次数である。δはクロネッカーのデルタであり、δ（ｃ_ｉ，ｃ_ｊ）は、ノードｉが属するクラスタｃ_ｉ及びノードｊが属するクラスタｃ_ｊが同一である場合は１であり、同一でない場合は０である。 Generally, the modularity Q can be calculated using the following equation (1).

Note that m is the sum of weights of all edges in the graph. A _ij is the weight of the edge between node i and node j. For example, the weight is set such that the closer the distance between nodes, the higher the weight. k _i and k _j are the degrees of node i and node j, respectively. δ is Kronecker's delta, and δ(c _i , c _j ) is 1 if the cluster c _i to which node i belongs and the cluster c _j to which node j belongs are the same, and 0 otherwise. .

なお、ｍは、グラフ内のすべてのエッジの重みの総和である。Σ_ｉｎは、クラスタ内のエッジの重みの総和である。重みは、例えばノード間の距離が近いほど高くなるように設定する。Σ_ｔｏｔは、クラスタ内のノードに入るエッジの重みの総和である。ｋ_ｉは、ノー
ドｉの次数である。ｋ_ｉ，ｉｎは、ノードｉからクラスタ内のノードまでのエッジの重みの総和である。なお、重みは、ノード間の距離のほか、ＲＧＢ－Ｄカメラ２が出力する色の類似する度合い等を用いて設定するようにしてもよい。 Specifically, the target detection unit 433 clusters a group of nodes using a greedy method that integrates a plurality of nodes into a cluster so that the modularity increases, and repeats the integration until the modularity no longer increases. That is, in S3, the target detection unit 433 identifies, for each node, a node that has the largest difference (increment) in modularity when adjacent nodes are combined, and integrates the nodes. The modularity difference ΔQ can be calculated using the following equation (2).

Note that m is the sum of weights of all edges in the graph. Σ _in is the sum of the weights of the edges within the cluster. For example, the weight is set such that the closer the distance between nodes, the higher the weight. Σ _tot is the sum of the weights of the edges that fall into the nodes in the cluster. k _i is the degree of node i. k _i,in is the sum of edge weights from node i to nodes in the cluster. Note that the weight may be set using the degree of similarity of colors output by the RGB-D camera 2, etc., in addition to the distance between nodes.

Modularity difference ΔQ can be used to evaluate changes in modularity when adding a node to a cluster or deleting a node from a cluster. Therefore, it is also possible to easily calculate changes in modularity when a node once integrated into any cluster is replaced with another node. Further, when the target detection unit 433 determines that the modularity no longer changes (increases) due to node integration and replacement, it ends the clustering process in S3.

For example, by repeating local optimization and aggregation like the Louvain method, the time required for processing can be shortened.

FIG. 6 is a diagram showing an example of a group of nodes classified into clusters. FIG. 6 is also a plan view of the imaging target viewed from above. Furthermore, in the example of FIG. 6, nodes belonging to the same cluster are marked with the same hatching.

Further, the target detection unit 433 determines a cluster to be operated from among the plurality of clusters (FIG. 3: S4). In this step, the object detection unit 433 identifies the work 5 located on the surface layer of a pile formed by stacking in bulk. Specifically, first, voxels expressed in a camera viewpoint coordinate system with the RGB-D camera 2 as a reference are converted into a robot viewpoint coordinate system with a predetermined origin for the manipulator 3 as a reference. Then, in the robot viewpoint coordinate system, voxels constituting the workpiece 5 seen from the manipulator 3 that are not hidden by voxels constituting other workpieces 5 are to be operated. Therefore, a workpiece existing on the surface of a mountain, such as the top of a mountain, is selected as an operation target.

FIG. 7 is a diagram illustrating an example of a node group to be operated. FIG. 7 is also a plan view of the operation target viewed from above. Further, in the example of FIG. 7, as a group of nodes to be operated, nodes and edges belonging to a workpiece determined to exist near the center are shown by thick solid lines, and nodes and edges belonging to other works are shown by broken lines.

Further, the robot control unit 434 operates the workpiece to be operated (FIG. 3: S5). In this step, the robot control unit 434 operates the manipulator 3 via the input/output I/F 41, grips the workpiece to be operated, and moves it to a predetermined location. Specifically, the robot control unit 434 first determines the assumed center of gravity of the workpiece 5 and the size at which the gripper 31 should be opened. That is, the position of the center of gravity is calculated using a group of nodes forming the workpiece 5 specified as the operation target. Note that since the point cloud data of the imaging target output by the RGB-D camera 2 is generated based on the distance to the surface of the imaging target, the shape of the back side of the workpiece 5 is estimated to be symmetrical, for example, and the center of gravity is determined. You may ask for it. Then, the robot control unit 434 moves the gripper 31 with the center of gravity as a target. At this time, the robot control unit 434 controls the opening size of the gripper 31 so that all the node groups can be held. Note that even if the robot control unit 434 operates the manipulator 3 to obtain a three-dimensional solid such as a rectangular parallelepiped, an ellipsoid, or a sphere in which a group of nodes belonging to the workpiece 5 to be manipulated is inscribed, and grasp the three-dimensional solid. good.

FIG. 8 is a perspective view showing an example of a group of nodes belonging to the workpiece 5 to be operated, a three-dimensional solid inscribed therein, and a center of gravity. The squares in FIG. 8 are nodes. A rectangular parallelepiped indicated by a broken line is an example of a solid in which a group of nodes is inscribed. Furthermore, the black circle is the center of gravity of the node group or the solid. The two arrows indicate the direction in which the gripper 31, which is a two-finger gripper, grips the workpiece 5, for example.

The robot control unit 434 can determine that a grip has been achieved if the current value of the motor that operates the gripper 31 and the angle of the fingers for gripping are within the specified range, and if either the finger angle or the current value is not within the specified range. It can be determined that it has failed. Furthermore, the robot control unit 434 operates the manipulator 3 to move the gripped workpiece 5 to a container such as a lunch box and to serve the workpiece 5 therein.

<Effect>
By performing clustering on point cloud data generated based on a plurality of bulk works, it is possible to identify an individual work. Note that for clustering, a community detection method in a network can be used, and modularity may be calculated based on the distance between nodes. In this way, compared to a method using a graph cut, for example, many parameters depending on the type of work are not required, and versatility can be improved.

<Modified example>
The control device 4 may be configured to be communicably connected to the RGB-D camera 2 and the manipulator 3 via a network such as the Internet. That is, the information acquired from the RGB-D camera 2 is processed on the cloud. Further, the manipulator 3 functions as a device that outputs processing results transmitted via a network, like a so-called thin client. In this way, there is no need to install a control device on the production line, and the effort required to maintain each of the plurality of production lines is also reduced.

Furthermore, the location where the RGB-D camera 2 is installed is not limited to directly above the workpiece 5. For example, the RGB-D camera 2 may be provided integrally with a robot having a manipulator 3. For example, the RGB-D camera 2 may be provided at a position corresponding to the face of a humanoid robot. With this aspect, even when the workpiece 5 is imaged from diagonally above, it is possible to specify a single workpiece 5 that is located at a position that is not hidden by other objects with the manipulator as a reference. Furthermore, it is possible to realize a robot with a shape that is easy to mix with humans and be introduced into a production line.

Furthermore, in addition to the point cloud data included in the distance image, sensing data such as tactile data may be acquired and used for clustering. In this way, a robot equipped with a multimodal interface can be realized.

<Variation of clustering>
Furthermore, the clustering method is not limited to hard clustering such as the Louvain method. Hard clustering is a method of assigning one cluster to one node. For example, soft clustering, which is a method of allocating multiple clusters to one node, may be used. Examples of soft clustering include OCG (Overlapping Cluster Generator) and Link Community.

FIG. 9 is a schematic diagram showing an example of a node group classified by soft clustering. FIG. 9 shows an example where a plurality of works exist near the top of a mountain. Furthermore, clusters classified by hatching are shown within each node. That is, the plurality of nodes are classified into four clusters located on the top, bottom, left and right of the figure. Furthermore, four clusters are assigned to one central node. Note that in soft clustering, the central node can be classified into three or less clusters, and the five central nodes can also be classified into a fifth cluster.

Soft clustering, for example, selects the node at the highest position and repeats (1) the process of extracting the cluster to which the selected node belongs, and (2) the process of selecting the nodes that belong to the extracted cluster. . Furthermore, boundaries between adjacent clusters are detected based on predetermined rules. Two or more clusters may be assigned to nodes near the boundary. Alternatively, boundaries between clusters may be detected based on geometric information such as the direction of normal lines determined from point cloud data.

By using soft clustering, it is possible to improve the accuracy when performing community detection from graphs with various structures, such as when multiple works exist near the top of a mountain.

<Others>
Note that the configuration described above is an example, and the present invention is not limited to the illustrated configuration. The above-mentioned matters can be implemented in appropriate combinations without departing from the problems and technical idea of the present invention.

The present invention also includes a computer program that executes the above-described process. Furthermore, a computer-readable recording medium on which the program is recorded also belongs to the scope of the present invention. Regarding the recording medium on which the program is recorded, the above-mentioned processing becomes possible by causing a computer to read and execute the program on the recording medium.

Here, the computer-readable recording medium refers to a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read by a computer. Among such recording media, those that can be removed from the computer include flexible disks, magneto-optical disks, optical disks, magnetic tapes, and memory cards. Furthermore, examples of recording media fixed to the computer include a hard disk drive, ROM, and the like.

1: Bin picking system 2: RGB-D camera 3: Manipulator 31: Gripper 4: Control device 41: Input/output I/F
42: Storage device 43: Processor 431: Point cloud information acquisition section 432: Graph generation section 433: Object detection section 434: Robot control section 44: Signal line 5: Work

Claims (13)

a point cloud information acquisition unit that acquires the point cloud data from a distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk;
an object detection unit that performs clustering on a node group generated by modeling the point cloud data to identify a region where one object exists;
has
The clustering involves connecting nodes whose distances are within a predetermined range with edges, calculating modularity, which is a predetermined evaluation function, using edge weights according to the distance between nodes, and maximizing the modularity. is performed using a community detection technique that classifies nodes as
Image processing device. The image processing device according to claim 1, wherein the clustering is performed using a community detection method using hard clustering . An image processing device according to claim 1 or 2 ;
The distance image generation device;
a manipulator that moves an object existing in the area specified by the object detection section;
Equipped with a bin picking system. The bin picking system according to claim 3 , wherein the one object is an object located on the surface of the plurality of objects stacked in bulk and not hidden by other objects with respect to the manipulator. An image processing method executed by a computer connected to a distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk, the method comprising:
acquiring the point cloud data from the distance image generation device;
Clustering is performed on a group of nodes generated by modeling the point cloud data to identify an area where one object exists ,
The clustering involves connecting nodes whose distances are within a predetermined range with edges, calculating modularity, which is a predetermined evaluation function, using edge weights according to the distance between nodes, and maximizing the modularity. is performed using a community detection technique that classifies nodes as
Image processing method. A computer connected to a distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk,
acquiring the point cloud data from the distance image generation device;
perform clustering on a group of nodes generated by modeling the point cloud data, and perform processing to identify an area where one object exists ;
The clustering involves connecting nodes whose distances are within a predetermined range with edges, calculating modularity, which is a predetermined evaluation function, using edge weights according to the distance between nodes, and maximizing the modularity. is performed using a community detection technique that classifies nodes as
Image processing program. a distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk ;
a manipulator that moves the object;
A control device that controls the manipulator,
a point cloud information acquisition unit that acquires the point cloud data from the distance image generation device ;
an object detection unit that performs clustering on a node group generated by modeling the point cloud data to identify a region where one object exists;
For a group of nodes included in the area where the one target object identified by the target detection unit exists, the center of gravity is determined by estimating that the shape on the opposite side to the distance image generation device is symmetrical, and the center of gravity is calculated according to the center of gravity. a control unit that controls the manipulator;
a control device having ;
A bin picking system equipped with The bin picking system according to claim 7 , wherein the clustering is performed using a community detection method using hard clustering or soft clustering. The bin picking system according to claim 7 or 8, wherein the clustering is performed using a community detection method that classifies nodes so as to maximize modularity, which is a predetermined evaluation function, based on distances between nodes. The bin picking system according to claim 7 or 8 , wherein the clustering is performed using an OCG (Overlapping Cluster Generator) or a Link Community. The one object is an object located on the surface of the plurality of objects stacked in bulk and not hidden by other objects with the manipulator as a reference. bin picking system. Executed by a distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk, and a control device connected to a manipulator that moves the objects,
acquiring the point cloud data from the distance image generation device;
Performing clustering on a node group generated by modeling the point cloud data to identify an area where one target object exists;
For a group of nodes included in the area where the identified one target object exists, the center of gravity is determined by estimating that the shape on the opposite side to the distance image generation device is symmetrical, and the manipulator is operated according to the center of gravity. to control and
control methods including. A distance image generation device that outputs point cloud data including a plurality of points generated based on distances to a plurality of objects stacked in bulk, and a control device connected to a manipulator that moves the objects,
acquiring the point cloud data from the distance image generation device;
Performing clustering on a node group generated by modeling the point cloud data to identify an area where one target object exists;
For a group of nodes included in the area where the identified one target object exists, the center of gravity is determined by estimating that the shape on the opposite side to the distance image generation device is symmetrical, and the manipulator is operated according to the center of gravity. to control and
A control program to run.

Images