JP7848769B2 - Control device and control method - Google Patents

Description

This disclosure relates to a control device and control method, and more particularly to a control device and control method for a robot arm that picks up and moves a workpiece.

For example, the control device described in Patent Document 1 calculates the center of gravity of a workpiece when it is picked up by a robot arm, based on detection information from a force sensor installed on the robot arm, and determines the type of workpiece by referring to the calculated center of gravity.

Japanese Patent Publication No. 2022-67995

The applicant has identified the following problem: While the control device described in Patent Document 1 can identify the type of workpiece, it has the problem of not being able to determine a gripping point that allows for a good pickup of each workpiece.

This disclosure was made in view of these problems, and aims to realize a control device and control method that can determine a gripping point that allows for good pickup of each workpiece.

A control device according to one aspect of this disclosure is a control device for a robot arm that picks up and moves a workpiece,
A reinforcement learning unit learns the gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece when the robot arm picks up the workpiece, using a deep reinforcement learning model that takes the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece and the joint information of the robot arm when the robot arm picks up the workpiece as inputs and the gripping point of the workpiece as output.
A path generation unit that derives joint information of the robot arm so as to pick up the workpiece at the gripping point of the workpiece learned by the deep reinforcement learning model,
A control unit that controls the robot arm based on the derived joint information of the robot arm,
Equipped with,
The reinforcement learning unit uses the deep reinforcement learning model to control the robot arm based on the derived joint information of the robot arm, and uses the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece, along with the derived joint information of the robot arm, to relearn the gripping point of the workpiece.

The control device described above acquires information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and information regarding the types of workpieces that have been learned. Based on the information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and the types of workpieces that have been learned, the control device includes a determination unit that determines whether or not the workpiece to be picked up this time has been learned.
In the present case, if the workpiece to be picked up has already been learned, it is preferable to omit the learning process using the deep reinforcement learning model, and if the workpiece to be picked up has not already been learned, it is preferable to perform the learning process using the deep reinforcement learning model.

In the control device described above, it is preferable that the joint information of the robot arm includes, in addition to the angle information of each joint of the robot arm, at least one of the angular velocity information, angular acceleration information, or torque information of the motor that drives the joint of the robot arm.

A control method relating to one aspect of this disclosure is a control method for a robot arm that picks up and moves a workpiece,
A step of determining the center of gravity of the workpiece when the robot arm picks up the workpiece based on detection information from a force sensor provided on the robot arm,
A deep reinforcement learning model takes the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece as input, and the joint information of the robot arm when the robot arm picks up the workpiece as output, to learn the gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece.
The process of deriving joint information of the robot arm so that the workpiece is picked up at the gripping point of the workpiece learned by the deep reinforcement learning model,
A step of controlling the robot arm based on the joint information of the robot arm derived above,
The process involves using the deep reinforcement learning model to control the robot arm based on the joint information of the robot arm derived above, to relearn the gripping point of the workpiece, using the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece, and the derived joint information of the robot arm.
It is equipped with.

The control method described above includes a step of acquiring information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and information regarding the types of workpieces that have been learned, and determining whether the workpiece to be picked up this time has been learned based on the information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and the types of workpieces that have been learned.
In the present case, if the workpiece to be picked up has already been learned, it is preferable to omit the learning process using the deep reinforcement learning model, and if the workpiece to be picked up has not already been learned, it is preferable to perform the learning process using the deep reinforcement learning model.

According to this disclosure, a control device and control method can be realized that can determine a gripping point that allows for a good grip on each workpiece.

This is a block diagram showing the configuration of the robot arm system according to Embodiment 1. This diagram shows how the robot arm of the robot arm system of Embodiment 1 grips a workpiece. This is a flowchart illustrating the flow of controlling the robot arm of the robot arm system of Embodiment 1. This diagram shows the workpiece, picked up by the robot arm, stored in a storage box. This is a block diagram showing the configuration of the robot arm system of Embodiment 2. This is a flowchart illustrating the control flow of the robot arm of the robot arm system of Embodiment 2.

The following describes specific embodiments applying this disclosure, with reference to the drawings. However, this disclosure is not limited to the following embodiments. Furthermore, the following description and drawings have been simplified as appropriate.

<Embodiment 1>
First, the configuration of the robot arm system of this embodiment will be described. Figure 1 is a block diagram showing the configuration of the robot arm system of this embodiment. Figure 2 is a diagram showing how the robot arm of the robot arm system of this embodiment grips a workpiece.

As shown in Figure 1, the robot arm system 1 of this embodiment comprises a robot arm 2, a force sensor 3, a camera 4, a control device 5, and a database (DB) 6. These robot arm 2, force sensor 3, camera 4, and DB 6 are communicated to the control device 5 via wired or wireless means.

The robot arm 2 is a multi-joint robot arm, similar to a typical robot arm, and comprises a hand section 2a and an arm section 2b, as shown in Figure 2. The hand section 2a can be configured as, for example, a two-fingered hand. The arm section 2b can be configured as, for example, a six-axis robot. Each joint of these hand section 2a and arm section 2b operates based on the driving force of the motor 2c.

As shown in Figure 2, the force sensor 3 is positioned between the hand portion 2a and the arm portion 2b of the robot arm 2. In other words, the hand portion 2a is connected to the arm portion 2b via the force sensor 3. The camera 4 can be configured as, for example, an RGBD camera.

Camera 4, for example, photographs the workpiece W before it is grasped by the robot arm 2. While it is preferable for camera 4 to be mounted on the robot arm 2, it is sufficient if it is positioned in a location that allows it to photograph the workpiece W before it is grasped by the robot arm 2.

As shown in Figure 1, the control device 5 comprises a shape recognition unit 5a, a reinforcement learning unit 5b, a path generation unit 5c, and a control unit 5d. The shape recognition unit 5a recognizes the shape of the workpiece W based on image information captured by the camera 4. A general method can be used to recognize the shape of the workpiece W.

The reinforcement learning unit 5b, as will be described in detail later, derives the center of gravity G of the workpiece W when the robot arm 2 picks it up, based on the detection information from the force sensor 3, and learns the gripping point P of the workpiece W that minimizes the amount of deviation from the center of gravity G of the workpiece W using a deep reinforcement learning model.

The deep reinforcement learning model takes the following as inputs: the amount of displacement between the center of gravity G of the workpiece W and the gripping point P of the workpiece W when the robot arm 2 picks up the workpiece W; and the joint information of the robot arm 2 when the robot arm 2 picks up the workpiece W. The model outputs the gripping point P of the workpiece W. The center of gravity G, gripping point P, and displacement amount can preferably be represented by three-dimensional coordinate information with a predetermined origin.

The path generation unit 5c, as will be described in detail later, generates a path for the robot arm 2 to approach the gripping point P of the workpiece W, derived based on the shape of the recognized workpiece W, or the gripping point P of the workpiece W learned by the deep reinforcement learning model, and pick up the workpiece W. It also derives joint information for the robot arm 2, such as the angles of each joint, to realize the generated path.

Furthermore, the path generation unit 5c generates a path for moving the workpiece W from its gripping point P, which was learned by a deep reinforcement learning model, to a predetermined location. It then derives joint information for the robot arm 2, such as the angles of each joint, to realize the generated path.

The control unit 5d controls each motor 2c of the robot arm 2 based on the angle information of each joint of the robot arm 2 that it has derived. At this time, the control unit 5d may control the motor 2c based on the detection information of the encoder 2d provided on the motor 2c. The DB 6 stores the deep reinforcement learning model and result information such as whether the gripping of the workpiece W was successful or not for each gripping point P of the workpiece W.

Next, the control flow of the robot arm 2 of the robot arm system 1 of this embodiment will be explained. Figure 3 is a flowchart illustrating the control flow of the robot arm of the robot arm system of this embodiment.

Here, Figure 4 shows the state in which the workpiece W to be picked up by the robot arm 2 is stored in the storage box 11. In the following description, we will assume that the workpiece W is being picked up. In the example of Figure 4, a plate-shaped workpiece W, which has an eccentric shape in side view as shown in Figure 2, is stored inside the storage box 11 and propped up against a partition 11a provided in the storage box 11.

In this state, first, camera 4 photographs the workpiece W from above. Then, the shape recognition unit 5a of the control device 5 recognizes the shape of the workpiece W based on the image information of the workpiece W captured by camera 4 (S1). At this time, the shape recognition unit 5a should recognize the shape of the workpiece W as viewed from above. In the example in Figure 4, the shape recognition unit 5a recognizes that the top view of the workpiece W is a thin plate shape.

Next, the path generation unit 5c of the control device 5 derives the gripping point P of the workpiece W based on the recognized shape of the workpiece W (S2). For example, if the top view of the workpiece W is a thin plate shape, the path generation unit 5c sets the gripping point P to the center of the width dimension of the workpiece W (left-right direction in Figure 2), as shown in Figure 2. However, the path generation unit 5c may, for example, set the gripping point P to approximately the center of the top view of the workpiece W based on the recognized shape of the workpiece W.

Next, the path generation unit 5c of the control device 5 generates a path for the robot arm 2 to approach the gripping point P of the workpiece W and pick up the workpiece W. It then derives joint information for the robot arm 2, such as the angles of each joint, to realize the generated path.

Then, the control unit 5d of the control device 5 controls each motor 2c of the robot arm 2 based on the angle information of each joint of the robot arm 2 that it has derived, to pick up the workpiece W (S3). Here, the workpiece W can be picked up by, for example, pulling the workpiece W up vertically to a predetermined height.

Next, the reinforcement learning unit 5b of the control device 5 derives the center of gravity G of the workpiece W when the robot arm 2 picks up the workpiece W, based on the detection information from the force sensor 3. It then acquires the amount of deviation between the center of gravity G of the workpiece W and the gripping point P of the workpiece W, as well as the joint information of the robot arm 2 when the robot arm 2 picks up the workpiece W (S4).

Here, a general method can be used to determine the center of gravity G of the workpiece W using the force sensor 3. Furthermore, the amount of displacement between the center of gravity G of the workpiece W and the gripping point P of the workpiece W should ideally be the amount of displacement in the horizontal direction.

Furthermore, the joint information of the robot arm 2 may include, in addition to the angle information of each joint when the robot arm 2 grasps the workpiece W in order to pick it up, at least one of the angular velocity information, angular acceleration information, and torque information of the motor 2c.

Next, the reinforcement learning unit 5b of the control device 5 receives the amount of displacement between the center of gravity G of the workpiece W and the gripping point P of the workpiece W, obtained in step S4, and the joint information of the robot arm 2 when the robot arm 2 picked up the workpiece W, and learns the gripping point P of the workpiece W that minimizes the displacement relative to the center of gravity G of the workpiece W (S5).

In this case, if the gripping point P of the workpiece W is within a localized area, it is conceivable that the robot arm 2 may not be able to grip the workpiece W at its gripping point P due to factors such as the shape of the workpiece W or interference with the storage box 11. Therefore, the reinforcement learning unit 5b of the control device 5 should distribute the gripping points P of the workpiece W within a predetermined range relative to the center of gravity G of the workpiece W (in the example in Figure 2, a predetermined range in the width direction of the workpiece W relative to the center of gravity G of the workpiece W).

Next, the path generation unit 5c of the control device 5 generates a path for the robot arm 2 to approach the learned gripping point P of the workpiece W and pick up the workpiece W. It then derives joint information for the robot arm 2, such as the angles of each joint, to realize the generated path.

Then, the control unit 5d of the control device 5 controls each motor 2c of the robot arm 2 based on the angle information of each joint of the robot arm 2 that it has derived, in order to pick up the workpiece W (S6).

Next, the reinforcement learning unit 5b of the control device 5 acquires the center of gravity G of the workpiece W when the robot arm 2 grasps the gripping point P of the workpiece W and picks up the workpiece W, based on the detection information of the force sensor 3. It also acquires the amount of deviation between the center of gravity G of the workpiece W and the gripping point P of the workpiece W when the robot arm 2 picks up the workpiece W, and the joint information of the robot arm 2 when the robot arm 2 picks up the workpiece W (S7).

Next, the reinforcement learning unit 5b of the control device 5 receives the amount of displacement between the center of gravity G of the workpiece W and the gripping point P of the workpiece W, obtained in step S7, and the joint information of the robot arm 2 when the robot arm 2 picked up the workpiece W, and relearns the gripping point P of the workpiece W that minimizes the displacement relative to the center of gravity G of the workpiece W (S8).

At this point, if, in step S5, the reinforcement learning unit 5b distributes the gripping points P of the workpiece W within a predetermined range relative to the centroid G of the workpiece W, it is advisable to input the result information of whether or not the gripping of the workpiece W was successful into the deep reinforcement learning model and retrain the gripping points P of the workpiece W that minimize the amount of displacement relative to the centroid G of the workpiece W.

This allows for secure gripping of the workpiece W at the gripping point P, where the amount of displacement relative to the workpiece W's center of gravity G is minimized. Therefore, even with large workpieces W, for example, the moment generated when gripping the workpiece W can be reduced, allowing for efficient pick-up of the workpiece W.

Next, the path generation unit 5c of the control device 5 generates a path for the robot arm 2 to approach the gripping point P of the relearned workpiece W, pick up the workpiece W, and move it to a predetermined location. It then derives joint information for the robot arm 2, such as the angles of each joint, to realize the generated path.

Then, the control unit 5d of the control device 5 controls each motor 2c of the robot arm 2 based on the angle information of each joint of the robot arm 2 that it has derived, to pick up the workpiece W and move it to the predetermined location (S9).

Thus, the control device 5 and control method of this embodiment learn the gripping point P of the workpiece W that minimizes the amount of displacement relative to the center of gravity G of the workpiece W, using a deep reinforcement learning model. Therefore, the control device 5 and control method of this embodiment can easily derive the gripping point P that allows for good gripping of each workpiece W.

Furthermore, it eliminates the need for operators to manually input the gripping point P for each workpiece W. Compared to the control device described in Patent Document 1, it allows for easy expansion of the number of grippable workpieces W, and enables the picking up of workpieces W regardless of their shape or packaging.

<Embodiment 2>
Figure 5 is a block diagram showing the configuration of the robot arm system of this embodiment. As shown in Figure 5, the robot arm system 21 of this embodiment is substantially the same as the robot arm system 1 of Embodiment 1.

Therefore, although redundant explanations will be omitted, as shown in Figure 5, the control device 22 is equipped with a determination unit 22a. This determination unit 22a determines whether the workpiece W to be picked up has already been learned, and based on the determination result, it decides whether or not to learn the gripping point P of the workpiece W.

Furthermore, DB23 stores information such as the type of workpiece W to be picked up by the robot arm 2, the order in which the workpieces W are picked up, information about learned workpiece W types, information about the gripping points P of learned workpiece W, and information about the joints of learned workpiece W.

Figure 6 is a flowchart illustrating the control flow of the robot arm in this embodiment of the robot arm system. The control flow of the robot arm 2 in the robot arm system 21 of this embodiment is substantially the same as the control flow of the robot arm 2 in the robot arm system 1 of Embodiment 1, as shown in Figure 6.

Therefore, although redundant explanations will be omitted, before step S1, the determination unit 22a of the control device 22 reads information from DB 23 regarding the type of workpiece W to be picked up by the robot arm 2, the order in which the workpieces W will be picked up, and information regarding the types of workpieces W that have already been learned. Based on this information, the determination unit 22a determines whether the workpiece W to be picked up this time has already been learned (S11).

If the workpiece W to be picked up is not already learned (NO in S11), steps S1 to S9 are executed, and the robot arm 2 is controlled to pick up and move the workpiece W. In other words, the control device 22 learns the gripping point of the workpiece W to be picked up this time.

On the other hand, if the workpiece W to be picked up is already learned (YES in S11), the control unit 5d of the control device 22 reads the joint information of the workpiece W from DB 23, and based on the read joint information of the workpiece W, controls each of the motors 2c of the robot arm 2 to attempt to grasp the workpiece W, and determines whether or not the grasp of the workpiece W was successful (S12).

If the gripping of workpiece W fails (NO in S12), steps S1 to S9 are executed, and the robot arm 2 is controlled to pick up and move the workpiece W. In this case, even if workpiece W has been learned, the failure to grip it may be due to a change in the packaging of workpiece W.

On the other hand, if the gripping of the workpiece W is successful (YES in S12), the robot arm 2 controls each of its motors 2c based on the read joint information of the workpiece W to continue moving the workpiece W (S13). In other words, the control device 22 omits learning the gripping point of the workpiece W to be picked up this time.

Thus, the control device 22 and control method of this embodiment also learn the gripping point P of the workpiece W that minimizes the amount of displacement relative to the centroid G of the workpiece W using a deep reinforcement learning model. Therefore, the control device 22 and control method of this embodiment can easily derive the gripping point P that allows for good gripping of each workpiece W.

In particular, the control device 22 and control method of this embodiment determine whether the workpiece W to be picked up has already been learned, and based on the determination result, the learning process using the deep reinforcement learning model is omitted. Therefore, it is not necessary to learn the gripping point P of the workpiece W each time it is picked up, thereby reducing the computational burden on the control device 22.

In this embodiment, the determination unit 22a of the control device 22 reads information from DB 23 regarding the type of workpiece W to be picked up by the robot arm 2, the order in which the workpieces W are picked up, and information regarding the types of workpieces W that have already been learned. Based on this information, the determination unit 22a determines whether the workpiece W to be picked up this time has already been learned. However, the determination may also be made based on the shape of the workpiece W recognized from image information of the workpiece W to be picked up this time.

Furthermore, although step S12 is performed in this embodiment, step S12 may be omitted.

<Other Embodiments>
In the embodiments described above, the disclosure was explained as a hardware configuration, but the disclosure is not limited thereto. The disclosure can also be implemented by having a CPU (Central Processing Unit) execute a program for each process.

Here, a program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions. A program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drives (SSDs), or other memory technologies, CD-ROMs, digital versatile discs (DVDs), Blu-ray® discs, or other optical disc storage, magnetic cassettes, magnetic tapes, magnetic disk storage, or other magnetic storage devices. A program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable mediums or communication media that include electrically, optically, acoustically, or otherwise propagating signals.

This disclosure is not limited to the embodiments described above, and may be modified as appropriate without departing from the spirit of the invention.

1 Robot arm system 2 Robot arm, 2a Hand section, 2b Arm section, 2c Motor, 2d Encoder 3 Force sensor 4 Camera 5 Control device, 5a Shape recognition section, 5b Reinforcement learning section, 5c Path generation section, 5d Control unit 6 Database 11 Storage box 21 Robot arm system 22 Control device, 22a Judgment section 23 Database G Center of gravity of workpiece P Gripping point of workpiece W Workpiece

Claims (5)

A control device for a robot arm that picks up and moves a workpiece,
A reinforcement learning unit learns the gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece when the robot arm picks up the workpiece, using a deep reinforcement learning model that takes the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece and the joint information of the robot arm when the robot arm picks up the workpiece as inputs and the gripping point of the workpiece as output.
A path generation unit that derives joint information of the robot arm so as to pick up the workpiece at the gripping point of the workpiece learned by the deep reinforcement learning model,
A control unit that controls the robot arm based on the derived joint information of the robot arm,
Equipped with,
The reinforcement learning unit is a control device that, using the deep reinforcement learning model, controls the robot arm based on the joint information of the robot arm derived to pick up the workpiece, and uses the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece, along with the derived joint information of the robot arm, to relearn the gripping point of the workpiece. The system includes a determination unit that acquires information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and the types of workpieces that have been learned, and determines whether the workpiece to be picked up this time has been learned or not based on the information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and the types of workpieces that have been learned.
The control device according to claim 1, wherein if the workpiece to be picked up this time has already been learned, the learning by the deep reinforcement learning model is omitted, and if the workpiece to be picked up this time has not been learned, the learning by the deep reinforcement learning model is performed. The control device according to claim 1 or 2, wherein, in addition to the angle information of each joint of the robot arm, the joint information of the robot arm includes at least one of the angular velocity information, angular acceleration information, or torque information of the motor that drives the joint of the robot arm. A method for controlling a robot arm that picks up and moves a workpiece,
A step of determining the center of gravity of the workpiece when the robot arm picks up the workpiece based on detection information from a force sensor provided on the robot arm,
A deep reinforcement learning model takes the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece as input, and the joint information of the robot arm when the robot arm picks up the workpiece as output, to learn the gripping point of the workpiece that minimizes the amount of deviation from the center of gravity of the workpiece.
The process of deriving joint information of the robot arm so that the workpiece is picked up at the gripping point of the workpiece learned by the deep reinforcement learning model,
A step of controlling the robot arm based on the joint information of the robot arm derived above,
The process involves using the deep reinforcement learning model to control the robot arm based on the joint information of the robot arm derived above, to relearn the gripping point of the workpiece, using the amount of deviation between the center of gravity of the workpiece and the gripping point of the workpiece when the robot arm picks up the workpiece, and the derived joint information of the robot arm.
A control method comprising: The system includes a step of acquiring information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and the types of workpieces that have been learned, and determining whether the workpiece to be picked up this time has been learned based on the information regarding the type of workpiece to be picked up by the robot arm, the order in which the workpieces are picked up, and the types of workpieces that have been learned,
The control method according to claim 4, wherein if the workpiece to be picked up this time has already been learned, the learning by the deep reinforcement learning model is omitted, and if the workpiece to be picked up this time has not been learned, the learning by the deep reinforcement learning model is performed.