JP7788948B2 - Processing device, robot control system and machine learning method - Google Patents

Description

The present invention relates to a processing device, a robot control system, and a machine learning method.

High-mix, low-volume production systems are attracting attention in order to respond to the rapid changes in the business environment and the diversification of people's needs. While robotic automation is considered a way to increase production efficiency, controlling robots requires enormous programming costs and a high level of specialized knowledge, posing a challenge in terms of the large amount of work required for implementation. Therefore, to reduce the amount of work required for robot implementation, machine learning is being used in all robot functions. For example, in object recognition, numerous studies have been reported that use machine learning to quantitatively calculate object features from contact information with the robot and image information in order to estimate the type, position, and orientation of an object.

On the other hand, when it comes to motion generation, numerous studies have been reported that apply machine learning to the autonomous control of robots in order to realize movements that are difficult to describe in programs or that respond to environmental changes. Reinforcement learning is one example of an autonomous control technology. Robots acquire the optimal movements to perform a task through trial and error, so there is no need to explicitly teach them movements. However, acquiring the optimal movements requires a huge amount of trial and error (learning time).

Imitation learning is an autonomous control technology with a short learning time. Imitation learning consists of three phases: teaching, learning, and execution. In the teaching phase, the instructor operates the robot to teach it the desired behavior, and collects and records time-series information output by the robot's onboard encoders, temperature sensors, vision sensors, ultrasonic sensors, and other sensors. Robots can be taught to perform certain actions by remote control using a controller, direct teaching, in which the robot is directly touched, or by using a control program.

The time-series information obtained in the teaching phase is called teaching data. The type of teaching data depends on the movement generation method. For example, when generating movements based on the positions of the robot or object, the teaching data includes the robot's position, joint angle information, etc. Furthermore, when generating movements based on visual information, images of the robot and the work environment may be included. Furthermore, when generating movements based on the state of the object, position and posture information of the object may be included. Furthermore, when generating movements based on the robot's speed, for example, the teaching data includes the robot's speed, joint angular velocity information, etc. Furthermore, when generating movements based on the robot's force, for example, the teaching data includes tactile information from the robot.

In the learning phase, training data is created based on the teaching data collected in the teaching phase, and this training data is used to train the movement generation model. The teaching data may be used as is. Alternatively, training data may be created by converting the teaching data to remove noise, missing values, error values, etc. contained in the teaching data. Converting teaching data into training data is called preprocessing. Preprocessing includes data cleaning, which removes data containing missing values and outliers, and normalization, which converts data values so that they fall within a specified range. The training data is used to update the weights of the machine learning model so that when a certain value is input into the model, the desired value is output. For example, when joint angle information and an image at a certain time are input, the model is trained to predict joint angle information and an image at the next time.

In the execution phase, the task is executed by generating movements using the movement generation model obtained in the learning phase and the robot's sensor data. For example, in the learning phase, a machine learning model is trained to predict the joint angle information and image at the next time point when joint angle information and an image at a certain time point are input. By using this machine learning model in the execution phase, joint angle information and images at each time point can be input, and the output joint angle information can be used as control command values for the robot, allowing it to generate movements autonomously.

In order to improve the accuracy of motion generation in imitation learning, the following imitation learning is described in Patent Document 1. Patent Document 1 states that "imitation learning is performed based on observational information grasped by the operator during the model operation."

Patent Publication No. 2021-10984

One of the challenges with imitation learning is the difficulty of simultaneously reducing implementation effort (efforts in the teaching and learning phases) and acquiring generalized behavior. For example, with program-based teaching, the robot is operated by computer control, so more balanced teaching data can be obtained than when operated by a human, making it easier to learn the relationships between the teaching data and improving generalized behavior performance. However, because the robot's movements must be programmed one by one, implementation effort is high.

Teaching methods that require less initial effort than program-based methods include remotely controlling the robot using a controller or by directly touching it. These teaching methods require less initial effort because they do not require programming each and every robot movement.

However, when a robot is operated by a human, variations in the movement speed, movement timing, etc. occur over time for each piece of teaching data, compared to program-based teaching methods, making it difficult to learn the relationships between pieces of teaching data. Ignoring the effects of this variation over time requires a huge amount of training data, which increases the number of teaching sessions.

The present invention aims to provide a processing device, robot control system, and machine learning method that can reduce the temporal variability in teaching data that impedes movement learning. Note that while the technology described in Patent Document 1 is similar in that it manipulates learning data to improve movement generation accuracy, it does not mention reducing the temporal variability.

To achieve the above objective, one example of a processing device of the present invention includes a processor that segments teaching data for a robot into groups of similar movements of the robot, corrects the segmented teaching data to align the speed or timing of similar movements of the robot, combines the corrected teaching data, and performs machine learning using the combined teaching data.

This invention can reduce the temporal variation in teaching data that hinders motion learning. Issues, configurations, and advantages other than those described above will become clear from the description of the following embodiments.

1 is a schematic diagram showing an example of the configuration of a robot control system to which the present invention is applied; FIG. 2 is a block diagram showing an example of the hardware configuration of a computer that realizes a motion planning unit of the robot control system. FIG. 2 is a block diagram showing an example of the functional configuration of a computer included in an operation planning unit of the robot control system. 2 is a block diagram illustrating an example of a functional configuration of a consistency verification device provided in an operation planning unit according to the first exemplary embodiment of the present invention. FIG. FIG. 4 is an explanatory diagram of an example of a method for implementing speed adjustment in the first embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of the functional configuration of a machine learning device included in an action planning unit according to the first exemplary embodiment of the present invention. 5 is a flowchart showing an example of the operation of a consistency verification device provided in an operation planning unit in the first exemplary embodiment of the present invention. FIG. 10 is an explanatory diagram of a reaching task to show an example of the effect of the consistency verification device provided in the operation planning unit in the first embodiment of the present invention. FIG. 10 is a diagram showing the relationship between time and the hand position in the Y-axis direction during a teaching operation. FIG. 10 is a diagram showing the state of a reaching task when there is variation in the teaching data in the time direction. FIG. 10 is a diagram showing the relationship between time and the hand position in the Y-axis direction during a teaching operation with variation in the time direction. FIG. 8C is an explanatory diagram of the application of the consistency verification device to the reaching task shown in FIGS. 8C and 8D. FIG. 10 is a block diagram illustrating an example of the internal configuration of a consistency verification device provided in an operation planning unit according to a second exemplary embodiment of the present invention. 10 is a flowchart showing an example of the operation of a consistency verification device provided in an operation planning unit in the second exemplary embodiment of the present invention. FIG. 10 is a schematic diagram illustrating an example of the configuration of a robot control system according to a third embodiment of the present invention. FIG. 11 is a block diagram illustrating an example of the functional configuration of a computer included in an operation planning unit of a robot control system according to a third embodiment of the present invention. FIG. 11 is a block diagram showing an example of the hardware configuration of a computer that realizes an operation planning unit of a robot control system according to a third embodiment of the present invention. FIG. 11 is a block diagram showing an example of the functional configuration of a screening device provided in an operation planning unit of a robot control system according to a third embodiment of the present invention. FIG. 11 is a block diagram showing an example of the functional configuration of a screen operation unit included in a robot control system according to a third embodiment of the present invention. FIG. 11 is a block diagram showing an example of the functional configuration of an operation planning unit included in a robot control system according to a fourth embodiment of the present invention. FIG. 13 is a block diagram showing an example of the functional configuration of an operation parameter adjustment device provided in an operation planning unit of a robot control system according to a fourth embodiment of the present invention.

The following describes first to fourth embodiments with reference to the accompanying drawings. These embodiments relate to robot motion generation technology that learns time-series data from a robot and autonomously generates motion. The purpose of these embodiments is to reduce the time-dependent variation in teaching data that impedes motion learning, thereby achieving both reduced implementation effort and the acquisition of generalized motion.

(First embodiment)
First, a robot control system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 3. FIG.

Figure 1 shows an example of the configuration of a robot control system to which the present invention is applied. The robot control system 100 shown in Figure 1 comprises a robot device 1, a sensor data acquisition unit 2, a motion planning unit 3, and a control unit 4. A single-arm robot arm is shown as a specific example of the robot device 1, but the configuration of the robot device 1 is not critical and it may be, for example, a double-arm arm. It may also be a moving device such as legs, crawlers, wheels, or propellers.

The sensor data acquisition unit 2 acquires sensor data output from the robot device 1. The sensor data includes sensor data of the robot device 1 and the environment acquired during work.

The motion planning unit 3 uses the data obtained by the sensor data acquisition unit 2 as teaching data and learns and generates motions in the following three steps. First, it creates training data by performing preprocessing to reduce variability in the teaching data. Next, it constructs a motion generation model by performing motion learning based on the training data. Finally, it generates motion command values using the constructed motion generation model.

The control unit 4 provides the robot device 1 with the command values calculated by the motion planning unit 3, thereby driving the actuators installed in the robot device 1.

[Hardware configuration of the motion planning unit]
Next, the hardware configuration of the motion planning unit 3 included in the robot control system 100 will be described with reference to Fig. 2. Here, the hardware configuration of a computer that realizes the motion planning unit 3 will be described.

Figure 2 is a block diagram showing an example of the hardware configuration of a computer that realizes the action planning unit 3. The illustrated computer 20 is an example of hardware that constitutes the computer used in the sensor data acquisition unit 2, action planning unit 3, and control unit 4. A personal computer, for example, can be used as the computer 20.

The computer 20 includes a ROM (Read Only Memory) 22, a CPU (Central Processing Unit) 23, a RAM (Random Access Memory) 24, non-volatile storage 25, an input/output interface 26, and a network interface 27, all connected to a bus 21.

The ROM 22 stores software program code that realizes the functions of the sensor data acquisition unit 2, motion planning unit 3, and control unit 4 according to this embodiment.

The CPU 23 functions as an arithmetic processing device that reads from the ROM 22 the program code of the software that realizes the functions of the sensor data acquisition unit 2, the action planning unit 3, and the control unit 4 according to this embodiment, and loads and executes the program into the RAM 24. Values of variables, parameters, etc. that arise during the arithmetic processing of the CPU 23 are temporarily written to the RAM 24. The values of variables, parameters, etc. written to the RAM 24 are read by the CPU 23 as appropriate. Although a CPU is used as the arithmetic processing device, other processors such as an MPU (Micro Processing Unit) may also be used.

Non-volatile storage 25 is an example of a recording medium, and can store data used by programs and data obtained by running programs. For example, non-volatile storage 25 stores learning data and behavior generation models, which will be described later. Non-volatile storage 25 may also store an OS (Operating System) and programs executed by CPU 23. Magnetic recording media, optical recording media, semiconductor recording media, etc. can be used as non-volatile storage 25.

The input/output interface 26 is an interface that communicates signals and data from each sensor and actuator included in the robot control system 100. The input/output interface 26 may also have the functionality of an A/D (Analog/Digital) converter and/or a D/A converter (not shown) that processes input or output signals. In this specification, sensor data includes not only information obtained from each sensor but also information obtained from each actuator.

The network interface 27 may be, for example, a network interface card (NIC) or a modem. The network interface 27 is configured to be able to send and receive various types of data to and from external devices via a communication network such as a LAN or the Internet to which a terminal is connected, or via a dedicated line. For example, the network interface 27 can be used to send and receive various types of data to and from the robot device 1.

[Functional configuration of the motion planning unit]
Next, the functional configuration of the motion planning unit 3 included in the robot control system 100 will be described with reference to FIG. 3. FIG. 3 shows an example of the functional configuration of the motion planning unit 3 embodying the present invention. In the motion planning unit 3 shown in FIG. 3, the sensor data storage device 31 is a device that records the sensor data acquired by the sensor data acquisition unit 2 as time-series data. The consistency verification device 32 is a device that converts the time-series data recorded in the sensor data storage device 31 and outputs learning data. The learning data storage device 33 is a device that records the learning data output from the consistency verification device 32. The machine learning device 34 is a device that learns the learning data recorded in the learning data storage device 33 and outputs a command value to the control unit 4.

Next, the configuration of the consistency verification device 32 of the motion planning unit 3 will be explained using Figure 4. In the consistency verification device 32, the motion segmentation unit 41 is a device that divides sensor data into groups of motions. Methods for achieving motion segmentation include segmentation based on speed changes, segmentation by clustering, and segmentation by deep learning.

Segmentation based on speed change is a method based on the robot's movement speed, and can be used when the data characteristics are simple. Segmentation by clustering can be used when the teaching data characteristics are complex, but the number of segments must be set in advance. Segmentation by deep learning can be used when the teaching data characteristics are complex, and there is no need to set the number of segments in advance. However, compared to segmentation based on speed change and segmentation by clustering, it has higher implementation and computational costs.

The speed adjustment unit 42 adjusts the movement speed to reduce variations in movement speed between teaching data, which can hinder the acquisition of generalized movements. As an example of how the speed adjustment unit can be implemented, we will explain an algorithm that aligns the speed between teaching data by quantizing the teaching data and removing noise.

Figure 5 shows the procedure and effect of this algorithm. Graph g5A shows data of different types with different movement speeds, with the horizontal axis representing time and the vertical axis representing the hand position along a certain axis in the world coordinate system (hereafter referred to as the x-axis). Graph g5B shows the results of quantization, which converts the hand position values into discrete approximate values, for the two teaching data shown in graph g5A. As shown in graph g5B, quantization makes the amount of change in the teaching data per time constant.

Note that if the amount of change per unit time in the teaching data is greater than the quantization width, the amount of change after quantization will not necessarily be a constant value. Therefore, the size of the quantization width must be the same as or greater than the amount of change per unit time in the teaching data. In other words, after speed adjustment, the speed will be the same as or greater than the original teaching data.

However, by performing upsampling, as described in Variation 2, it is possible to adjust the speed to a slower speed than the original teaching data. Enlarged view 5C shows that quantizing the hand in one direction results in periods of time when the hand position does not change, i.e., periods of time when the speed is zero. Graph g5D shows the results of noise removal from the quantized teaching data shown in graph g5B. Noise removal means that the periods of time when the speed is zero, which occur after quantization, are treated as noise and removed. As shown in graph g5D, it can be seen that the hand speeds of the two teaching data are consistent. It can also be seen that the final hand position remains the same as before the speed adjustment. From the above, it can be seen that speed adjustment can be achieved by performing quantization and noise removal.

The timing adjustment unit 43 shown in Figure 4 adjusts the start and end timing of movements to reduce variations in the timing of movement start and end between teaching data, which can hinder the acquisition of generalized movements. One example of a method for achieving timing adjustment is to increase or decrease the rest time of the robot device 1. For example, to align the movement start times of the robot device 1 for all teaching data, it is possible to calculate the average movement start times of all teaching data, and increase or decrease the rest time so that the movement start time for each teaching data is equal to this average.

The dynamics adjustment unit 44 adjusts the dynamics of the sensor data, such as acceleration, jerk, and torque, to prevent the generated motion from becoming unstable due to excessive magnitude or change in the dynamics of acceleration, jerk, torque, etc. in the teaching data after speed adjustment. One example of a method for implementing dynamics adjustment is to apply a moving average filter to the hand position data.

The data processing unit 45 converts the data output from the dynamics adjustment unit 44 into training data so that it can be learned by a deep learning model. Specifically, for each piece of teaching data, the training data is constructed by synthesizing each adjusted segment.

Next, the configuration of the machine learning device 34 will be described using Figure 6. In the machine learning device 34, the machine learning model definition unit 61 defines the structure and parameters of the machine learning model.

The learning unit 62 uses the learning data stored in the learning data storage device 33 to update the weights of the machine learning model defined by the machine learning model definition unit 61, so that when a certain value is input to the model, the desired value is output. For example, when the joint angle of the robot at a certain time is input, the predicted value of the joint angle of the robot at the next time is output.

The trained weight storage unit 63 stores the weight parameters of trained models.

The inference unit 64 constructs a movement generation model using the machine learning model read from the machine learning model definition unit 61 and the weight parameters of the model read from the learned weight accumulation unit 63, and generates movement by inputting the same sensor data as during learning into the model.

For example, suppose that during learning, a model is constructed so that when the robot's joint angles at a certain time are input, it outputs a predicted value for the robot's joint angles at the next time. In this case, when generating movement, the joint angles at each time are input into the model to obtain the joint angles at the next time, and the robot's movement can be generated by using the obtained joint angles as command values.

[Example of operation of the operation planning unit 3]
Next, we will explain the order in which the functions of the motion planning unit 3 explained so far operate. The operation order can be broadly divided into three phases: teaching, learning, and execution. In the teaching phase, the robot device 1 is taught a motion by operating it, and the sensor data obtained during this process is recorded in the sensor data storage device 31 using the sensor data acquisition unit 2.

In the learning phase, the consistency verification device 32 converts the sensor data from the sensor data storage device 31 into learning data, and the data is trained using the machine learning model defined by the machine learning model definition unit 61 to obtain learned weights. The consistency verification device 32 converts the sensor data through three steps: motion segmentation, speed adjustment, and timing adjustment. These three steps are explained with reference to Figure 7.

Figure 7 is a flowchart showing an example of the operation of the consistency verification device 32. After preprocessing begins (S1), all teaching data is first segmented into movements (S2). In motion segmentation, the motion segmentation unit 41 divides the sensor data in the time direction, separating it into groups of movements. After motion segmentation, each segment is subjected to speed adjustment (S5), timing adjustment (S6), and dynamics adjustment (S7) (S3, S4). The data processed by the timing adjustment unit 43 is converted into learning data using the data processing unit 45 (S8), and preprocessing ends (S9).

As described above, by processing the teaching data with the consistency verification device 32, learning data can be obtained that reduces the variation in the time direction between the teaching data. Note that it is not necessary to perform both speed adjustment and timing adjustment; in some cases, it is sufficient to perform only one of them. For example, if only the speed of the teaching movement differs, only the speed needs to be adjusted, and timing adjustment is not necessary. Furthermore, if only the timing of the teaching movement differs, only the timing needs to be adjusted, and speed adjustment is not necessary.

The training data constructed by the consistency verification device 32 is input to the training unit 62 of the machine learning device 34. The training unit 62 calls a machine learning model from the machine learning model definition unit 61 and trains it based on the training data. The behavior generation model obtained as a result of training is stored in the trained weight storage unit 63.

In the execution phase, the task is executed by generating actions using the action generation model obtained in the learning phase. The inference unit 64 of the machine learning device 34 is used to calculate command values for the robot device 1.

[Example of effect of motion planning section]
To verify the effectiveness of the motion planning unit 3, we will explain how the operation of the motion planning unit 3 reduces the variation in the time direction between pieces of teaching data, with reference to Figures 8 (8A to 8D) and 9. Figures 8 (8A to 8D) show a reaching task in which the hand of a robot arm is moved linearly to a target hand position. We will use Figures 8 (8A to 8D) to explain the effect that variation in the time direction in the teaching data has on the reaching task.

First, using Figures 8A and 8B (graph g8B), we will explain why a task will be successful if there is no variation in the teaching data over time. The hand depicted in Figure 8A indicates the position of the robot hand at the initial time. Also, p, q, and r depicted in Figure 8A respectively represent the target hand positions. X and Y in Figure 8A are the names of the coordinate axes set in the task environment, and here the X and Y axes indicate position [cm]. Now, suppose that a movement corresponding to the target hand positions p and r is taught. In Figure 8A, the arrows (solid lines) pointing from the hand to p and r represent the trajectory of the hand during the teaching movement.

Figure 8B (graph g8B) shows the relationship between time and hand position in the Y-axis direction during a teaching motion. As shown in graph g8B, the teaching motions for target hand positions p and r have the same hand velocity and reaching start and end times, and there is no variation in the time direction. By learning the teaching motions for target hand positions p and r in the motion learning phase, a motion generation model is acquired that, when current sensor data is input, outputs command values for the robot arm at the next time.

Next, with reference to Figure 8A and graph g8B, consider the movement generated for target hand position q in the movement execution phase. In the movement execution phase, the target hand position q is generalized to the movement generation model constructed in the movement learning phase through point-to-point learning, so that it is located exactly halfway between target hand positions p and r. At this time, the hand position generated for q passes exactly halfway between the hand positions of the taught movements for p and r at each time point. Figure 8A and graph g8B show the movement generated for q with a dashed line, and the hand is stable with no wobble in the Y-axis direction. This shows that the reaching task will be successful if there is no variation in the taught data over time.

Next, we will use Figures 8C and 8D (graph g8D) to explain why variability causes task failure. Figure 8C shows the reaching task when there is variability in the teaching data in the time direction. Each symbol in the figure has the same meaning as in Figure 8A.

Figure 8D (graph g8D) shows the relationship between time and hand position in the Y-axis direction during a teaching operation. There is variation in the time direction between the teaching operations for target hand positions p and r. Specifically, the movement speed is faster for the teaching operation for target hand position r than for target hand position p. The reaching start time (two-dot chain line) and completion time (single-dot chain line) are also earlier.

Next, with reference to Figure 8C and graph g8D, consider the movement generated for the target hand position q. As mentioned above, due to the positional relationship between p, q, and r, the movement generated for q passes exactly halfway between the taught movements for p and r. In Figure 8C and graph g8D, the movement generated for q is shown by the dashed line. Looking at the movement generated for q, we can see that the hand is significantly unstable in the Y-axis direction. This shows that if there is variation in the taught data over time, the reaching task will fail.

Next, using Figure 9, we will explain how using the consistency verification device 32 can acquire generalized behavior even when there is variation in the teaching data in the time direction. Figure 9 is an explanatory diagram of the application of the consistency verification device 32 to the reaching task shown in Figures 8C and 8D. The same reference numerals in Figure 9 indicate the same parts as in Figures 8C and 8D, so repeated explanations will be omitted.

Graph g9A shows the teaching data for the target hand positions p and r shown in graph g8D. As mentioned above, there is variation in the teaching data for the target hand positions p and r over time.

Graph g9B shows the segmented movements of the taught movements for the target hand positions p and r shown in graph g9A. The dashed-dotted lines in graph g9B represent the segmented times. As shown in graph g9B, the movements are divided into the reaching start time and the reaching end time. For example, as shown in graph g9B, to detect the reaching start time, the movement segmentation unit 41 can perform segmentation based on speed changes to find the time when the hand starts moving from a stationary state.

Graph g9C is an explanatory diagram for aligning the hand speed for the hand movement shown in graph g9B. As shown in graph g9C, the arm's hand speed from the reaching start time to the reaching completion time is adjusted to be equal to the average hand speed of the two teaching data + α (constant). More preferably, the arm's hand speed from the reaching start time to the reaching completion time is set equal to the average value.

Graph g9D is an explanatory diagram illustrating how to align the timing of the hand movements shown in graph g9C. As shown in graph g9D, the reaching start time and reaching end time are made equal between the teaching data.

Graph g9E shows the motion generated for the target hand position q in Figure 8A during the execution phase. Since the target hand position q is located exactly halfway between the target hand positions p and r, the motion generated for q also passes exactly halfway between the taught motions for p and r. Looking at the motion generated for q, the hand does not wobble in the Y-axis direction and is stable.

<Modification 1>
In this embodiment, the motion segmentation unit 41 divides the sensor data into groups of motions. However, if the data characteristics are complex, segmentation may not work well. To address this issue, it is possible to use recurrent dropout. Recurrent dropout is a method of performing learning while randomly deactivating nodes in a recurrent neural network. Using recurrent dropout is expected to improve generalization performance with respect to temporal variations in the training data.

<Modification 2>
In this embodiment, the sensor data is quantized by the speed adjustment unit 42. However, if the amount of change per unit time of the sensor data is larger than the quantization width (i.e., if the speed after conversion is desired to be increased), the amount of change per unit time of the quantized sensor data exceeds the quantization width, and therefore the amount of change per unit time after quantization is not equal, and it fails to make the speeds uniform.

To avoid this, the speed adjustment unit 42 must perform upsampling before quantization to reduce the amount of change per unit time of the sensor. Upsampling means increasing the sampling frequency of time-series data. Methods for achieving upsampling include linear interpolation, nearest neighbor interpolation, and spline interpolation. However, these interpolation methods are known to be inapplicable to image data, and a separate technique called frame interpolation is required. Details will be explained in the second embodiment.

The main features of this embodiment can be summarized as follows:

The processor (CPU 23) of the processing device (computer 20) segments the teaching data for the robot (robot device 1) into groups of similar robot movements (movement segmentation unit 41). The processor (CPU 23) corrects the segmented multiple pieces of teaching data to align the speed or timing of similar robot movements (speed adjustment unit 42, timing adjustment unit 43). The processor (CPU 23) combines the corrected multiple pieces of teaching data (data processing unit 45). The processor (CPU 23) performs machine learning using the combined teaching data (machine learning device 34).

This reduces the variation in the teaching data over time. As a result, the implementation effort (the effort required for the teaching and learning phases) is reduced, and it also makes it easier to learn the relationships between teaching data, improving generalization performance.

The processor (CPU 23) applies corrections to the segmented teaching data to smooth the dynamics values of the robot's similar movements (dynamics adjustment unit 44). For example, dynamics values (acceleration, jerk, torque, etc.) are smoothed by taking a moving average. This makes it possible to stabilize the robot's movements.

The teaching data includes, for example, at least one of position information indicating the robot's position, joint angle information indicating the joint angles, speed information indicating the robot's speed, joint angular velocity information indicating the robot's joint angular velocity, and tactile information indicating the sensor values of tactile sensors provided on the robot. In this embodiment, the teaching data includes position information indicating the robot's position (gripper position). This allows machine learning to be performed using the robot's state as teaching data.

The teaching data may also include at least one of position information indicating the position of an object on which the robot is to perform work, and posture information indicating the posture of the object. This allows machine learning to be performed using the state of the object as teaching data.

Dynamics include, for example, at least one of acceleration, acceleration change, jerk, jerk change, torque, and torque change. This allows the robot's movements to be smooth.

The processor (CPU 23) quantizes the segmented teaching data and removes noise to align the speed of similar robot movements (Figure 5). This allows for rapid reduction of variation in the teaching data over time.

The processor (CPU 23) adjusts the timing of similar robot movements by increasing or decreasing the robot's stationary time for each segmented piece of teaching data (Figure 9). This makes it easy to reduce the variation in the teaching data over time.

The robot control system 100 includes a processing device (computer 20) and a robot (robot device 1). The processor (CPU 23) uses a trained machine learning model to generate command values for the robot's movements (machine learning device 34). The robot operates in accordance with the command values. This allows the robot to be controlled autonomously.

Second Embodiment
Next, a second embodiment of the present invention will be described with reference to FIGS. 10 and 11, in which sensor data includes images.

[Example of functional configuration]
FIG. 10 is a block diagram showing an example of the internal configuration of a consistency verification device provided in a motion planning unit according to a second embodiment of the present invention. In FIG. 10, the same reference numerals as in FIG. 4 denote the same components, and therefore a repeated description will be omitted. As shown in FIG. 10, the second embodiment is the same as the first embodiment in that sensor data is divided by a motion segmentation unit 41, time-domain variations are reduced by a speed adjustment unit 42 and/or a timing adjustment unit 43, dynamics are adjusted by a dynamics adjustment unit 44, and the data is converted into training data by a data processing unit 45. The second embodiment differs from the first embodiment in that the consistency verification device 32 includes a frame interpolation unit 101. The frame interpolation unit 101 increases the frame rate of image time-series data.

Methods for achieving frame interpolation include frame interpolation based on optical flow and frame interpolation using deep learning. Optical flow-based frame interpolation consists of three steps: feature extraction, motion change calculation, and interpolated image generation. For example, consider two consecutive images from a time series of images. Feature extraction involves extracting the features of the two images. Motion change calculation focuses on the same feature calculated in feature extraction and calculates its motion change. This motion change of the feature is called optical flow. Interpolated image generation estimates the image between the two original images by moving pixels in the two images based on the motion change.

Examples of frame interpolation using deep learning include models such as FLAVR and FILM. FLAVR is a deep learning model that enables highly accurate frame interpolation by performing optical flow and 3D convolution calculations internally. FILM is a deep learning model that takes into account the scale of the image (enlargement and reduction). By inputting image time series data into FLAVR (or FILM), it is possible to obtain image time series data after frame interpolation.

[Example of operation]
FIG. 11 is a flowchart showing an example of the operation of the consistency verification device 32 in the second embodiment of the present invention. As shown in FIG. 11, the second embodiment is similar to the first embodiment in that it segments the motion of sensor data (S2), adjusts speed (S6) and timing (S7) for each segment (S3, S4), and synthesizes the adjusted segments for each teaching data (S8, S9) to construct training data. Note that in FIG. 11, the same reference numerals as in FIG. 7 indicate the same components, and therefore, a repeated description will be omitted. The difference from the first embodiment is that in the second embodiment, frame interpolation (S13) is performed before speed adjustment (S6).

The main features of this embodiment can be summarized as follows:

The teaching data includes images of the robot (robot device 1) or the work environment.

The processor (CPU 23) performs frame interpolation of the images contained in the segmented teaching data, and then performs correction to align the speeds of similar robot movements (frame interpolation unit 101). This makes it easy to reduce the variation in the teaching data over time, even if the teaching data includes images of the robot (robot device 1) or the work environment.

(Third embodiment)
Next, a third embodiment of the present invention will be described with reference to FIGS. 12, 13, and 14, in which only teaching data effective for improving generalization performance is extracted.

Figure 12 is a block diagram showing an example configuration of a robot control system 100 according to a third embodiment of the present invention. Note that in Figure 12, the same reference numerals as in Figure 1 indicate the same components, and therefore a repeated description will be omitted. The difference from the first embodiment is that in the third embodiment, the robot control system 100 is provided with a screen operation unit 5. The screen operation unit 5 can display the processing results of the motion planning unit. In addition, various parameters in the motion planning unit 3 are determined based on operation input received from the user.

Figure 13 is a block diagram showing an example configuration of the motion planning unit 3 in the third embodiment of the present invention. Note that in Figure 13, the same reference numerals as in Figure 3 indicate the same components, and therefore a repeated description will be omitted. The difference from the first embodiment is that in the third embodiment, the motion planning unit 3 is equipped with a screening device 131. The screening device 131 extracts only sensor data that is effective in improving generalization performance from the sensor data stored in the sensor data storage device 31. The extracted teaching data is then output to the consistency verification device 32 and the screen operation unit 5.

[Hardware configuration example]
Next, the hardware configuration of the motion planning unit 3 included in the robot control system 100 according to the third embodiment of the present invention will be described with reference to Fig. 14. In Fig. 14, the same reference numerals as in Fig. 3 indicate the same components, and therefore a repeated description will be omitted. The difference from the first embodiment is that the third embodiment includes a video output interface 141.

The video output interface 141 may be, for example, a VGA (Video Graphics Array), a DVI (Digital Visual Interface), an HDMI (High-Definition Multimedia Interface, registered trademark), or a Display Port. The video output interface 141 is configured to be able to transmit video to a display via a dedicated line or the like.

[Example of functional configuration]
Next, an example of the functional configuration of the screening device 131 of the motion planning unit 3 included in the robot control system 100 according to the third embodiment of the present invention will be described with reference to FIG.

Figure 15 shows an example of the functional configuration of a screening device 131 embodying the present invention. Note that in Figure 15, the same reference numerals as in Figure 13 indicate the same parts, so repeated explanations will be omitted.

In the screening device 131 shown in Figure 15, the grouping unit 151 groups the sensor data stored in the sensor data storage device 31 so that data with similar movements are grouped together. One possible method for implementing the grouping unit is to group sensor data for the same target hand position, for example.

The representative data calculation unit 152 calculates time series data (called representative data) that represents each group obtained by the grouping unit 151. One possible method for implementing the representative data calculation unit 152 is to use, as representative data, time series data obtained by calculating the median for each time point for all time series data within a group.

For each group obtained by the grouping unit 151, the outlier data detection unit 153 detects time series data (referred to as outlier data) that is not similar to the representative data calculated by the representative data calculation unit 152. Possible methods for implementing the outlier data detection unit include, for example, using DTW (Dynamic Time Warping) and IQR (Interquartile Range). DTW is an index of the similarity between time series data. DTW can take values greater than or equal to 0, with values closer to 0 indicating greater similarity between the time series data. First, a DTW series is obtained by calculating the DTW between the representative data and all time series data within a group. Next, the IQR of the calculated DTW series is calculated. IQR is an index that indicates the degree of dispersion of data and is calculated by subtracting the first quartile from the third quartile. Finally, outlier data is defined as time series data whose DTW for the representative data is greater than (third quartile + α x IQR). The default value of α is set to 1.5. The user can also change this value using the screen operation unit 5.

The calculation result output unit 154 outputs sensor data other than the outlier data to the consistency verification device 32. It also outputs information about the outlier data to the screen operation unit 5.

Next, an example of the functional configuration of the screen operation unit 5 provided in the robot control system 100 in the third embodiment of the present invention will be described with reference to FIG. 16.

Figure 16 shows an example of the functional configuration of the screen operation unit 5 embodying the present invention. Note that in Figure 16, the same reference numerals as in Figure 1 indicate the same components, so repeated explanations will be omitted.

In the screen operation unit 5 shown in FIG. 16, the operation input unit 161 is composed of input devices such as a mouse and keyboard, and accepts mouse input, keyboard input, etc. from the user.

The screen display unit 162 is composed of, for example, a display, and visualizes information obtained from the operation input unit 161 and the motion planning unit 3.

The screen control unit 163 receives information from the action planning unit 3 and the operation input unit 161. It also outputs information to the action planning unit 3 and the screen display unit 162.

The main features of this embodiment can be summarized as follows:

The processor (CPU 23) performs screening by removing data that deviates from the teaching data (screening device 131). This can further improve generalization performance.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described with reference to FIG. 17, in which an arbitrary motion speed and/or an arbitrary magnitude of force is realized when generating a motion.

Figure 17 is a block diagram showing an example of the configuration of a motion planning unit in the fourth embodiment of the present invention. Note that in Figure 17, the same reference numerals as in Figure 3 indicate the same parts, and therefore a repeated explanation will be omitted. The difference from the first embodiment is that in the fourth embodiment, the motion planning unit 3 is equipped with a motion parameter adjustment device 171. The motion parameter adjustment device 171 realizes an arbitrary motion speed and/or an arbitrary force magnitude when generating motion from the sensor data stored in the sensor data storage device 31.

[Hardware configuration of the operation parameter adjustment device]
The hardware configuration of the motion planning unit 3 included in the robot control system 100 in the fourth embodiment of the present invention is the same as that in the third embodiment of the present invention, and therefore will not be described again.

[Example of functional configuration of operation parameter adjustment device]
Next, an example of the functional configuration of the operation parameter adjustment device 171 of the operation planning unit 3 included in the robot control system 100 according to the fourth embodiment of the present invention will be described with reference to FIG.

Figure 18 shows an example of the functional configuration of an operation parameter adjustment device 171 embodying the present invention. Note that in Figure 18, the same reference numerals as in Figures 12 and 13 indicate the same components, so repeated explanations will be omitted.

In the operation parameter adjustment device 171 shown in FIG. 18, the operation parameter storage unit 181 stores the values of parameters required for operation parameter adjustment. Parameter types include, for example, the control period of the operation generation model, the range of current values, and the range of torque sensor values. Parameter values can be changed by the user via the screen operation unit 5.

The motion parameter adjustment calculation unit 182 adjusts the motion speed and/or the magnitude of force when generating motion based on the parameter values stored in the motion parameter storage unit 181. A specific method for changing the motion speed in the motion parameter adjustment calculation unit is, for example, changing the control period when generating motion. For example, if you want to increase the motion speed, you can make the control period when generating motion shorter than the sampling period of the teaching data. A specific method for changing the magnitude of force in the motion parameter adjustment calculation unit is, for example, changing the current value or torque sensor value. For example, if you want to decrease the magnitude of force, you can make the upper limit of the current value or torque sensor value smaller than the teaching data.

The main features of this embodiment can be summarized as follows:

The processing device (computer 20) includes a storage device (operation parameter storage unit 181) that stores operation parameters that indicate parameters for adjusting the operation of the robot (robot device 1) after machine learning. The storage device is composed of, for example, RAM 24, non-volatile storage 25, etc.

The processor (CPU 23) uses the trained machine learning model to generate command values for the robot's (robot device 1) movement based on the movement parameters (machine learning device 34). This allows the robot's movement to be adjusted without having to repeat teaching and learning.

The movement parameter is, for example, the sampling period of the teaching data. The processor (CPU 23) increases the movement speed of the robot (robot device 1) by making the control period of the robot shorter than the sampling period in the command value, or decreases the movement speed of the robot by making the control period of the robot longer than the sampling period in the command value (movement parameter adjustment device 171). This makes it possible to adjust the movement speed of the robot without having to repeat teaching and learning.

The operating parameter is, for example, the maximum value M of the robot's torque or a value correlated thereto in the teaching data. The processor (CPU 23) increases the robot's force by increasing the upper limit of the robot's (robot device 1) torque or a value correlated thereto (for example, the value of the actuator's drive current) in the command value above the maximum value M, or decreases the robot's force by decreasing the upper limit of the robot's torque or a value correlated thereto below the maximum value M in the command value. This makes it possible to adjust the robot's force without having to repeat teaching or learning.

The present invention is not limited to the above-described embodiments, and includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

Furthermore, some or all of the above-mentioned configurations, functions, processing units, etc. may be implemented in hardware, for example by designing them as integrated circuits. Broad processor devices such as FPGAs (Field Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits) may also be used as hardware.

Furthermore, each component of the motion planning unit 3 according to each of the above-described embodiments may be implemented in the control unit 4. Furthermore, the processing performed by a processing unit of the motion planning unit 3 may be realized by a single piece of hardware, or may be realized by distributed processing using multiple pieces of hardware.

In each of the above-described embodiments, the sensor data acquisition unit 2, the motion planning unit 3, and the control unit 4 are implemented by a single computer 20, for example, but they may also be implemented by separate computers. Implementing the motion planning unit 3, which has a heavy computational load, on a high-performance computer can improve overall throughput. The computers are connected to each other via a communication network such as a LAN or the Internet.

The present invention may also be embodied in the following ways:

(1) A machine learning system that performs machine learning to generate robot movements, comprising: an acquisition unit that acquires teaching data (such as time-series sensor information) for the robot; a movement segmentation unit that segments the teaching data based on the robot's movements; a speed adjustment unit that corrects the teaching data to align the speeds of similar movements among the segmented movements of different teaching data that share a common task when the speeds of the same movements differ; a timing adjustment unit that corrects the teaching data to align the timing of similar movements among the segmented movements of different teaching data that share a common task when the timings of the same movements differ; and a dynamics adjustment unit that corrects the robot's dynamics in the teaching data, and the teaching data with at least one of the speed, timing, and dynamics corrected is used as learning data for the machine learning system.

(2) In (1), the machine learning system is a machine learning system equipped with an acquisition unit that acquires the robot's position, joint angle information, etc. as teaching data for the robot.

(3) In (2), the machine learning system is a machine learning system that is equipped with an acquisition unit that acquires images of the robot and the work environment in addition to the robot's position and joint angle information.

(4) In (2), the machine learning system is a machine learning system that includes an acquisition unit that acquires the position and posture information of the target object in addition to the robot's position and joint angle information.

(5) In (1), the machine learning system is a machine learning system equipped with an acquisition unit that acquires the robot's speed, joint angular velocity information, etc. as teaching data for the robot.

(6) In (1), the machine learning system is a machine learning system equipped with an acquisition unit that acquires tactile information from the robot as teaching data for the robot.

(7) In (1), the machine learning system is a machine learning system equipped with a dynamics adjustment unit that adjusts acceleration and/or acceleration changes among the robot's dynamics.

(8) In (1), the machine learning system is a machine learning system equipped with a dynamics adjustment unit that adjusts the jerk and/or jerk change of the robot's dynamics.

(9) In (1), the machine learning system is a machine learning system equipped with a dynamics adjustment unit that adjusts torque and/or torque changes among the robot's dynamics.

(10) In the machine learning system of (1), when the teaching data for the robot includes camera images, the speed adjustment unit performs frame interpolation between the teaching data and then corrects the teaching data to align the speeds of the same type of movement.

(11) In (1), the machine learning system is equipped with a screening device that extracts, from the acquired teaching data, teaching data that is effective in improving behavioral generalization performance.

(12) In (1), the machine learning system is a machine learning system equipped with a motion parameter adjustment device that adjusts the control period of the robot to achieve an arbitrary motion speed when generating motion using a trained machine learning model.

(13) In (1), the machine learning system is a machine learning system equipped with a motion parameter adjustment device that adjusts the torque of the robot to achieve a desired force magnitude when generating motion using a trained machine learning model.

According to (1) to (13), by reducing the temporal variation between teaching data that hinders motion learning, it is expected that both the implementation effort can be reduced and generalization performance can be achieved.

1...Robot device, 2...Sensor data acquisition unit, 3...Motion planning unit, 4...Control unit, 5...Screen operation unit, 22...ROM, 23...CPU, 24...RAM, 25...Non-volatile storage, 26...Input/output interface, 27...Network interface, 31...Sensor data storage device, 32...Consistency verification device, 33...Learning data storage device, 34...Machine learning device, 41...Motion segmentation unit, 42...Speed adjustment unit, 43...Timing adjustment unit, 44...Dynamics adjustment unit, 45...Data Processing unit, 61...machine learning model definition unit, 62...learning unit, 63...trained weight accumulation unit, 64...inference unit, 101...frame interpolation unit, 131...screening device, 161...operation input unit, 151...grouping unit, 152...representative data calculation unit, 153...outlier data detection unit, 154...calculation result output unit, 161...operation input unit, 162...screen display unit, 163...screen control unit, 171...operation parameter adjustment device, 181...operation parameter storage unit, 182...operation parameter adjustment calculation unit

Claims (15)

Segmenting the teaching data for the robot into groups of similar movements of the robot;
correcting the segmented teaching data to align the speed or timing of the same type of motion of the robot;
Combining the corrected teaching data;
A processing device comprising a processor that performs machine learning using the synthesized teaching data. 2. The processing device according to claim 1,
The processor:
A processing device characterized by performing correction for smoothing dynamics values of the same type of motion of the robot on the segmented plurality of pieces of teaching data. 3. The processing device according to claim 2,
The teaching data is
position information indicating the position of the robot, joint angle information indicating joint angles,
a processing device including at least one of velocity information indicating the velocity of the robot, joint angular velocity information indicating the joint angular velocity of the robot, and tactile information indicating a sensor value of a tactile sensor provided on the robot. 4. The processing device according to claim 3,
The teaching data is
A processing device comprising an image of the robot or the working environment. 4. The processing device according to claim 3,
The teaching data is
A processing device comprising at least one of position information indicating a position of an object on which the robot is to perform work, and posture information indicating a posture of the object. 3. The processing device according to claim 2,
The dynamics are
Acceleration, change in acceleration,
Jerk, jerk change,
The processing device includes at least one of torque and torque change. 5. The processing device according to claim 4,
The processor:
A processing device characterized by performing frame interpolation of images included in the segmented teaching data for the plurality of pieces of teaching data, and then performing correction to align the speeds of similar movements of the robot. 2. The processing device according to claim 1,
The processor:
A processing device characterized in that screening is performed by removing data that deviates from the teaching data. 2. The processing device according to claim 1,
a storage device that stores operation parameters indicating parameters for adjusting the operation of the robot after machine learning;
The processor:
A processing device that generates a command value for the robot's operation based on the operation parameters using a trained machine learning model. 10. The processing device according to claim 9,
the operation parameter is a sampling period of the teaching data,
The processor:
a processing device characterized in that, in the command value, the control period of the robot is made shorter than the sampling period to increase the speed of the robot's movement, or, in the command value, the control period of the robot is made longer than the sampling period to decrease the speed of the robot's movement. 10. The processing device according to claim 9,
the operation parameter is a maximum value of a torque of the robot or a value correlated therewith in the teaching data,
The processor:
a processing device characterized in that, in the command value, the force of the robot is increased by making the upper limit of the torque of the robot or a value correlated thereto greater than the maximum value, or, in the command value, the force of the robot is decreased by making the upper limit of the torque of the robot or a value correlated thereto less than the maximum value. 2. The processing device according to claim 1,
The processor:
A processing device characterized by quantizing the segmented multiple pieces of teaching data and removing noise to make the speeds of similar movements of the robot uniform. 2. The processing device according to claim 1,
The processor:
A processing device characterized by performing corrections to align the timing of similar movements of the robot by increasing or decreasing the rest time of the robot for the plurality of segmented teaching data. A robot control system including the processing device according to claim 1 and a robot,
The processor generates a command value for the robot's operation using a trained machine learning model;
The robot control system is characterized in that the robot operates in accordance with the command value. A step of segmenting teaching data for a robot into groups of similar operations of the robot;
a step of correcting the segmented teaching data to align the speed or timing of the same type of motion of the robot;
a step of synthesizing the corrected teaching data;
performing machine learning using the synthesized training data;
Machine learning methods, including