JP7726730B2 - Computer system and model learning method - Google Patents

Description

The present invention relates to learning models for controlling robots that perform tasks including grasping and moving objects.

With advances in AI technology, systems are emerging that use AI to automatically control robots that perform tasks including grasping objects and moving items. AI that automatically controls robots is realized by models generated through machine learning.

Learning a model requires a large amount of training data. Obtaining training data from a robot presents the following problems. First, it takes time to obtain the training data, which lengthens the training time. Second, the robot must be operated for long periods of time, and there is a possibility that the robot may be forced to perform unreasonable movements, which could lead to breakdowns.

The technology described in Patent Document 1 is known to address the above-mentioned issues. Patent Document 1 states, "First, a loose pile of multiple virtual objects is generated using a virtual environment (101). Then, holding positions where a virtual robot device will hold the virtual objects are generated (102). The virtual robot device is then caused to perform an action to pick up the loosely piled virtual objects (104). Machine learning of the results of having the virtual robot device perform the action to pick up the virtual object is used to generate a trained model that can output the priorities of multiple possible holding positions from a specific position and orientation of the object (200). This trained model is then used to have an actual robot manipulator pick up the object. Re-learning is then performed to update the priorities of the multiple holding positions of the trained model using machine learning of the results of the pick-up action (203)."

Japanese Patent Application Laid-Open No. 2021-13996

The technology described in Patent Document 1 does not take trajectories into account, and therefore cannot generate models for controlling robots that perform tasks including grasping and moving objects. The present invention aims to provide a system and method that generates highly accurate models that are robust to various trajectories while reducing learning time.

A representative example of the invention disclosed in the present application is as follows: That is, a computer system for learning a model for controlling a robot that performs a task including grasping and moving an object includes at least one computer having an arithmetic unit, a storage device connected to the arithmetic unit, and an interface connected to the arithmetic unit, wherein the at least one computer performs a first process of receiving input of information related to the task and an initial model, a second process of setting a virtual environment consisting of a virtual workspace, virtual equipment, and a virtual robot based on the information related to the task, and setting a first trajectory for the virtual environment, and a third process of moving the virtual robot along the first trajectory in the virtual environment using the initial model. and a third process of updating the initial model so that the task time is short and the task success probability is high based on a machine learning algorithm using the results of the simulation, wherein in the second process, the at least one computer randomly sets the virtual environment and the first trajectory, and in the first learning process, the at least one computer repeatedly executes the second process and the third process until a learning termination condition is met, and records the initial model when the learning termination condition is met as a first model.

The present invention makes it possible to generate highly accurate models that are robust to a variety of trajectories while minimizing learning time. Issues, configurations, and advantages other than those described above will become clearer in the following description of the embodiments.

FIG. 1 illustrates an example of the configuration of a system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of robot configuration information according to the first embodiment. FIG. 2 is a diagram illustrating an example of a data structure of equipment configuration information according to the first embodiment. FIG. 3 is a diagram illustrating an example of a data structure of trajectory information according to the first embodiment. FIG. 4 is a diagram illustrating an example of a data structure of environment adjustment information according to the first embodiment. FIG. 10 is a diagram illustrating an example of a data structure of model management information according to the first embodiment. 10 is a flowchart illustrating an example of a learning process executed by a computer according to a first embodiment. 10 is a flowchart illustrating an example of a learning process executed by a computer according to a first embodiment. 10 is a flowchart illustrating an example of a learning process executed by a computer according to a first embodiment. FIG. 10 is a diagram illustrating an example of a screen displayed by a computer according to the first embodiment. FIG. 10 is a diagram illustrating a specific example of processing executed by an environment setting unit according to the first embodiment. FIG. 10 is a diagram illustrating a specific example of processing executed by an environment setting unit according to the first embodiment. FIG. 10 is a diagram illustrating a specific example of processing executed by an environment setting unit according to the first embodiment. FIG. 10 is a diagram illustrating an example of a screen displayed by a computer according to the first embodiment.

The following describes an embodiment of the present invention with reference to the accompanying drawings. However, the present invention should not be construed as being limited to the description of the embodiment shown below. Those skilled in the art will readily understand that the specific configuration can be modified without departing from the spirit or intent of the present invention.

In the configuration of the invention described below, identical or similar components or functions are designated by the same reference numerals, and duplicate explanations will be omitted.

The terms "first," "second," "third," etc. used in this specification are used to identify components and do not necessarily limit the number or order.

The position, size, shape, and range of each component shown in the drawings may not represent the actual position, size, shape, and range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the position, size, shape, and range, etc., disclosed in the drawings, etc.

Figure 1 shows an example of the system configuration for Example 1.

The system consists of a robot 100 and a computer 101. The robot 100 and the computer 101 are connected directly or via a network.

The robot 100 performs tasks including grasping an object (workpiece) and moving it from a starting point to an end point based on control information output from the computer 101. The robot 100 is equipped with a group of working devices 110, a controller 111, and a measuring device 112.

The working device group 110 is a group of devices that realize the grasping and movement of objects, such as hands, links, and drive motors.

The controller 111 controls the working device group 110 based on control information received from the computer 101. For example, the controller 111 moves the hand by driving drive motors that function as joints and connect links in accordance with the control information. The controller 111 outputs operating status information to the computer 101, including the angles, angular velocities, and angular accelerations of the joints, as well as the torque and current values of the drive motors.

The measuring device 112 measures values to understand the state of an object due to work by the robot 100. The values output from the measuring device 112 are also referred to as work state information. The measuring device 112 is, for example, an acceleration sensor, a force sensor, a camera, a contact sensor, a current sensor, etc. The robot 100 may be equipped with multiple measuring devices 112 of different types for each measurement object. The present invention is not limited to the installation location or number of measuring devices 112.

The robot 100 may also transmit the operating status information and work status information together as a single piece of information.

The computer 101 generates information on the trajectory (trajectory information), which is the movement path of the robot 100, and also generates control information based on the trajectory information, operating status information, and work status information. The control information includes, for example, the following values:
(1) Next target angle, angular velocity , and angular acceleration of the control axis
(2) Torque and drive current of the drive motor
(3) Object target coordinates, movement speed, and acceleration

The computer 101 includes an arithmetic unit 120, a storage unit 121, a communication unit 122, an input unit 123, and an output unit 124. Each hardware element is connected via an internal bus.

The storage device 121 is a device, such as a memory, that stores programs and information executed by the computing device 120. The storage device 121 stores robot configuration information 140, equipment configuration information 141, trajectory information 142, environmental adjustment information 143, and model management information 144. The storage device 121 is also used as a work area.

Robot configuration information 140 is information about the configuration of the robot 100. The data structure of the robot configuration information 140 will be explained using Figure 2.

The equipment configuration information 141 is information about the equipment on which the robot 100 performs work. The data structure of the equipment configuration information 141 will be explained using Figure 3.

Trajectory information 142 is information about the trajectory, which is the movement path of robot 100. The data structure of trajectory information 142 will be explained using Figure 4.

Environment adjustment information 143 is information for adjusting the virtual environment in which a task simulation is performed. The data structure of environment adjustment information 143 will be explained using Figure 5.

Model management information 144 is information for managing models that generate control information. Here, a model is a function, a table, a neural network, or the like. In this embodiment, the model is a neural network. The data structure of model management information 144 will be explained using Figure 6.

The arithmetic device 120 is a device that controls the entire computer 101, and is, for example, a processor. The arithmetic device 120 executes programs stored in the memory device 121. By executing processing in accordance with the programs, the arithmetic device 120 operates as a functional unit (module) that realizes a specific function. In the following explanation, when processing is described using a functional unit as the subject, it indicates that the arithmetic device 120 is executing a program that realizes the functional unit. In this embodiment, the arithmetic device 120 functions as a trajectory information generation unit 130, an environment setting unit 131, a learning unit 132, a simulator 133, a model integration unit 134, and a control information generation unit 135.

The trajectory information generation unit 130 determines the trajectory of the hand based on input information such as the work content and the position of the object. The trajectory information generation unit 130 also determines the trajectory of the tool based on the robot configuration information 140, the equipment configuration information 141, and the hand trajectory. The trajectory information generation unit 130 generates trajectory information 142 indicating the determined trajectory. Note that the method of determining the trajectory is a well-known technique, so a detailed explanation will be omitted. The environment setting unit 131 sets the virtual environment. The learning unit 132 executes learning processing. The simulator 133 simulates the movement of the robot 100 in the virtual environment. The model integration unit 134 integrates multiple models. The control information generation unit 135 controls the robot 100 performing work by generating control information based on the trajectory information 142, operating status information, work status information, and models.

Regarding each functional unit of the computer 101, multiple functional units may be combined into one functional unit, or one functional unit may be divided into multiple functional units for each function. For example, the learning unit 132 may include the functions of the environment setting unit 131.

The communication device 122 is a device for communicating with external devices, such as a NIC (Network Interface Card).

The input device 123 is a device for inputting data, commands, etc. into the computer 101, and is, for example, a keyboard, mouse, touch panel, etc.

The output device 124 is a device for outputting the calculation results of the calculator 101, and is, for example, a display, projector, printer, etc.

Figure 2 shows an example of the data structure of robot configuration information 140 in Example 1.

Robot configuration information 140 stores entries including ID 201, classification 202, item 203, and content 204.

ID 201 is a field that stores the identification information of the entry. Classification 202 is a field that stores the classification of the elements that make up the robot 100. Item 203 is a field that stores the management item of the element. Content 204 is a field that stores the content of the management item. Content 204 stores files, numbers, character strings, etc.

For links, the shape of the links is managed. For joints that connect links, the connecting links, the joint type, and constraints on joint movement are managed. Note that for joints, the items managed as constraints on joint movement differ depending on the joint type.

Figure 3 shows an example of the data structure of the equipment configuration information 141 in Example 1.

The equipment configuration information 141 stores entries including an ID 301, equipment name 302, attachment target 303, relative position 304, and relative orientation 305.

ID 301 is a field that stores the identification information of the entry. Equipment name 302 is a field that stores the name of the equipment. Mounting target 303 is a field that stores the name of the object to be mounted on the equipment. Relative position 304 is a field that stores information (e.g., coordinates) indicating the installation position relative to the equipment of the mounting target. Relative orientation 305 is a field that stores information (e.g., coordinates) indicating the orientation relative to the equipment of the mounting target.

An entry with ID 301 of "0" indicates that the position and orientation of other equipment are determined based on assembly cell A. An entry with ID 301 of "1" stores the relative position and orientation of RoboA installed with respect to assembly cell A.

Figure 4 shows an example of the data structure of orbit information 142 in Example 1.

Trajectory information 142 stores a table 400 showing the hand trajectory and a table 410 showing the tool trajectory.

Table 400 stores entries including via points 401, positions 402, attitudes 403, and times 404.

Way point 401 is a field that stores identification information for way points on the trajectory. Position 402 is a group of fields that store values indicating the coordinates of the hand at the way point. Each field of position 402 stores a value in the Cartesian coordinate system. Orientation 403 is a group of fields that store values indicating the orientation of the hand at the way point. Each field of orientation 403 stores a value defined by a quaternion. Time 404 is a field that stores the time at which the hand, having started moving from the starting point, reaches the way point.

Table 410 stores entries including waypoints 411, attitudes 412, and times 413.

Way point 411 is the same field as way point 401. Posture 412 is a group of fields that store values indicating the posture of each joint at the way point. Each field of posture 412 stores the angle of the joint. Time 413 is the same field as Time 404.

The trajectory information 142 includes a set of table 400 and table 410 for each task. If the robot 100 main body moves, the trajectory information 142 may also include a table relating to the trajectory of the robot 100 main body.

Figure 5 shows an example of the data structure of the environment adjustment information 143 in Example 1.

Environment adjustment information 143 stores entries (adjustment data) including ID 501, adjustment target 502, and adjustment item 503.

ID 501 is a field that stores the identification information of the entry. Adjustment target 502 is a field that stores the identification information of the object to be adjusted in the environment settings. Examples of adjustment targets include assembly cells, workpieces, and obstacles. Adjustment item 503 is a field that stores the item to be adjusted and the adjustment range.

Figure 6 shows an example of the data structure of model management information 144 in Example 1.

Model management information 144 stores entries (model data) including ID 601, model 602, task 603, learning time (simulation) 604, learning time (actual machine) 605, task time 606, and success probability 607.

ID 601 is a field that stores the identification information of the entry. Model 602 is a field that stores the model entity (data). The path to the storage destination of the model may also be stored. Task 603 is a field that stores the type of task. In this embodiment, one or more models are generated for one task. Learning time (simulation) 604 is a field that stores the processing time of the learning process by simulation. Learning time (actual machine) 605 is a field that stores the processing time of the learning process using the actual robot 100. Task time 606 is a field that stores the time required for the robot 100 to complete the task. Success probability 607 is a field that stores the probability of success of the task by the robot 100.

In this embodiment, the computer 101 learns the model using the following procedure.

(Learning process 1) The computer 101 uses the initial model and simulator 133 to learn the model (primary model).

(Learning process 2) The computer 101 applies the model generated by learning using the simulator 133 to the actual robot 100, and actually moves the robot 100 to learn the model (secondary model).

(Learning process 3) The computer 101 integrates the primary model and the secondary model, and uses the integrated model and the simulator 133 to learn the model (primary model).

Computer 101 repeatedly executes (Learning Process 2) and (Learning Process 3) in response to a request from the user.

Each learning process will be explained below using a flowchart. Figures 7, 8, and 9 are flowcharts illustrating an example of the learning process executed by the computer 101 of Example 1. Figure 10 is a diagram showing an example of a screen displayed by the computer 101 of Example 1. Figures 11A, 11B, and 11C are diagrams showing specific examples of processes executed by the environment setting unit 131 of Example 1. Figure 12 is a diagram showing an example of a screen displayed by the computer 101 of Example 1.

First, we will explain (Learning Process 1) using Figure 7.

The learning unit 132 displays a screen 1000 such as that shown in FIG. 10 via the output device 124 and accepts user input regarding the type of work, the equipment and robot 100 to be used, the model to be used, etc. (Step S101). The learning unit 132 may also accept input regarding the adjustment data to be used.

Screen 1000 includes a display field 1001, selection fields 1002 and 1004, load buttons 1003 and 1005, a model selection field 1006, and an execute button 1007.

The display field 1001 is a field for displaying the virtual environment set by the environment setting unit 131. The selection field 1002 is a field for selecting the equipment to be used for the work. When the read button 1003 is operated, values related to the equipment specified in the selection field 1002 are read from the equipment configuration information 141. The selection field 1004 is a field for selecting the work to be performed by the robot 100. When the read button 1005 is operated, the tables 400 and 410 corresponding to the work are read. In addition, entries related to the work are read from the environment adjustment information 143.

The model selection field 1006 is a field for selecting a model to use for learning. The model selection field 1006 displays entries including ID 1011, model 1012, work time 1013, and selection 1014. One entry corresponds to one model. Note that an entry may include fields other than those mentioned above.

ID 1011, Model 1012, and Working Time 1013 are the same fields as ID 601, Model 602, and Working Time 606. Selection 1014 is a field for selecting the model to use for learning. In learning process 1, the initial model is selected. In learning processes 2 and 3, a model other than the initial model is selected.

When an initial model is selected and the execute button 1007 is operated, the processing described below is executed. First, the learning unit 132 calls the environment setting unit 131 and instructs it to set a base virtual environment. The environment setting unit 131 sets the base virtual environment using the robot configuration information 140 and the equipment configuration information 141 (step S102). The environment setting unit 131 stores information about the base virtual environment in a work area and notifies the learning unit 132 that processing is complete.

Specifically, the environment setting unit 131 acquires data on the equipment, robot 100, workpiece, etc. related to the work from the robot configuration information 140 and the equipment configuration information 141 based on user input. Based on the acquired data, the environment setting unit 131 sets a base virtual environment consisting of a virtual work space, virtual equipment, a virtual robot, a virtual workpiece, etc. Note that the base virtual environment may also include virtual obstacles.

The learning unit 132 acquires model data for the model specified by the user and the environment adjustment information 143 from the model management information 144 (step S103). Note that if the user has specified adjustment data, the learning unit 132 acquires the specified adjustment data from the environment adjustment information 143.

The learning unit 132 starts the learning process (step S104).

The learning unit 132 calls the environment setting unit 131 and instructs it to set up a virtual environment. The environment setting unit 131 sets up a virtual environment based on the base virtual environment and the environment adjustment information 143 (step S105). The environment setting unit 131 also randomly sets the start and end points of a trajectory in the virtual work space (step S106). The environment setting unit 131 stores information about the virtual environment, the start and end points in a work area, and notifies the learning unit 132 that processing is complete.

For example, the environment setting unit 131 randomly changes the size and position of equipment and obstacles included in the base virtual environment, and also randomly changes the size and weight of work, based on the environment adjustment information 143.

As shown in Figures 11A, 11B, and 11C, the environment setting unit 131 randomly sets the size and position of obstacles included in the virtual environment, the size and weight of the workpiece, and the start and end points of the trajectory to generate different trajectories. This makes it possible to efficiently learn control that corresponds to multiple trajectories, i.e., a model that achieves robust control for any trajectory. In addition, because the sensor values measured in different virtual environments also change, it is also possible to absorb sensor characteristic errors that occur between the actual machine and the virtual environment.

The learning unit 132 calls the trajectory information generation unit 130 and instructs it to generate trajectory information 142. At this time, the learning unit 132 inputs information about the virtual facility and robot 100, as well as information about the start and end points of the trajectory, to the trajectory information generation unit 130. The trajectory information generation unit 130 generates trajectory information 142 based on the information input by the learning unit 132 (step S107) and stores it in the work area. The method for generating trajectory information 142 is a well-known technique, so a detailed explanation will be omitted.

The learning unit 132 starts the control process (step S108). The control process is repeatedly executed until a series of tasks, such as grasping the object and moving the object along the trajectory path, is completed. At this time, the learning unit 132 outputs information about the virtual environment and trajectory information 142 to the simulator 133.

The learning unit 132 acquires operating status information and work status information from the simulator 133, which simulates the movement of the virtual robot 100 (step S109). Note that step S109 may be omitted at the start point of the trajectory.

The learning unit 132 generates control information by inputting the operating status information, work status information, and trajectory information 142 into a model, and outputs the control information to the virtual robot 100 (step S110). The simulator 133 simulates the movement of the virtual robot 100 based on the control information, and outputs the operating status information and work status information.

If the work is not complete, the learning unit 132 returns to step S108 and performs the same processing. If the work is complete, the learning unit 132 ends the control processing (step S111).

The learning unit 132 updates the model based on the operating status information, work status information, etc. acquired during the control process (step S112). For example, the model is updated using a gradient descent method, etc. Note that the present invention is not limited to the learning algorithm.

If the learning termination condition is not met, the learning unit 132 returns to step S104 and executes the same processing. For example, if the index indicating accuracy is greater than a threshold value or if the number of learning iterations is greater than a threshold value, the learning unit 132 determines that the learning termination condition is met. If the learning termination condition is met, the learning unit 132 terminates the learning process (step S113) and ends the series of learning processes. At this time, the learning unit 132 evaluates the accuracy of the model and updates the model management information 144. Specifically, the learning unit 132 calculates the task time and task success probability under control using the model as indicators for evaluating the accuracy of the model. The learning unit 132 adds an entry to the model management information 144 and sets identification information to the ID 601 of the added entry. The learning unit 132 sets the generated model to the model 602 of the added entry, sets the task type to the task 603, and sets the time required for learning to the learning time (simulation) 604. The learning unit 132 sets the learning time (actual machine) 605 of the added entry to "0". The learning unit 132 also sets the work time 606 and success probability 607 of the added entry to the indices calculated in the accuracy evaluation.

By randomly changing the virtual environment and trajectory and training the model, it is possible to generate a model that achieves robust control over the environment and trajectory. Furthermore, because the actual robot 100 is not used, the learning process can be sped up and failure of the robot 100 can be avoided.

Note that the computer 101 may display a screen 1200 such as that shown in FIG. 12 while the learning process is being performed. Screen 1200 includes display fields 1201, 1202, 1203, 1204, and 1205, and a stop button 1206.

Display field 1201 displays the working status of the robot in the virtual environment. Display fields 1202 and 1203 display sensor values (simulation results) obtained from the robot working in the virtual environment. Display fields 1204 and 1205 display the learning progress, i.e., the accuracy evaluation of the model. Display field 1204 displays the progress of the work time over time, and display field 1205 displays the progress of the success probability of the work.

The user can refer to display fields 1204 and 1205, and if they determine that the model is sufficiently accurate, they can operate stop button 1206 to end the learning process.

Next, we will explain (Learning Process 2) using Figure 8.

The learning unit 132 presents the user with a screen 1000 for accepting input, and accepts user input regarding the type of work, the equipment and robot 100 to be used, the model to be used, etc. (step S201). The learning unit 132 also references the model management information 144 and displays in the model selection field 1006 the model for which the specified work type is set in the work 603.

The learning unit 132 references the model management information 144 to obtain the model specified by the user, and also obtains information about the robot and equipment from the robot configuration information 140 and equipment configuration information 141 (step S202).

The learning unit 132 calls the trajectory information generation unit 130 and instructs it to generate trajectory information 142. At this time, the learning unit 132 inputs information about the equipment and the robot 100 to the trajectory information generation unit 130. The trajectory information generation unit 130 generates trajectory information 142 based on the information input by the learning unit 132 (step S203) and stores it in the work area.

The learning unit 132 starts the learning process (step S204) and also starts the control process (step S205). The control process is repeatedly executed until a series of tasks, such as grasping the object and moving the object along the trajectory path, is completed.

The learning unit 132 acquires operating status information and work status information from the robot 100 (step S206). Note that step S206 may be omitted at the start point of the trajectory.

The learning unit 132 generates control information by inputting the operating status information, work status information, and trajectory information 142 into the model, and outputs the control information to the robot 100 (step S207).

If the work is not complete, the learning unit 132 returns to step S205 and executes the same process. If the work is complete, the learning unit 132 ends the control process (step S208).

The learning unit 132 updates the model based on the operating status information, work status information, etc. acquired during the control process (step S209).

If the learning termination condition is not met, the learning unit 132 returns to step S204 and executes the same processing. If the learning termination condition is met, the learning unit 132 terminates the learning process (step S210) and ends the series of learning processes. At this time, the learning unit 132 evaluates the accuracy of the model and updates the model management information 144. Specifically, the learning unit 132 calculates the task time and the task success probability. The learning unit 132 adds an entry to the model management information 144 and sets identification information in the ID 601 of the added entry. The learning unit 132 sets the generated model in the model 602 of the added entry and sets the task type in the task 603. The learning unit 132 sets the value of the learning time (simulation) 604 of the entry corresponding to the model acquired in step S202 to the learning time (simulation) 604 of the added entry. The learning unit 132 sets the time required for learning in the learning time (actual machine) 605 of the added entry. The learning unit 132 also sets the work time 606 and success probability 607 of the added entry to the indices calculated in the accuracy evaluation.

By relearning a model generated through learning using the simulator 133 using an actual machine, learning can be completed in a shorter time than conventional learning. Furthermore, by relearning a model using an actual machine, it is possible to generate a model that corresponds to the movements of actual work. In other words, accuracy can be improved.

Next, we will explain (Learning Process 3) using Figure 9.

The learning unit 132 presents the user with a screen 1000 for accepting input, and accepts user input regarding the type of work, the equipment and robot 100 to be used, the model to be used, etc. (Step S301). The learning unit 132 may also accept input regarding the adjustment data to be used. Note that in (Learning Process 3), the model generated in (Learning Process 1) and the model generated in (Learning Process 2) are specified.

The learning unit 132 calls the environment setting unit 131 and instructs it to set a base virtual environment. The environment setting unit 131 sets the base virtual environment using the robot configuration information 140 and the equipment configuration information 141 (step S302). The environment setting unit 131 stores information about the base virtual environment in a work area and notifies the learning unit 132 that processing is complete. The processing in step S302 is the same as the processing in step S102.

The learning unit 132 acquires the model data of the model designated by the user from the model management information 144 and the environment adjustment information 143 (step S303). The process of step S303 is the same as the process of step S103 .

The learning unit 132 calls the model integration unit 134 and instructs it to integrate the models. The model integration unit 134 integrates the model generated in (learning process 1) and the model generated in (learning process 2) (step S304), and notifies the learning unit 132 that the processing is complete. For example, if the model is a neural network, the model integration unit 134 replaces some of the weights of the model generated in (learning process 2) with the weights of the model generated in (learning process 1). In this embodiment, the models are integrated to avoid local solution problems and overfitting.

The processing from step S305 to step S314 is the same as the processing from step S104 to step S113. However, the method for updating the model management information 144 is different.

The learning unit 132 adds an entry to the model management information 144 and sets identification information in the ID 601 of the added entry. The learning unit 132 sets the generated model in the model 602 of the added entry and sets the type of work in the work 603. The learning unit 132 sets the values of the learning time (simulation) 604 and learning time (actual machine) 605 of the entry corresponding to the model generated in (learning process 2) in the learning time (simulation) 604 and learning time (actual machine) 605. The learning unit 132 adds the time required for learning to the value of the learning time (simulation) 604 of the added entry. The learning unit 132 also sets the indicators calculated in the accuracy evaluation in the work time 606 and success probability 607 of the added entry.

Note that in step S304, multiple models using different integration methods may be generated. In this case, steps S305 to S314 are performed for each model.

By re-learning a model generated through training using a real aircraft using simulator 133, a highly accurate model that is robust to the environment and trajectory can be generated in a short amount of time.

When actually controlling the robot 100, the user accesses the computer 101 and refers to the model management information 144. The user selects a model to use based on the model management information 144. For example, if the user places importance on the operation time, the user selects a model based on the operation time, and if the user places importance on the stability of the operation, the user selects a model based on the success probability. The control information generator 135 controls the robot 100 using the selected model.

It is also possible to treat multiple task groups as a single task. For example, a product manufacturing task consisting of "transporting parts" and "assembling parts" could be considered. In this case, a model is selected for each elemental task that makes up the task. The control information generator 135 switches models for each elemental task to control the robot 100.

Although one computer 101 controls the robot 100, a computer system including multiple computers 101 may perform similar control. In this case, functional units may be distributed across multiple computers 101.

The present invention is not limited to the above-described embodiments, but includes various modifications. Furthermore, for example, the above-described embodiments provide detailed descriptions of the configurations in order to clearly explain the present invention, and the present invention is not necessarily limited to those that include all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with another configuration.

Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be implemented in part or in whole in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor in the computer reads the program code stored on the storage medium. In this case, the program code read from the storage medium itself implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C/C++, Perl, Shell, PHP, Python, and Java (registered trademark).

Furthermore, the software program code that realizes the functions of the embodiments may be distributed via a network and stored on a storage means such as a computer's hard disk or memory, or on a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored on the storage means or storage medium.

In the above examples, the control lines and information lines shown are those considered necessary for explanation, and not all control lines or information lines in the product are necessarily shown. All components may be interconnected.

100 Robot 101 Computer 110 Working device group 111 Controller 112 Measuring device 120 Arithmetic device 121 Storage device 122 Communication device 123 Input device 124 Output device 130 Trajectory information generation unit 131 Environment setting unit 132 Learning unit 133 Simulator 134 Model integration unit 140 Robot configuration information 141 Equipment configuration information 142 Trajectory information 143 Environment adjustment information 144 Model management information

Claims (12)

A computer system for learning a model for controlling a robot that performs a task including grasping and moving an object, comprising:
at least one computer having a computing device, a storage device connected to the computing device, and an interface connected to the computing device;
The at least one computer
a first process for receiving input of information about a task and an initial model;
a second process of setting a virtual environment including a virtual workspace, a virtual facility, and a virtual robot based on the information about the work, and setting a first trajectory for the virtual environment;
a third process of using the initial model to simulate a control process for the task, including movement of an object by the virtual robot along the first trajectory in the virtual environment, and updating the initial model based on a machine learning algorithm using a result of the simulation, so that the task time is short and the task success probability is high;
In the second process, the at least one computer randomly sets the virtual environment and randomly sets the first trajectory;
A computer system characterized in that in the first learning process, the at least one computer repeatedly executes the second process and the third process until the learning termination condition is met, and records the initial model when the learning termination condition is met as the first model. 2. The computer system of claim 1,
storing environment adjustment information for setting the virtual environment;
A computer system characterized in that, in the second processing, the at least one computer randomly sets the virtual environment based on the information related to the task and the environment adjustment information. 2. The computer system of claim 1,
The at least one computer
a fourth process for receiving input of information about the work and the first model to be used;
a fifth process of setting a second trajectory based on the information about the work;
a sixth process of using the first model to perform a control process for the task including movement of an object by the robot along the second trajectory, and updating the first model based on a machine learning algorithm using a result of the control process so that the task time is short and the task success probability is high;
A computer system characterized in that in the second learning process, the at least one computer repeatedly executes the sixth process until the learning termination condition is met, and records the first model as the second model when the learning termination condition is met. 4. The computer system according to claim 3,
The at least one computer
a seventh process of receiving input of information about the work, the first model to be used, and the second model to be used;
an eighth process of integrating the first model and the second model;
The second process;
a ninth process of performing a simulation of a control process for the task, including movement of an object by the virtual robot along the first trajectory, in the virtual environment using an integrated model obtained by integrating the first model and the second model, and updating the integrated model based on a machine learning algorithm using a result of the simulation , so that the task time is short and the success probability of the task is high;
In the third learning process, the at least one computer repeatedly executes the second process and the ninth process until the learning termination condition is met, and records the integrated model when the learning termination condition is met as the third model. 5. The computer system of claim 4,
in the first learning process, the at least one computer calculates a task time for the task and a success probability for the task in the first model as indices related to accuracy of the first model, and records data associating the processing time for the first learning process with the indices of the first model in model management information;
in the second learning process, the at least one computer calculates a task time for the task and a success probability for the task in the second model as indices related to accuracy of the second model, and records data associating the processing time for the second learning process with the indices of the second model in the model management information;
in the third learning process, the at least one computer calculates a task time for the task and a success probability for the task in the third model as indices related to accuracy of the third model, and records data associating the processing time for the third learning process with the indices of the third model in the model management information;
The computer system is characterized in that the at least one computer presents the model management information to a user. 6. The computer system according to claim 5,
A computer system, wherein the initial model, the first model, the second model, and the third model are neural networks. A method for learning a model for controlling a robot that performs a task including grasping and moving an object, the method being executed by a computer system, comprising:
the computer system includes at least one computer having a computing device, a storage device connected to the computing device, and an interface connected to the computing device;
The model learning method includes:
The at least one computer
a first process for receiving input of information about a task and an initial model;
a second process of setting a virtual environment including a virtual workspace, a virtual facility, and a virtual robot based on the information about the work, and setting a first trajectory for the virtual environment;
a third process of using the initial model to simulate a control process for the task, including movement of an object by the virtual robot along the first trajectory in the virtual environment, and updating the initial model based on a machine learning algorithm using a result of the simulation, so that the task time is short and the task success probability is high;
In the second process, the at least one computer randomly sets the virtual environment and randomly sets the first trajectory;
A model learning method characterized in that, in the first learning process, the at least one computer repeatedly executes the second process and the third process until a learning termination condition is met, and when the learning termination condition is met, the initial model is recorded as a first model. A method for training a model according to claim 7, comprising:
the computer system holds environmental adjustment information for setting the virtual environment;
A model learning method, wherein in the second process, the at least one computer randomly sets the virtual environment based on the information about the task and the environment adjustment information. A method for training a model according to claim 7, comprising:
The at least one computer
a fourth process for receiving input of information about the work and the first model to be used;
a fifth process of setting a second trajectory based on the information about the work;
a sixth process of using the first model to perform a control process for the task including movement of an object by the robot along the second trajectory, and updating the first model based on a machine learning algorithm using a result of the control process so that the task time is short and the task success probability is high;
A model learning method characterized in that, in the second learning process, the at least one computer repeatedly executes the sixth process until a learning termination condition is met, and when the learning termination condition is met, the first model is recorded as a second model. A method for training a model according to claim 9, comprising:
The at least one computer
a seventh process of receiving input of information about the work, the first model to be used, and the second model to be used;
an eighth process of integrating the first model and the second model;
The second process;
a ninth process of performing a simulation of a control process for the task, including movement of an object by the virtual robot along the first trajectory, in the virtual environment using an integrated model obtained by integrating the first model and the second model, and updating the integrated model based on a machine learning algorithm using a result of the simulation , so that the task time is short and the success probability of the task is high;
A model learning method characterized in that, in the third learning process, the at least one computer repeatedly executes the second process and the ninth process until a learning termination condition is met, and when the learning termination condition is met, the integrated model is recorded as a third model. A method for training a model according to claim 10, comprising:
in the first learning process, the at least one computer calculates a task time for the task and a success probability for the task in the first model as indices related to accuracy of the first model, and records data associating the processing time for the first learning process with the indices of the first model in model management information;
in the second learning process, the at least one computer calculates a task time for the task and a success probability for the task in the second model as indices related to accuracy of the second model, and records data associating the processing time for the second learning process with the indices of the second model in the model management information;
in the third learning process, the at least one computer calculates a task time for the task and a success probability for the task in the third model as indices related to accuracy of the third model, and records data associating the processing time for the third learning process with the indices of the third model in the model management information;
The model learning method is characterized in that the at least one computer presents the model management information to a user. A method for training a model according to claim 11, comprising:
A model learning method, wherein the initial model, the first model, the second model, and the third model are neural networks.