JP6978722B2 - Learning device, learning control method, and its program - Google Patents

Description

The present invention relates to a learning device, a learning control method, and a program thereof.

Conventionally, research on artificial intelligence technology such as neural networks (hereinafter referred to as "AI technology") has been widely conducted (see, for example, Patent Document 1). In particular, with the rise of AI technology called deep learning, for example, the recognition rate of object recognition technology using images has improved rapidly in the last few years, and the level of image classification exceeds the human recognition rate. Is reaching. Deep learning technology is not limited to image recognition, but also voice recognition, personal authentication, behavior prediction, sentence summarization, automatic translation, monitoring, automatic driving, failure prediction, sensor data analysis, song genre judgment, content generation, It is expected to be applied to security systems and a wide range of other fields.

In machine learning such as deep learning, it is possible to have a machine perform learning to acquire a predetermined ability. At this time, the learning device that performs machine learning repeatedly executes the learning operation until a predetermined ability is acquired.

For example, Patent Document 1 discloses a learning control method for a robot. In the learning control method described in Patent Document 1, the driving unit of the robot is based on an error that occurs between a target trajectory that is a target of robot motion preset by a person and an actual trajectory when the robot actually operates. Correct the input value supplied to.

Japanese Unexamined Patent Publication No. 6-289918

In learning devices that control actuators based on a large number of sensor information, such as automobile engine and driving control, or chemical plants, control and sensor information output affect each other, so a control method is acquired. In order to do so, we need to do more complicated learning. Therefore, in a learning device that performs such complicated learning, it is not easy for a person to set a target value of a controlled amount in advance as in Patent Document 1. On the other hand, if the learning device is made to perform learning without setting a target value, it is necessary to repeat a large number of trial errors, which is inefficient.

Therefore, an object of the present invention is to provide a technique for shortening the time required for a learning device to achieve a learning purpose without human intervention.

The learning device according to one aspect of the present invention is a learning device that learns the control of a series of operations related to a predetermined task, divides the task into a plurality of scenes, and performs a series of operations in each of the divided scenes. Among them, a classification unit for specifying a partial motion performed in the scene and a partial learning unit for learning the control of the partial motion for each divided scene are provided.

According to the above configuration, the learning device can classify a series of actions related to learning into smaller units according to the divided scenes. As a result, learning can be performed for each scene, so that the learning purpose can be achieved in a shorter period of time.

Further, the learning device may include an output unit that outputs the classification result of the partial operation. According to this aspect, the classification of the learned partial motion can be utilized in other learning devices.

Further, the learning device may include a control unit that rearranges the classified partial movements or the divided scenes according to the progress order of the tasks. According to this aspect, the classified partial movements can be rearranged and learned in the order of scenes according to the task.

Further, the automatic driving control learning device according to one aspect of the present invention is an automatic driving control learning device that learns control about a series of movements related to automatic driving of a vehicle orbiting a predetermined course, and is with respect to a predetermined reference position. The course is divided into a plurality of scenes based on the position or orientation on the course, and each of the divided scenes is divided into a classification unit that specifies a partial movement performed in the scene among a series of movements. It is equipped with a partial learning unit that learns the control of partial movements for each scene.

Further, the robot control learning device according to one aspect of the present invention is a robot control learning device that learns control about a series of operations related to a task of grasping a predetermined work and stacking it in a mounting place according to the shape of the work. Therefore, the task is divided into a plurality of scenes based on the displacement amount of the movement related to the learning of the task, and in each of the divided scenes, a classification that specifies a partial movement performed in the scene among a series of movements. It includes a unit and a partial learning unit that learns the control of partial motion for each divided scene.

Further, the learning method according to one aspect of the present invention is a learning method for learning the control of a series of operations related to a predetermined task executed by a computer provided with a control unit, and the control unit performs a task in a plurality of situations. In each of the divided scenes, a step of specifying a partial operation performed in the scene and a step of learning the control of the partial operation are executed for each divided scene.

Further, the program according to one aspect of the present invention divides a task into a plurality of scenes on a computer that learns control of a series of movements related to a predetermined task, and in each of the divided scenes, the said one of the series of movements. The procedure for specifying the partial motion performed in the scene and the procedure for learning the control of the partial motion for each divided scene are executed.

Further, the device according to one aspect of the present invention is a device that executes a predetermined task, and is controlled by a first sensor that senses information about a scene in which the device performs a predetermined task, an actuator, and an actuator. The second sensor that senses information about the operation of the device, the control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor, and the learning result performed by the above learning device are stored. Memory unit and
The control unit determines the control amount according to the sensor values output from the first sensor and the second sensor based on the learning result stored in the storage unit.

Further, the automatic travel control device according to one aspect of the present invention is an automatic travel control device that orbits a predetermined course, and is a first sensor that senses information on a position and orientation of the vehicle on a predetermined course, and a vehicle. The actuator that controls the drive, the second sensor that senses the control state of the vehicle by the actuator, the control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor, and the above learning device. A storage unit for storing the performed learning result is provided, and the control unit is a control amount according to the sensor values output from the first sensor and the second sensor based on the learning result stored in the storage unit. To determine.

According to the present invention, it is possible to provide a technique for shortening the time required for a learning device to achieve a learning object without human intervention.

It is a block diagram which shows the schematic structure of the learning apparatus in 1st Embodiment. It is a schematic diagram which shows the course in which the vehicle controlled by the learning device in 1st Embodiment automatically travels. It is a flowchart which shows the outline of the processing of the learning apparatus in 1st Embodiment. It is a block diagram which shows the detailed structure of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the processing of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the processing of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the processing of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the processing of the learning apparatus in 1st Embodiment. It is a figure which shows an example of the hardware composition of the learning apparatus in 1st Embodiment. It is a block diagram which shows the schematic structure of the learning apparatus in 2nd Embodiment.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same elements are designated by the same reference numerals, and duplicate description will be omitted. Further, the following embodiments are examples for explaining the present invention, and the present invention is not intended to be limited only to the embodiments thereof. Further, the present invention can be modified in various ways as long as it does not deviate from the gist thereof.

<1. System overview>
The outline of the system in this embodiment will be described with reference to FIGS. 1 to 3.
FIG. 1 is a block diagram showing a schematic configuration of the learning device 1 according to the present embodiment. The learning device 1 learns a predetermined task. The learning device 1 according to the present embodiment is mounted on an automatic driving control vehicle (hereinafter, also simply referred to as “vehicle”) 90 as an example, and controls the vehicle 90 for automatically traveling on a predetermined course (see FIG. 2). To learn. At this time, learning data is given to the learning device 1 by, for example, an operator or the like. The learning data is data including, for example, the following learning objectives and learning requirements.

(Learning purpose)
・ Complete 10 laps of the course within the specified time to reach the goal.
(Learning requirements)
・ Do not go out of the course ・ Clockwise in the lap direction ・ Goal ・ At the initial stage level, “Go around the course once”

It should be noted that the task is an operation related to learning (the "operation related to learning" in the present embodiment is various controls necessary for the automatic traveling of the vehicle 90. It is considered that the operation is executed by the vehicle 90 by the various controls. It may be achieved.), And in this embodiment, it is to go around the course. In addition, the purpose of learning is the level at which the task should be achieved, and in the present embodiment, as described above, "to finish 10 laps of the course within a predetermined time". Then, in the present embodiment, it is considered that the learning requirement is that the task can be performed in the learning at the initial stage level.

Further, in the following description, the learning device 1 will be described as being composed of a computer such as a PC (Personal Computer) or a server device, but the learning device 1 is not limited to this, and is not limited to this, for example, any set having a processor, RAM, and ROM. It may be realized by a built-in device. Further, the configuration implemented in each device is not limited to the configuration realized by software. Any configuration included in each device may be a configuration realized by hardware. For example, the neural network 22 described later may be configured by an electronic circuit such as a custom LSI (Large-Scale Integration) or an FPGA (Field-Programmable Gate Array).

As shown in FIG. 1, the learning device 1 has a control unit 10, a machine learning unit 20, an operation classification unit 30, and a storage unit 40.

The control unit 10 is connected to the control sensor 91, the actuator 92, and the state detection sensor 93 provided outside the learning device 1 in the vehicle 90. The control unit 10 controls the actuator 92 in response to the outputs from the control sensor 91 and the state detection sensor 93 to automatically drive the vehicle 90.

The control sensor 91 is a group of sensors for automatically controlling the traveling of the vehicle 90. For example, the control sensor 91 is composed of an in-vehicle camera, an external obstacle detection sensor such as a laser, a road surface condition detection sensor, and the like. On the other hand, the state detection sensor 93 is a group of sensors that detect the control state of the automatically traveling vehicle 90. For example, the state detection sensor 93 is composed of a vibration sensor, a noise sensor, a fuel consumption detection sensor, a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and the like.

The actuator 92 is controlled by the control unit 10 in order to automatically drive the vehicle 90. The actuator 92 is composed of, for example, an accelerator actuator, a brake actuator, a steering actuator, and the like. The accelerator actuator controls the driving force of the vehicle by controlling the throttle opening degree according to the control signal from the control unit 10. The brake actuator controls the braking force on the wheels of the vehicle by controlling the operation amount of the brake pedal according to the control signal from the control unit 10. The steering actuator controls the driving of the steering assist motor of the electric power steering system in response to the control signal from the control unit 10 to control the steering action of the vehicle.

Next, the procedure for the learning device 1 to perform learning will be roughly described with reference to FIG. The details of the processing of each step will be described later. FIG. 3 is a flowchart showing an outline of a processing flow when the learning device 1 performs learning. First, as the initial stage of learning (S1), learning is performed for the purpose of being able to perform a task (that is, being able to perform an operation that satisfies the learning requirements of the initial stage). Since the learning device 1 in the present embodiment learns the control for automatically traveling a predetermined course, "to go around the course and reach the goal" is given as a learning requirement at the initial stage.

After clearing the purpose of the initial stage level, the operation classification (S2) is performed next. At this stage, by analyzing the learning content performed in the initial stage of learning in S1, the task is divided into a plurality of parts based on predetermined parameters (hereinafter, the divided tasks are also referred to as "scenes") and divided. In each of the scenes performed, the movement performed in the scene among a series of movements related to the task (hereinafter, also referred to as "partial movement"). Predetermined parameters for dividing a task are, for example, the displacement amount of the operation related to the learning of the task, the environment for executing the operation related to the learning of the task (elapsed time from the start time of the task, and the start location of the task). Position, etc.). In the present embodiment, the position from the start place of the task (environment for executing the operation related to the learning of the task) is used as a predetermined parameter. That is, in the present embodiment, the learning device 1 divides the task into scenes based on the position on the course, and a series of movements related to learning are performed based on the movements performed in the course units corresponding to the divided scenes. It is classified into scenes. By learning in partial motion units classified according to the situation, it is possible to improve the efficiency of learning. The efficiency of learning may mean, for example, shortening the time required from the start of learning to the achievement of the learning purpose.

When the motions are classified, as the next step, learning of control (S3) leading to a state in which learning cannot be continued is performed for each classified partial motion. Here, the state in which learning cannot be continued means a state in which the movement to be learned cannot be continued by learning. In the present embodiment, the state in which learning cannot be continued is, for example, a state in which the student goes out of the course, crashes into a wall or the like and becomes stuck, or breaks down. By learning the control leading to the learning continuation impossible state in advance, it is possible to avoid falling into the learning continuation impossible state in learning the optimum control in a later step. This makes it possible to perform learning more efficiently.

In the final stage of learning (S4), learning is optimized. At this stage, learning is performed in which the partial movements learned by classifying each scene are combined, and then the movements are optimally performed from the start to the end. In the present embodiment, as the final stage of learning, learning to complete 10 laps of the course within a predetermined time is performed.

<2. Detailed processing>
Next, the details of the processing of the learning device 1 in each step will be described with reference to FIGS. 4 to 8. FIG. 4 is a block diagram showing a detailed configuration of the learning device 1 according to the present embodiment. As shown in FIG. 4, the machine learning unit 20 includes a learning data input / output unit 21, a neural network 22, and a learning result output unit 23. Further, the motion classification unit 30 is composed of a control data extraction unit 31 and an motion classification result extraction unit 32.
In the following, the details of the processing of each part will be described for each step of FIG.

(2-1. Initial stage of learning)
FIG. 5 is a flowchart showing a detailed processing flow in the initial stage of learning of S1 shown in FIG. First, in the initial stage of learning (first learning), the learning data input / output unit 21 receives the learning data (S101). The learning data is, for example, data including the above-mentioned learning purpose and learning requirement.

In the next step (S102), machine learning is performed. In the present embodiment, since the conditions for limiting the individual control operations are not specified in advance, the learning device 1 itself learns the control operations. Specifically, the control unit 10 sets a random control amount for the actuator 92 and operates the actuator 92. At this time, since the vehicle 90 cannot naturally run along the course, the vehicle 90 will run randomly while going off the course. The control unit 10 reads an output (hereinafter, also referred to as “sensor value”) from the control sensor 91 and the state detection sensor 93 for a randomly given control amount, and stores these data (control amount and sensor value). Store in part 40. The neural network 22 refers to the storage unit 40, reads the stored control amount and the sensor value, and learns the control operation adapted to the learning requirement by Deep Learning (S102).

In the learning requirements, "to go around the course and reach the goal" is set as the purpose of the initial stage level. Therefore, in the learning device 1, it is determined that the machine learning has reached the initial stage level at the time when it is determined that the goal has been reached by making one round of the course based on the output from the control sensor 91, for example (S103: Y). , Finish the initial learning.

(2-2. Classification of movement)
FIG. 6 is a flowchart showing a detailed processing flow in the operation classification of S2 shown in FIG. First, in performing the motion classification process, the control data extraction unit 31 determines the sensor value of the control sensor 91 at the end of the learning initial stage, the control amount of the actuator 92, and the sensor value of the state detection sensor 93. Extract from the storage unit 40 (S201). The control data extraction unit 31 inputs each extracted value to the neural network 22 as learning data.

Next, the neural network 22 performs machine learning based on the learning data input by the control data extraction unit 31 (S202). At this time, the neural network 22 classifies the orbiting motion into a predetermined number of scenes.

The process of classifying the orbiting motion into the scene by the neural network 22 will be described in more detail. The neural network 22 classifies the orbiting motion into scenes based on the scene vector and the motion vector. In the present embodiment, the scene vector represents a scene of a task performed by the vehicle 90. The scene vector is acquired from, for example, a sensor value output by the control sensor 91 (for example, a position (or distance) from the start point and a direction from the start point). As an example, assuming x and y coordinates with the starting point as the origin, the scene vector at the point l can be represented by _{(l x} , _y).

On the other hand, the motion vector represents the control state of the traveling vehicle 90. The motion vector is acquired from, for example, a sensor value (for example, speed, acceleration, angular velocity, angular acceleration, etc.) output by the state detection sensor 93. As an example, the motion vector at a certain point l is represented by _{(v l} , a _{l) using the velocity v and the acceleration a at the point l.}

The neural network 22 divides the task into scenes based on the scene vector (l _x , _y ), and based on the motion vector (v _l , a _l ), the motion to be learned in the scene for each divided scene. Learn classification. As a result, the learning device 1 can learn the control of the partial motion according to the scene by determining which scene the learning device 1 is currently in. As an example, the neural network 22 grasps the acceleration / deceleration / change of direction of the motion of the vehicle 90 by paying attention to the change point of the motion vector in addition to the position represented by the scene vector, and is a series based on the change point. It is possible to classify the movements of the above into movements according to the situation. Further, for example, the neural network 22 can learn the classification of motion based on the similarity of motion vectors.

In the example of the course shown in FIG. 2, the task is divided into five scenes from A to O based on the position on the course. The partial operations classified into each scene are as follows, for example.
Scene a: First straight partial operation (for example, control of deceleration timing, running position, etc. when approaching the next first corner)
Scene a: Partial movement of the first corner (for example, control of steering wheel operation at a corner, acceleration timing when entering the second straight, etc.)
Scene c: Second straight partial operation (for example, control of deceleration timing, running position, etc. when approaching the next second corner)
Scene d: Partial movement of the second corner (for example, control of steering wheel operation at the corner, acceleration timing when entering the third straight, etc.)
Scene E: Partial movement of the third straight (for example, control of acceleration when entering the first straight)

It is preferable that the neural network 22 can rearrange the divided scenes according to the order of progress of the tasks. For example, in the present embodiment, the neural network 22 can rearrange the scenes in the order in which the positions on the course are closer to the start position (start position) of the task along the traveling direction of the vehicle 90.

The motion classification result extraction unit 32 extracts the classification of the partial motion learned by the neural network 22 and stores it in the storage unit 40 (S203). At this time, the learning result output unit 23 can output the learning result related to the classification of the partial motion. As a result, the neural network 22 can omit the learning related to the classification of the partial motion by receiving the classification result of the partial learning as the learning data from, for example, another learning device 1'having the same configuration.

(2-3. Learning of control leading to a state where learning cannot be continued)
FIG. 7 is a flowchart showing a detailed processing flow in the learning (second learning) of the control leading to the learning continuation impossible state of S3 shown in FIG. First, the learning data input / output unit 21 refers to the storage unit 40, selects one of the partial operations classified in the processing of S2, and controls the actuator 92 required for the partial operation. To extract. Further, the learning data input / output unit 21 executes control with the controlled amount extracted with reference to the storage unit 40, and as a result, whether or not the learning continuation impossible state is reached is based on, for example, the output from the state detection sensor 93. To judge. The learning data input / output unit 21 reads out the extracted control amount and the information as a result of whether or not the learning continuation impossible state is reached as learning data, and gives it to the neural network 22 as learning data. The neural network 22 performs learning by Deep Learning based on the given learning data (S301).

At this time, the learning result output unit 23 can output the learning result of the control leading to the learning continuation impossible state. As a result, the neural network 22 can receive, for example, from another learning device 1'having a similar configuration, the control that has reached the state in which learning cannot be continued as learning data, and can perform additional learning (S302). This makes it possible to perform more efficient learning. The process of S302 is not an essential process.

The learning device 1 performs the processing of S301 (and S302) for all the classified partial operations (S303).

Although not essential, the learning device 1 can also learn again through a series of movements after learning the control leading to the learning continuation impossible state for all the classified partial movements (S304). This makes it possible to perform faster lap control.

As described above, the learning device 1 according to the present embodiment can learn the classified partial motions by first learning the control leading to the state in which learning cannot be continued, thereby avoiding the control in the subsequent learning. become. This makes it possible to perform more efficient learning.

(2-4. Optimization learning)
FIG. 8 is a flowchart showing a detailed processing flow in the optimization learning (third learning) of S4 shown in FIG. In the optimized learning, the learning purpose given as learning data at the start of learning by optimizing the learning performed in the steps up to S3 (in the present embodiment, "10 laps of the course within a predetermined time". To achieve the goal. ”). In the optimized learning, the learning is performed excluding the control leading to the learning continuation impossible state learned in S3. At this time, the learning data input / output unit 21 refers to the storage unit 40 and extracts the learning data (set by the operator) input in the initial stage of learning (S1 in FIG. 3). Further, the learning data input / output unit 21 further refers to the storage unit 40 to extract the state of the neural network 22 after learning the control leading to the learning continuation impossible state. The learning data input / output unit 21 sets these extracted data in the control unit 10.

The control unit 10 outputs a control amount for the actuator 92 based on the set data described above, and acquires the sensor values of the control sensor 91 and the state detection sensor 93 for the control amount. The control unit 10 stores the given control amount and the sensor value output to the control amount in the storage unit 40.

The neural network 22 reads out the control amount and the sensor value stored in the control unit 10 in the above process, and performs learning by deep learning (S401). As a result, the neural network 22 becomes more efficient in the control operation that adapts to the learning requirement from the start to the end of the operation (that is, from the start to the goal of the course) in the state where the control leading to the learning continuation impossible state is learned. You can learn well. The process of S401 is repeated until the entire learning is optimized (S402). The result of the optimization learning is extracted by the learning result output unit 23 and stored in the storage unit 40. As a result, in the optimized learning, the learning can be performed by excluding the control leading to the learning continuation impossible state.

As described above, according to the learning device 1 according to the present embodiment, the learning device 1 itself can classify the movements related to learning into partial movements and perform learning. As a result, individual optimization can be achieved for each classified operation, so learning can be performed more efficiently. Further, according to the learning device 1 according to the present embodiment, when learning the control of the partial motion, first, the control leading to the learning continuation impossible state is learned. As a result, it is possible for a person to efficiently perform learning without setting detailed conditions for each operation in advance.

(Hardware configuration)
With reference to FIG. 9, an example of the hardware configuration in the case where the learning device 1 described above is realized by the computer 800 will be described. It should be noted that the configuration of each device can be realized by dividing it into a plurality of devices.

As shown in FIG. 9, the computer 800 includes a processor 801 and a memory 803, a storage device 805, an input interface unit (input I / F unit) 807, a data interface unit (data I / F unit) 809, and a communication interface unit (communication). The I / F unit) 811 and the display device 813 are included.

The processor 801 controls various processes in the computer 800 by executing a program stored in the memory 803. For example, by executing the program stored in the memory 803 by the processor 801, the control unit 10, the machine learning unit 20, the operation classification unit 30, and the like of the learning device 1 can be realized.

The memory 803 is a storage medium such as, for example, a RAM (Random Access Memory). The memory 803 temporarily stores the program code of the program executed by the processor 801 and the data required when the program is executed.

The storage device 805 is, for example, an auxiliary storage device such as a hard disk drive (HDD) or a solid state drive, or a non-volatile storage medium such as a flash memory. The storage device 805 stores an operating system and various programs for realizing each of the above configurations. Such programs and data are referenced from the processor 801 by being loaded into the memory 803 as needed. For example, the above-mentioned storage unit 40 is realized by the storage device 805.

The input I / F unit 807 is a device for receiving an input from the administrator. Specific examples of the input I / F unit 807 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, and the like. The input I / F unit 807 may be connected to the computer 800 via an interface such as USB (Universal Serial Bus).

The data I / F unit 809 is a device for inputting data from the outside of the computer 800. Specific examples of the data I / F unit 809 include a drive device for reading data stored in various storage media. It is also conceivable that the data I / F unit 809 is provided outside the computer 800. In that case, the data I / F unit 809 is connected to the computer 800 via an interface such as USB.

The communication I / F unit 811 is a device for performing data communication via the Internet N by wire or wirelessly with an external device of the computer 800. It is also conceivable that the communication I / F unit 811 is provided outside the computer 800. In that case, the communication I / F unit 811 is connected to the computer 800 via an interface such as USB.

The display device 813 is a device for displaying various information. Specific examples of the display device 813 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display device 813 may be provided outside the computer 800. In that case, the display device 813 is connected to the computer 800 via, for example, a display cable or the like.

[Second Embodiment]
In the first embodiment, an example in which the learning device 1 is used for the automatic driving control vehicle 90 has been described. However, the device to which the learning device 1 is applied is not limited to the example shown in the first embodiment, and can be applied to various devices. In this embodiment, an example applied to the control of a robot whose task is to perform a pick-and-place operation will be described. In the second embodiment, the differences from the first embodiment will be mainly described.

First, the difference between the system configuration according to the present embodiment and the first embodiment will be described with reference to FIG. The configuration of the learning device 1 is the same as that of the first embodiment. On the other hand, regarding the configuration outside the learning device 1, in the present embodiment, the control sensor 91'is composed of a group of sensors for performing a pick-and-place operation. Specifically, it is composed of a work detection sensor (image sensor), a robot gripping force sensor, and the like. Further, the control sensor 91'has an image recognition algorithm and can recognize the shape of the workpiece to be gripped. Other configurations outside the learning device 1 are the same as those in the first embodiment.

Next, the difference between the learning according to the present embodiment and the learning according to the first embodiment will be described.
The pick-and-place operation, which is a task according to the present embodiment, refers to an operation performed by the following procedure.
1. 1. Recognize and grip the work shape.
2. 2. Lift the gripped work.
3. 3. The lifted work is moved to a predetermined position according to the shape of the work.
4. Stack each work shape in the cylinder.

Further, in the learning of robot control according to the present embodiment, the learning objectives and learning requirements given are as follows.

(Learning purpose)
-From a container in which works with three different shapes (for example, three types of cylindrical work, quadrangular prism work, and triangular prism work) are stacked separately, the work shape is changed to the work shape within a predetermined time by pick-and-place operation. Stack 10 workpieces in a cylinder (circle, quadrangle, triangle) with corresponding entrances.
(Learning requirements)
・ Do not place the work in any position other than the specified position. ・ Stack 10 works for each work shape in the cylinder. ・ At the initial level, "Put one work in the cylinder of the appropriate work shape."

In this embodiment, the task is to stack the workpieces in a cylinder according to the shape. Further, in the present embodiment, the pick-and-place motion to be learned may be classified by the same procedure as that for classifying the course on which the vehicle 90 travels in the first embodiment. .. For example, in the present embodiment, the task corresponds to the scene corresponding to the movement of gripping the work, the scene corresponding to the movement of carrying the work, and the movement of stacking the work based on the displacement amount of the movement related to the learning of the task. The scene is divided into.

Further, in the present embodiment, the state in which learning cannot be continued means, for example, a state in which the work cannot enter the cylinder. Therefore, in the learning stage of the control leading to the state in which learning cannot be continued, the control to be learned is, for example, as follows.
・ Wrong place of placement (the shape of the work and the shape of the entrance of the cylinder are different)
・ The direction in which the workpieces are stacked is incorrect (the orientation of the workpiece and the orientation of the cylinder are different).

In the learning device 1 according to the present embodiment, by learning in advance the control leading to the above-mentioned learning continuation impossible state, the shape of the work and the shape of the cylinder are appropriately recognized, and the orientation when gripping the work is learned in advance. can do. As a result, in the final stage of learning, it is possible to avoid a state in which learning cannot be continued, so that learning efficiency can be further improved.
Other configurations are the same as those of the first embodiment.

The embodiment of the present invention has been described above. It should be noted that the present embodiment is for facilitating the understanding of the present invention, and is not for limiting the interpretation of the present invention. Further, the present invention can be changed or improved without deviating from the gist thereof. For example, each step in the above-mentioned processing flow can omit a part of each step within a range that does not cause a contradiction in the processing contents, or can arbitrarily change the order of each processing step or execute them in parallel.

In the above-described embodiment, an example in which the system according to the present invention is used to manage the capacity acquired by the machine by AI technology such as deep learning has been described, but the present invention is not limited to this and covers a wide range of fields. Can be applied. For example, it can be applied to the identification of good and defective products, various industrial fields such as food, machine parts, chemical products, chemicals, fishery field, agriculture field, forestry field, service industry, medical and health field. Further, the present invention may be applied to a case where AI technology is applied to a product in the embedded field, a system utilizing IT technology such as a social system, big data analysis, classification processing in a wide range of control devices, and the like.

In addition, in this specification, a "part", a "means", and a "procedure" do not simply mean a physical configuration, but also include a case where a process executed by the "part" or the like is realized by software. Further, even if one "part", "means", "procedure", or a process executed by the device is realized by two or more physical configurations or devices, the process performed by two or more "parts" or the device. May be realized by one physical means or device.

In addition, some or all of the above embodiments may be described as in the following appendix, but are not limited to the following.
(Appendix 1)
Equipped with at least one hardware processor
The hardware processor is
When learning to control a series of actions related to a given task,
The task is divided into a plurality of scenes, and in each of the divided scenes, a partial operation performed in the scene is specified in the series of operations.
The control of the partial motion is learned for each of the divided scenes.
(Appendix 2)
By at least one hardware processor
It is a step to learn the control of a series of movements related to a predetermined task.
A step of dividing the task into a plurality of scenes and specifying a partial operation performed in the scene among the series of operations in each of the divided scenes.
A step of learning the control of the partial motion for each of the divided scenes,
How to learn to perform.

1 Learning device 10 Control unit 20 Machine learning unit 21 Learning data input unit 21 Learning data input / output unit 22 Neural network 23 Learning result output unit 30 Operation classification unit 31 Control data extraction unit 32 Operation classification result extraction unit 40 Storage unit 90 Automatic running Control vehicle 91 Control sensor 92 Actuator 93 State detection sensor

Claims (9)

A learning device that learns the control of a series of movements related to a predetermined task.
Based on the position and orientation information or motion information obtained from the control amount and sensor value when learning is performed until the motion that meets the learning requirements of the initial stage can be performed, the task is applied to multiple scenes. In each of the divided scenes, a classification unit that specifies a partial operation performed in the scene in the series of operations, and a classification unit.
The control of the partial motion is learned in advance so that the motion to be learned cannot be continued due to the learning for each of the specified partial motions, and then the divided scenes are described. A partial learning unit that learns the control of partial movements,
A learning device equipped with. The learning device is
The learning device according to claim 1, further comprising an output unit that outputs the classification result of the partial operation. The learning device is
A control unit that rearranges the classified partial movements or the divided scenes according to the progress order of the tasks.
The learning device according to claim 1, further comprising. It is an automatic driving control learning device that learns control about a series of operations related to automatic driving of a vehicle that goes around a predetermined course.
The position on the course or the position on the course with respect to a predetermined reference position determined by using the information on the position and orientation acquired from the control amount and the sensor value when learning is performed until the operation satisfying the learning requirements in the initial stage can be performed. Based on the orientation, the course is divided into a plurality of scenes, and in each of the divided scenes, a classification unit that specifies a partial movement performed in the scene among the series of movements, and a classification unit.
The control of the partial motion is learned in advance so that the motion to be learned cannot be continued due to the learning for each of the specified partial motions, and then the portion is divided for each of the divided scenes. A partial learning unit that learns motion control,
An automatic driving control learning device equipped with. It is a robot control learning device that learns control about a series of operations related to a task of grasping a predetermined work and stacking it in a placement place according to the shape of the work.
Based on the displacement amount of the movement during learning of the task, which is determined by using the control amount when learning until the movement satisfying the learning requirements in the initial stage and the information about the movement obtained from the sensor value are used. The task is divided into a plurality of scenes, and in each of the divided scenes, a classification unit that specifies a partial operation performed in the scene among the series of operations, and a classification unit.
The control of the partial motion is learned in advance so that the motion to be learned cannot be continued due to the learning for each of the specified partial motions, and then the portion is divided for each of the divided scenes. A partial learning unit that learns motion control,
A robot control learning device equipped with. It is a learning method for learning the control of a series of operations related to a predetermined task, which is executed by a computer equipped with a control unit.
The control unit
Based on the position and orientation information or motion information obtained from the control amount and sensor value when learning is performed until the motion that meets the learning requirements of the initial stage can be performed, the task is applied to multiple scenes. In each of the divided scenes, a step of specifying a partial operation performed in the scene in the series of operations, and a step of specifying the partial operation performed in the scene.
The control of the partial motion is learned in advance so that the motion to be learned cannot be continued due to the learning for each of the specified partial motions, and then the divided scenes are described. Steps to learn the control of partial movement,
How to learn to perform. To a computer that learns to control a series of actions related to a given task,
Based on the position and orientation information or motion information obtained from the control amount and sensor value when learning is performed until the motion that meets the learning requirements of the initial stage can be performed, the task is applied to multiple scenes. In each of the divided scenes, a procedure for specifying a partial operation performed in the scene in the series of operations, and a procedure for specifying a partial operation performed in the scene.
The control of the partial motion is learned in advance so that the motion to be learned cannot be continued due to the learning for each of the specified partial motions, and then the divided scenes are described. Procedures for learning to control partial movements,
A program to execute. A device equipped with the learning device according to any one of claims 1 to 3 and performing a predetermined task.
A first sensor that senses information about a scene in which the device performs a predetermined task,
Actuator and
A second sensor that senses information about the operation of the device controlled by the actuator, and
A control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor .
Equipped with a,
The control unit
Based on the learning result of the learning by prior Symbol partial learning unit, determines a control amount corresponding to the sensor value output from the first sensor and the second sensor,
Device. An automatic traveling control device equipped with the learning device according to any one of claims 1 to 3 and controlling a vehicle orbiting a predetermined course.
A first sensor that senses information about the position and orientation of the vehicle on the predetermined course,
The actuator that controls the drive of the vehicle and
A second sensor that senses the control state of the vehicle by the actuator,
A control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor .
Equipped with a,
The control unit
Based on the learning result of the learning by prior Symbol partial learning unit, determines a control amount corresponding to the sensor value output from the first sensor and the second sensor,
Automatic driving control device.

Images