JP6863082B2 - Learning device, learning control method, and its program - Google Patents

Description

The present invention relates to a learning device, a learning control method, and a program thereof.

Conventionally, research on artificial intelligence technology such as neural networks (hereinafter referred to as "AI technology") has been widely conducted (see, for example, Patent Document 1). In particular, with the rise of AI technology called deep learning, for example, the recognition rate of object recognition technology using images has improved rapidly in the last few years, and the level of image classification exceeds the human recognition rate. Is reaching. Deep learning technology is not limited to image recognition, but also voice recognition, personal authentication, behavior prediction, sentence summarization, automatic translation, monitoring, automatic driving, failure prediction, sensor data analysis, song genre judgment, content generation, It is expected to be applied to security systems and a wide range of other fields.

In machine learning such as deep learning, it is possible to have a machine perform learning to acquire a predetermined ability. At this time, the learning device that performs machine learning repeatedly executes the learning operation until a predetermined ability is acquired.

For example, Patent Document 1 discloses a learning control method for a robot. In the learning control method described in Patent Document 1, the driving unit of the robot is based on an error that occurs between a target trajectory that is a target of robot movement preset by a person and an actual trajectory when the robot actually moves. Correct the input value supplied to.

Japanese Unexamined Patent Publication No. 6-289918

In learning devices that control actuators based on a large amount of sensor information, such as automobile engine and running control, or chemical plants, control and sensor information output affect each other, so a control method is acquired. In order to do so, we need to do more complicated learning. Therefore, in a learning device that performs such complicated learning, it is not easy for a person to set a target value of a controlled amount in advance as in Patent Document 1. On the other hand, if the learning device is made to perform learning without setting a target value, it is necessary to repeat a large number of trial errors, which is inefficient.

Therefore, an object of the present invention is to provide a technique for shortening the time required for a learning device to achieve a learning purpose without human intervention.

The learning device according to one aspect of the present invention is a learning device that learns the control of movements related to a predetermined task, and is learning data including a learning purpose and an allowable requirement of learning movements performed when learning the control. A learning data receiving unit that accepts learning data, a neural network that executes learning based on learning data, and an output unit that outputs learning results by the neural network. The first learning is executed, and based on the result of the first learning, the second learning is executed to learn the control range in which the learning operation meets the permissible requirements, and the control range is within the control range based on the result of the second learning. In, the third learning for achieving the learning purpose is executed.

According to the above configuration, learning of the control range in which the learning operation meets the permissible requirements is performed before the third learning for achieving the learning purpose. As a result, the device itself can perform learning within a control range in which the learning operation meets the permissible requirements without being given a condition for limiting the control operation by a person, so that the learning purpose can be achieved in a shorter period of time. be able to.

Further, the output unit may output the result of the second learning. According to this aspect, the learning result of the control range in which the learning operation meets the permissible requirements can be utilized in other learning devices.

Further, the learning device is a learning device that learns the control of a series of movements related to a predetermined task. The task is divided into a plurality of scenes, and in each of the divided scenes, a line in the series of movements is performed. The neural network may execute the second learning and the third learning for each partial motion, further including a classification unit for specifying the partial motion.

According to this aspect, the learning device can classify the movements related to learning into partial movements, which are smaller units according to the scene, and learn each of the classified partial movements. As a result, the learning purpose can be achieved in a shorter period of time.

The automatic running control learning device according to one aspect of the present invention is an automatic running control learning device that learns control about a series of operations related to automatic running of a vehicle orbiting a predetermined course, and determines the course within a predetermined time. A learning data reception unit that accepts learning data including the learning purpose for the purpose of orbiting the number of times of learning, and the permissible requirements of the learning operation performed when learning the control, and a neural network that executes learning based on the learning data. And an output unit that outputs the learning result by the neural network, the neural network executes the first learning to achieve the ability to go around the course once, and based on the result of the first learning, the learning operation. Executes the second learning to learn the control range that meets the permissible requirements, and based on the result of the second learning, executes the third learning to achieve the learning purpose within the control range.

The robot control learning device according to one aspect of the present invention is a robot control learning device that learns control of a series of operations related to a task of transporting a predetermined object to a predetermined mounting location, stacking a predetermined number of objects, and packing the objects. The learning data reception unit that accepts learning data including the learning purpose for the purpose of completing a series of operations as soon as possible and the permissible requirements for the learning operation performed when learning the control, and the learning data. Based on this, a neural network that executes learning and an output unit that outputs learning results by the neural network are provided, and the neural network executes the first learning to achieve control of a series of operations related to a task. , Based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed, and based on the result of the second learning, the learning purpose is set within the control range. Perform a third learning to achieve.

The learning method according to one aspect of the present invention is a learning method for learning the control of an operation related to a predetermined task executed by a computer provided with a control unit, and when the control unit learns a learning purpose and control. The step of accepting the learning data including the permissible requirements of the learning operation performed in, the step of executing the learning based on the learning data, and the step of outputting the learning result by the step of executing the learning are executed and learned. The step of executing the first learning is to execute the first learning to achieve the initial stage of the learning purpose, and based on the result of the first learning, the second learning to learn the control range in which the learning operation meets the permissible requirements. Includes a step of executing and performing a third learning to achieve the learning objective within the control range based on the result of the second learning.

The program according to one aspect of the present invention is a procedure for receiving learning data including a learning purpose and a permissible requirement of a learning motion performed when learning the control into a computer that learns the control of the motion related to a predetermined task. Based on the learning data, the procedure for executing the learning and the procedure for outputting the learning result by the procedure for executing the learning are executed, and the procedure for executing the learning is the first learning for achieving the initial stage of the learning purpose. Is executed, and based on the result of the first learning, the second learning is executed to learn the control range in which the learning operation meets the permissible requirements, and based on the result of the second learning, the learning purpose is within the control range. Includes a procedure to perform a third learning to achieve.

The device according to one aspect of the present invention is a device that executes a predetermined task, and is a state of the first sensor that senses information necessary for the device to perform an operation, an actuator, and the device by the actuator. A second sensor that senses changes, a control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor, and a storage unit that stores the learning results performed by the above learning device. Based on the learning result stored in the storage unit, the control unit determines the control amount according to the sensor values output from the first sensor and the second sensor so as to be within the control range.

According to the present invention, it is possible to provide a technique for shortening the time required for a learning device to achieve a learning object without human intervention.

It is a block diagram which shows the schematic structure of the learning apparatus in 1st Embodiment. It is a schematic diagram which shows the course in which the vehicle controlled by the learning device in 1st Embodiment automatically travels. It is a flowchart which shows the outline of the process of the learning apparatus in 1st Embodiment. It is a block diagram which shows the detailed structure of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a figure which shows an example of the hardware composition of the learning apparatus in 1st Embodiment. It is a block diagram which shows the schematic structure of the learning apparatus in 2nd Embodiment.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same elements are designated by the same reference numerals, and duplicate description will be omitted. Further, the following embodiments are examples for explaining the present invention, and the present invention is not intended to be limited only to the embodiments thereof. Furthermore, the present invention can be modified in various ways as long as it does not deviate from the gist thereof.

<1. System overview>
The outline of the system in this embodiment will be described with reference to FIGS. 1 to 3.
FIG. 1 is a block diagram showing a schematic configuration of the learning device 1 according to the present embodiment. The learning device 1 learns a predetermined task. The learning device 1 according to the present embodiment is mounted on an automatic driving control vehicle (hereinafter, also simply referred to as “vehicle”) 90 as an example, and controls the vehicle 90 for automatically traveling on a predetermined course (see FIG. 2). To learn. At this time, learning data is given to the learning device 1 by, for example, an operator or the like. The learning data is data including, for example, the following learning objectives, learning requirements, and permissible requirements.
(Learning purpose)
・ Complete 10 laps of the course within the specified time to reach the goal.
(Learning requirements)
・ Do not go out of the course ・ Clockwise in the lap direction ・ Goal ・ At the initial stage level, “Go around the course once”
(Allowable requirements)
・ Do not touch anything else

The task is an operation related to learning (the "operation related to learning" in the present embodiment is various controls required for automatic traveling of the vehicle 90. It is considered that the operation is executed by the vehicle 90 by the various controls. It may be achieved.), And in this embodiment, it is to go around the course. In addition, the purpose of learning is the level at which the task should be achieved, and in the present embodiment, as described above, "to finish 10 laps of the course within a predetermined time". Then, in the present embodiment, it is considered that the learning requirement is that the task can be performed in the learning at the initial stage level.

Further, the permissible requirement is a requirement to be satisfied by the operation (learning operation) performed when the learning is performed. That is, when learning takes place, for example, it is permissible to go out of the course, but not to come into contact with something else. The learning purpose, learning requirements, and permissible requirements are not limited to this.

In the present embodiment, the learning data is input from the learning data input / output unit 21 described later by an operator or the like, but the allowable requirements may be automatically generated by the learning device 1 based on the specification values of each component. ..

In the following description, the learning device 1 will be described as being composed of a computer such as a PC (Personal Computer) or a server device, but the present invention is not limited to this, and for example, any device having a processor, RAM, ROM, or the like is used. It may be realized by an embedded device. Further, the configuration implemented in each device is not limited to the configuration realized by software. Any configuration included in each device may be a configuration realized by hardware. For example, the neural network 22 described later may be configured by an electronic circuit such as a custom LSI (Large-Scale Integration) or an FPGA (Field-Programmable Gate Array).

As shown in FIG. 1, the learning device 1 includes a control unit 10, a machine learning unit 20, an motion classification unit 30, and a storage unit 40.

The control unit 10 is connected to the control sensor 91, the actuator 92, and the state detection sensor 93 provided outside the learning device 1 in the vehicle 90. The control unit 10 controls the actuator 92 in response to the outputs from the control sensor 91 and the state detection sensor 93 to automatically drive the vehicle 90.

The control sensor 91 is a group of sensors for performing automatic driving control of the vehicle 90. For example, the control sensor 91 is composed of an in-vehicle camera, an external obstacle detection sensor such as a laser, a road surface condition detection sensor, and the like. On the other hand, the state detection sensor 93 is a group of sensors that detect the control state of the vehicle 90 that is automatically traveling. For example, the state detection sensor 93 is composed of a vibration sensor, a noise sensor, a fuel consumption detection sensor, a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and the like.

The actuator 92 is controlled by the control unit 10 to automatically drive the vehicle 90. The actuator 92 is composed of, for example, an accelerator actuator, a brake actuator, a steering actuator, and the like. The accelerator actuator controls the driving force of the vehicle by controlling the throttle opening degree according to the control signal from the control unit 10. The brake actuator controls the braking force on the wheels of the vehicle by controlling the operation amount of the brake pedal in response to the control signal from the control unit 10. The steering actuator controls the driving of the steering assist motor of the electric power steering system in response to the control signal from the control unit 10 to control the steering action of the vehicle.

Next, the procedure for the learning device 1 to perform learning will be roughly described with reference to FIG. The details of the processing of each step will be described later. FIG. 3 is a flowchart showing an outline of a processing flow when the learning device 1 performs learning. First, as the initial stage of learning (S1), learning is performed for the purpose of being able to perform a task (that is, being able to perform an operation that satisfies the learning requirements of the initial stage). Since the learning device 1 in the present embodiment learns the control for automatically traveling on a predetermined course, "one lap of the course and a goal" is given as a learning requirement at the initial stage.

After clearing the purpose of the initial stage, the operation classification (S2) is performed next. In this stage, the task is divided into a plurality of tasks based on predetermined parameters by analyzing the learning contents performed in the initial learning stage of S1 (hereinafter, the divided tasks are also referred to as "scenes") and divided. In each of the performed scenes, a process for specifying an operation (hereinafter, also referred to as "partial operation") performed in the scene among a series of operations related to the task is performed. Predetermined parameters for dividing a task are, for example, the displacement amount of the operation related to the learning of the task, the environment for executing the operation related to the learning of the task (elapsed time from the start time of the task, and the start location of the task). Relative position, etc.). In this embodiment, a position relative to the start location of the task is used as a predetermined parameter. That is, in the present embodiment, the learning device 1 divides the task into scenes based on the position on the course, and based on the movements performed in the course units corresponding to the divided scenes, a series of movements related to learning are scenes. are categorized. Learning can be made more efficient by learning in sub-motion units classified according to the situation. The efficiency of learning may mean, for example, shortening the time required from the start of learning to the achievement of the learning purpose.

When the movements are classified, as the next step, learning of the movement allowable range (S3) is performed for each classified partial movement. Here, the operation allowable range means an appropriate range of the control amount given to the controlled object to be learned by the control unit 10 during learning. More specifically, the motion permissible range is a control amount in which the learning motion meets the permissible requirements when the control unit 10 gives a predetermined control amount during learning to move the controlled object (learning motion). Refers to the range of. In the present embodiment, the operation allowable range is the result of the actuator 92 operating the vehicle 90 based on the given control amount when the control unit 10 gives a predetermined control amount to the actuator 92 for control. It is a range of control amount that allows the vehicle 90 to operate without touching anything else. Specifically, as an operation allowable range, the maximum value can be learned from the minimum value of the control amount of the actuator (control to the brake, accelerator, steering, etc.). By learning in advance the movement allowable range that satisfies the permissible requirement, it is possible to learn the optimum control in a later step while avoiding performing the movement that deviates from the permissible requirement. This makes it possible to perform learning more efficiently.

At the final stage of learning (S4), learning is optimized. At this stage, learning is performed in which the partial movements learned by classifying each scene are combined, and then the movements are optimally performed from the start to the end. In the present embodiment, as the final stage of learning, learning to complete 10 laps of the course within a predetermined time is performed.

<2. Detailed processing>
Next, the details of the processing of the learning device 1 in each step will be described with reference to FIGS. 4 to 8. FIG. 4 is a block diagram showing a detailed configuration of the learning device 1 according to the present embodiment. As shown in FIG. 4, the machine learning unit 20 includes a learning data input / output unit 21, a neural network 22, and a learning result output unit 23. Further, the motion classification unit 30 is composed of a control data extraction unit 31 and an motion classification result extraction unit 32.
In the following, the details of the processing of each part will be described for each step of FIG.

(2-1. Initial stage of learning)
FIG. 5 is a flowchart showing a detailed processing flow in the initial stage of learning of S1 shown in FIG. First, in the initial stage of learning (first learning), the learning data input / output unit 21 receives the learning data (S101). The learning data is, for example, data including the above-mentioned learning purpose, learning requirement, and allowable requirement.

In the next step (S102), machine learning is performed. In the present embodiment, since the conditions for limiting the individual control operations are not specified in advance, the learning device 1 itself learns the control operations. Specifically, the control unit 10 sets a random control amount with respect to the actuator 92 and operates the actuator 92. At this time, since the vehicle 90 cannot naturally travel along the course, the vehicle 90 will travel randomly while going off the course. The control unit 10 reads an output (hereinafter, also referred to as “sensor value”) from the control sensor 91 and the state detection sensor 93 for a randomly given control amount, and stores these data (control amount and sensor value). It is stored in the part 40. The neural network 22 refers to the storage unit 40, reads the stored control amount and the sensor value, and learns the control operation adapted to the learning requirement by deep learning (S102).

In the learning requirements, "to go around the course and reach the goal" is set as the purpose of the initial stage level. Therefore, in the learning device 1, it is determined that the machine learning has reached the initial stage level when it is determined that the goal has been reached by going around the course once based on the output from the control sensor 91, for example (S103: Y). , Finish the initial learning.

(2-2. Classification of movement)
FIG. 6 is a flowchart showing a detailed processing flow in the operation classification of S2 shown in FIG. First, in performing the motion classification process, the control data extraction unit 31 determines the sensor value of the control sensor 91 at the end of the initial learning stage, the control amount of the actuator 92, and the sensor value of the state detection sensor 93. Extract from the storage unit 40 (S201). The control data extraction unit 31 inputs each extracted value to the neural network 22 as learning data.

Next, the neural network 22 performs machine learning based on the learning data input by the control data extraction unit 31 (S202). At this time, the neural network 22 classifies the orbiting motion into a predetermined number of scenes.

The classification process of the neural network 22 into the scene of the orbiting motion will be described in more detail. The neural network 22 classifies the orbiting motion into scenes based on the scene vector and the motion vector. In the present embodiment, the scene vector represents a scene of a task performed by the vehicle 90. The scene vector is acquired from, for example, a sensor value output by the control sensor 91 (for example, a position (or distance) from the start point and a direction from the start point). As an example, assuming x and y coordinates with the start point as the origin, the scene vector at the point l can be represented by _{(l x} , _y).

On the other hand, the motion vector represents the control state of the traveling vehicle 90. The motion vector is acquired from, for example, a sensor value (for example, speed, acceleration, angular velocity, angular acceleration, etc.) output by the state detection sensor 93. As an example, the motion vector at a certain point l is represented by _{(v l} , a _{l) using the velocity v and the acceleration a at the point l.}

The neural network 22 divides the task into scenes based on the scene vector (l _x , _y ), and based on the motion vector (v _l , a _l ), the motion to be learned in each divided scene. Learn classification. As a result, the learning device 1 can learn the partial motion according to the scene by determining which scene it is currently in. As an example, the neural network 22 grasps the acceleration, deceleration, direction change, etc. of the motion of the vehicle 90 by paying attention to the change point of the motion vector in addition to the position represented by the scene vector, and is a series based on the change point. It is possible to classify the movements of the above into movements according to the scene. Further, for example, the neural network 22 can learn the classification of motions based on the similarity of motion vectors.

In the example of the course shown in FIG. 2, the task is divided into scenes corresponding to the five courses A to O. The partial operations classified into each scene are as follows, for example.
Scene A: First straight partial operation (for example, control of deceleration timing, running position, etc. when approaching the next first corner)
Scene a: Partial movement of the first corner (for example, control of steering wheel operation at a corner, acceleration timing when entering the second straight, etc.)
Scene c: Second straight partial operation (for example, control of deceleration timing, running position, etc. when approaching the next second corner)
Scene d: Partial movement of the second corner (for example, control of steering wheel operation at a corner, acceleration timing when entering the third straight, etc.)
Scene e: Partial movement of the third straight (for example, control of acceleration when entering the first straight)

It is preferable that the neural network 22 can rearrange the divided scenes according to the order of progress.

The motion classification result extraction unit 32 extracts the classification of the partial motion learned by the neural network 22 and stores it in the storage unit 40 (S203).

(2-3. Learning of allowable operation range)
FIG. 7 is a flowchart showing a detailed processing flow in learning (second learning) of the operation allowable range of S3 shown in FIG. First, the learning data input / output unit 21 refers to the storage unit 40, selects one of the partial operations classified in the processing of S2, and moves to the actuator 92 required for the selected partial operation. Extract the control amount of. The learning data input / output unit 21 executes control with the extracted control amount, and as a result, determines whether or not the allowable requirement is satisfied based on, for example, the output from the state detection sensor 93. At this time, for example, under the permissible requirement of not touching something else, it is determined whether or not the person has touched something or has stopped due to it. The learning data input / output unit 21 reads out information (determination result) as to whether or not the extracted control amount and the result learning operation conform to the permissible requirements as learning data, and gives it to the neural network 22 as learning data. The neural network 22 performs learning by deep learning based on the given learning data (S301). As a result of learning, the neural network 22 extracts the minimum value and the maximum value of the control amount of the actuator 92 that satisfies the permissible requirements. The learning result output unit 23 stores the extracted minimum value and maximum value in the storage unit 40 as an operation allowable range.

The learning device 1 performs the processing of S301 (and S302) for all the classified partial operations (S303). The learning result output unit 23 can also output the stored operation permissible range as a learning result of the operation permissible range. Thereby, for example, the learning result of the operation allowable range can be received from another learning device 1'having the same configuration, and the second learning can be omitted.

Although not essential, the learning device 1 can learn the movement allowable range for all the classified partial movements, and then perform learning again through a series of movements (S304). This makes it possible to perform faster lap control.

As described above, the learning device 1 according to the present embodiment learns the classified partial motions by first learning the control in which the learning motions fall within the permissible range of motions, thereby avoiding the controls in the subsequent learning. Becomes possible. As a result, more efficient learning can be performed.

(2-4. Optimization learning)
FIG. 8 is a flowchart showing a detailed processing flow in the optimization learning (third learning) of S4 shown in FIG. In the optimized learning, by optimizing the learning performed in the steps up to S3, the learning purpose given as the learning data at the start of the learning (in the present embodiment, "10 laps of the course within a predetermined time". To achieve the goal ”). In the optimization learning, the learning is performed within the range of the motions learned in S3. At this time, the learning data input / output unit 21 refers to the storage unit 40 and extracts the learning data (set by the operator) input in the initial stage of learning (S1 in FIG. 3). Further, the learning data input / output unit 21 further refers to the storage unit 40 to extract the state of the neural network 22 after learning the operation allowable range. The learning data input / output unit 21 sets these extracted data in the control unit 10.

The control unit 10 outputs a control amount for the actuator 92 based on the set data described above, and acquires sensor values of the control sensor 91 and the state detection sensor 93 for the control amount. The control unit 10 stores the given control amount and the sensor value output to the control amount in the storage unit 40. At this time, it is determined whether or not the control amount output to the actuator 92 is within the stored operation allowable range. If it is within the allowable operation range, the control amount is output, and the control amount outside the allowable operation range is not output.

The neural network 22 reads out the control amount and the sensor value stored in the control unit 10 in the above process, and performs learning by deep learning (S401). As a result, the neural network 22 learns the control motion that adapts to the learning requirement more efficiently from the start to the end of the motion (that is, from the start to the goal of the course) while learning the motion tolerance range. Can be done. The process of S401 is repeated until the entire learning is optimized (S402). The result of the optimization learning is extracted by the learning result output unit 23 and stored in the storage unit 40. As a result, in the optimized learning, learning can be performed within the operation allowable range.

As described above, according to the learning device 1 according to the present embodiment, the learning device 1 itself can classify the movements related to learning into partial movements and perform learning. As a result, individual optimization can be achieved for each classified operation, so learning can be performed more efficiently (that is, in a shorter period of time). Further, according to the learning device 1 according to the present embodiment, when learning the partial motion, the motion allowable range is first learned. As a result, the person can efficiently perform learning without setting detailed conditions for each operation in advance.

(Hardware configuration)
An example of the hardware configuration in the case where the learning device 1 described above is realized by the computer 800 will be described with reference to FIG. It should be noted that the configuration of each device can be realized by dividing it into a plurality of devices.

As shown in FIG. 9, the computer 800 includes a processor 801 and a memory 803, a storage device 805, an input interface unit (input I / F unit) 807, a data interface unit (data I / F unit) 809, and a communication interface unit (communication). The I / F section) 811 and the display device 813 are included.

The processor 801 controls various processes in the computer 800 by executing a program stored in the memory 803. For example, when the processor 801 executes the program stored in the memory 803, the control unit 10, the machine learning unit 20, the operation classification unit 30, and the like of the learning device 1 can be realized.

The memory 803 is, for example, a storage medium such as a RAM (Random Access Memory). The memory 803 temporarily stores the program code of the program executed by the processor 801 and the data required when the program is executed.

The storage device 805 is, for example, an auxiliary storage device such as a hard disk drive (HDD) or a solid state drive, or a non-volatile storage medium such as a flash memory. The storage device 805 stores an operating system and various programs for realizing each of the above configurations. Such programs and data are referred to by the processor 801 by being loaded into the memory 803 as needed. For example, the above-mentioned storage unit 40 is realized by the storage device 805.

The input I / F unit 807 is a device for receiving input from an administrator. Specific examples of the input I / F unit 807 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, and the like. The input I / F unit 807 may be connected to the computer 800 via an interface such as USB (Universal Serial Bus).

The data I / F unit 809 is a device for inputting data from the outside of the computer 800. Specific examples of the data I / F unit 809 include a drive device for reading data stored in various storage media. It is also conceivable that the data I / F unit 809 is provided outside the computer 800. In that case, the data I / F unit 809 is connected to the computer 800 via an interface such as USB.

The communication I / F unit 811 is a device for performing data communication via the Internet N by wire or wirelessly with an external device of the computer 800. It is also conceivable that the communication I / F unit 811 is provided outside the computer 800. In that case, the communication I / F unit 811 is connected to the computer 800 via an interface such as USB.

The display device 813 is a device for displaying various information. Specific examples of the display device 813 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display device 813 may be provided outside the computer 800. In that case, the display device 813 is connected to the computer 800 via, for example, a display cable or the like.

[Second Embodiment]
In the first embodiment, an example in which the learning device 1 is used for the automatic driving control vehicle 90 has been described. However, the device to which the learning device 1 is applied is not limited to the example shown in the first embodiment, and can be applied to various devices. In this embodiment, an example applied to the control of a robot, in which the task is to pack a paper cup, will be described. In the second embodiment, the differences from the first embodiment will be mainly described.

First, the difference between the system configuration according to the present embodiment and the first embodiment will be described with reference to FIG. The configuration of the learning device 1 is the same as that of the first embodiment. On the other hand, regarding the configuration outside the learning device 1, in the present embodiment, the control sensor 91'is composed of a group of sensors for performing a pick-and-place operation. Specifically, it is composed of a work detection sensor (image sensor), a robot gripping force sensor, and the like. Further, the control sensor 91'has an image recognition algorithm and can recognize the shape of the workpiece to be gripped. The other configurations outside the learning device 1 are the same as those in the first embodiment.

Next, the difference between the learning according to the present embodiment and the learning according to the first embodiment will be described.
The pick-and-place operation, which is a task according to the present embodiment, refers to an operation performed by the following procedure.
1) Grasp only one paper cup and place it in a predetermined position 2) Repeat the above work (grasping + mounting) a predetermined number of times to stack a predetermined number of paper cups 3) Stack a predetermined number of paper cups Put one of the bags in the bag 4) Twist the tip of the bag and fasten the mouth of the bag with a binding band 5) Pack the mouth of the bag in a cardboard box 6) Repeat the above steps 1) to 5) a predetermined number of times, and pack another one with the mouth of the bag fastened with a binding band into a cardboard box. 7) Repeat the above 6) a predetermined number of times. Close the lid while packed in a cardboard box

Further, in the learning of robot control according to the present embodiment, the learning objectives, learning requirements, and permissible requirements given are as follows.

(Learning purpose)
-Complete the above series of operations within the specified time.
(Learning requirements)
-Do not stick out of the bag or cardboard-At the initial stage level, "complete a series of tasks" (that is, "become able to control a series of operations related to a task")
(Allowable requirements)
・ Do not damage paper cups, bags, and cardboard boxes.

In the learning device 1 according to the present embodiment to which the above learning data is given, learning is performed with the minimum value and the maximum value of the force applied when gripping the corrugated cardboard or the paper cup as the operation allowable range as the operation allowable range.

The embodiment of the present invention has been described above. It should be noted that the present embodiment is for facilitating the understanding of the present invention, and is not for limiting and interpreting the present invention. Further, the present invention can be changed or improved without deviating from the gist thereof. For example, each step in the above-mentioned processing flow can omit a part of each step within a range that does not cause a contradiction in the processing contents, or can arbitrarily change the order of each processing step or execute them in parallel.

In the above-described embodiment, an example in which the system according to the present invention is used to manage the ability acquired by the machine by AI technology such as deep learning has been described, but the present invention is not limited to this and covers a wide range of fields. Can be applied. For example, it can be applied to distinguish between good and bad products, various industrial fields such as food, mechanical parts, chemical products and chemicals, fishery field, agriculture field, forestry field, service industry, medical and health field. Further, the present invention may be applied to a case where AI technology is applied to a product in the embedded field, a system utilizing IT technology such as a social system, big data analysis, classification processing in a wide range of control devices, and the like.

In addition, in this specification, a "part", a "means", and a "procedure" do not simply mean a physical configuration, but also include a case where the processing performed by the "part" or the like is realized by software. Further, even if the processing performed by one "part" or the device is executed by two or more physical configurations or devices, the processing performed by the two or more "parts" or the device is one physical means. And may be performed by the device.

In addition, some or all of the above embodiments may be described as in the following appendix, but are not limited to the following.
(Appendix 1)
With at least one hardware processor
The hardware processor
Accepts learning data including learning objectives and permissible requirements for learning movements performed when learning control,
Based on the learning data, the learning is executed and
The learning result by the neural network is output, and
Performing the learning
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed. Based on the result of the second learning, the third learning for achieving the learning purpose is executed within the control range.
(Appendix 2)
By at least one or more hardware processors
It ’s a learning step,
A step of accepting learning data including a learning purpose and a permissible requirement of a learning operation performed when learning the control, and a step of accepting the learning data.
Based on the learning data, the steps to execute the learning and
A step of outputting the learning result by the step of executing the learning and a step of outputting the learning result
And
The step of executing the learning is
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed. A learning method including a step of executing a third learning for achieving the learning purpose within the control range based on the result of the second learning.

1 Learning device 10 Control unit 20 Machine learning unit 21 Learning data input / output unit 22 Neural network 23 Learning result output unit 30 Motion classification unit 31 Control data extraction unit 32 Motion classification result extraction unit 40 Storage unit 90 Automatic driving control vehicle 91 For control Sensor 92 Actuator 93 State detection sensor

Claims (8)

A learning device that learns the control of movements related to a predetermined task.
A learning data reception unit that receives learning data including a learning purpose and a permissible requirement for a learning operation performed when learning the control, and a learning data reception unit.
A neural network that executes learning based on the training data,
An output unit that outputs the learning result of the neural network and
With
The neural network
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed. Based on the result of the second learning, the third learning for achieving the learning purpose is executed within the control range.
Learning device. The output unit
Output the result of the second learning,
The learning device according to claim 1. The learning device is
A learning device that learns the control of a series of movements related to a predetermined task.
The task is divided into a plurality of scenes, and in each of the divided scenes, a classification unit for specifying a partial operation performed in the scene among the series of operations is further provided.
The neural network executes the second learning and the third learning for each partial motion.
The learning device according to claim 1. It is an automatic driving control learning device that learns control about a series of movements related to automatic driving of a vehicle that goes around a predetermined course.
A learning data receiving unit that receives learning data including a learning purpose for the purpose of lapping the course a predetermined number of times within a predetermined time, and a permissible requirement for a learning operation performed when learning the control.
A neural network that executes learning based on the training data,
An output unit that outputs the learning result of the neural network and
With
The neural network
The first learning for achieving the ability to go around the course is executed, and based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed. Based on the result of the second learning, the third learning for achieving the learning purpose is executed within the control range.
Automatic driving control learning device. A robot control learning device that learns control of a series of operations related to a task of transporting a predetermined object to a predetermined mounting location, stacking a predetermined number of objects, and packing the object.
A learning data receiving unit that accepts learning data including a learning purpose for completing the task as soon as possible and a permissible requirement for a learning operation performed when learning the control.
A neural network that executes learning based on the training data,
An output unit that outputs the learning result of the neural network and
With
The neural network
The second learning that executes the first learning for achieving the controllability of a series of movements related to the task, and learns the control range in which the learning movement meets the permissible requirements based on the result of the first learning. Is executed, and based on the result of the second learning, the third learning for achieving the learning purpose is executed within the control range.
Robot control learning device. It is a learning method for learning the control of an operation related to a predetermined task executed by a computer provided with a control unit.
The control unit
A step of accepting learning data including a learning purpose and a permissible requirement of a learning operation performed when learning the control, and a step of accepting the learning data.
Based on the learning data, the steps to execute the learning and
A step of outputting the learning result by the step of executing the learning and a step of outputting the learning result
And
The step of executing the learning is
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed. A learning method including a step of executing a third learning for achieving the learning purpose within the control range based on the result of the second learning. To a computer that learns to control movements related to a given task
A procedure for accepting learning data including a learning purpose and a permissible requirement for a learning motion performed when learning the control.
A procedure for executing learning based on the learning data, and a procedure for outputting the learning result by the procedure for executing the learning.
To execute,
The procedure for performing the learning is
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control range in which the learning operation meets the permissible requirements is executed. Based on the result of the second learning, the procedure for executing the third learning for achieving the learning purpose within the control range is included.
program. A device that performs a given task
A first sensor that senses information necessary for the device to perform a task,
Actuator and
A second sensor that senses the state change of the device by the actuator, and
A control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor.
A storage unit that stores the learning results performed by the learning device according to any one of claims 1 to 3.
With
The control unit
Based on the learning result stored in the storage unit, the control amount according to the sensor values output from the first sensor and the second sensor is determined so as to be within the control range.
apparatus.

Images