JP6863081B2 - Learning device, learning control method, and its program - Google Patents

Description

The present invention relates to a learning device, a learning control method, and a program thereof.

Conventionally, research on artificial intelligence technology such as neural networks (hereinafter referred to as "AI technology") has been widely conducted (see, for example, Patent Document 1). In particular, with the rise of AI technology called deep learning, for example, the recognition rate of object recognition technology using images has improved rapidly in the last few years, and the level of image classification exceeds the human recognition rate. Is reaching. Deep learning technology is not limited to image recognition, but also voice recognition, personal authentication, behavior prediction, sentence summarization, automatic translation, monitoring, automatic driving, failure prediction, sensor data analysis, song genre judgment, content generation, It is expected to be applied to security systems and a wide range of other fields.

In machine learning such as deep learning, it is possible to have a machine perform learning to acquire a predetermined ability. At this time, the learning device that performs machine learning repeatedly executes the learning operation until a predetermined ability is acquired.

For example, Patent Document 1 discloses a learning control method for a robot. In the learning control method described in Patent Document 1, the driving unit of the robot is based on an error that occurs between a target trajectory that is a target of robot movement preset by a person and an actual trajectory when the robot actually moves. Correct the input value supplied to.

Japanese Unexamined Patent Publication No. 6-289918

In learning devices that control actuators based on a large amount of sensor information, such as automobile engine and running control, or chemical plants, control and sensor information output affect each other, so a control method is acquired. In order to do so, we need to do more complicated learning. Therefore, in a learning device that performs such complicated learning, it is not easy for a person to set a target value of a controlled amount in advance as in Patent Document 1. On the other hand, if the learning device is made to perform learning without setting a target value, it is necessary to repeat a large number of trial errors, which is inefficient.

Therefore, an object of the present invention is to provide a technique for shortening the time required for a learning device to achieve a learning purpose without human intervention.

The learning device according to one aspect of the present invention is a learning device that learns the control of movements related to a predetermined task, and learns based on a learning data receiving unit that receives learning data including a learning purpose and learning data. A neural network to be executed and an output unit for outputting the learning result by the neural network are provided, and the neural network executes the first learning for achieving the initial stage of the learning purpose and is based on the result of the first learning. Then, the second learning is executed to learn the control that leads to the state in which the operation related to learning cannot be continued, and based on the result of the second learning, the control that leads to the state in which the learning cannot be continued is excluded and the learning purpose is achieved. Perform the third learning to do.

According to the above configuration, before the third learning for achieving the learning purpose, the control learning to reach a state in which the operation related to the learning cannot be continued is performed. As a result, the learning purpose can be achieved in a shorter period of time because the learning can be performed excluding the control that leads to the inability to continue the device itself without giving the condition for restricting the control operation by a person. it can.

Further, the output unit may output the result of the second learning. According to this aspect, the learning result of the control leading to the non-continuation state can be utilized in other learning devices.

Further, the learning device is a learning device that learns the control of a series of movements related to a predetermined task. The task is divided into a plurality of scenes, and in each of the divided scenes, a line in the series of movements is performed. The neural network may execute the second learning and the third learning for each partial motion, further including a classification unit for specifying the partial motion.

According to this aspect, the learning device can classify the movements related to learning into partial movements, which are smaller units according to the scene, and can learn each of the classified partial movements. As a result, the learning purpose can be achieved in a shorter period of time.

The automatic running control learning device according to one aspect of the present invention is an automatic running control learning device that learns control about a series of operations related to automatic running of a vehicle orbiting a predetermined course, and determines the course within a predetermined time. A learning data receiving unit that receives learning data including a learning purpose for the purpose of orbiting the number of times, a neural network that executes learning based on the learning data, and an output unit that outputs learning results by the neural network. In preparation, the neural network executes the first learning to achieve the ability to go around the course, and based on the result of the first learning, learns the control to reach the state where the operation related to the learning cannot be continued. 2 Learning is executed, and based on the result of the second learning, the third learning for achieving the learning purpose is executed by excluding the control leading to the state of being unable to continue.

Further, the robot control learning device according to one aspect of the present invention is a robot control learning device that learns control about a series of operations related to a task of grasping a predetermined work and stacking it in a mounting place according to the shape of the work. A learning data reception unit that accepts learning data including a learning purpose for the purpose of stacking a predetermined number of works in a predetermined number of places within a predetermined time, and a neural network that executes learning based on the learning data. And an output unit that outputs the learning result by the neural network, the neural network executes the first learning to achieve stacking the work in the previously described place, and the result of the first learning. Based on the above, the second learning is executed to learn the control that leads to the state in which the operation related to learning cannot be continued, and based on the result of the second learning, the control that leads to the state in which the learning cannot be continued is excluded and learned. Perform the third learning to achieve the purpose.

Further, the learning method according to one aspect of the present invention is a learning method for learning the control of an operation related to a predetermined task executed by a computer provided with a control unit, and the control unit receives learning data including a learning purpose. The step of executing the learning, the step of executing the learning based on the learning data, the step of outputting the learning result by the step of executing the learning, and the step of executing the learning achieve the initial stage of the learning purpose. The first learning for learning is executed, and based on the result of the first learning, the second learning that learns the control to reach the state where the operation related to the learning cannot be continued is executed, and based on the result of the second learning. A learning method that includes a step of performing a third learning to achieve a learning objective, excluding control leading to a state of being unable to continue.

The program according to one aspect of the present invention includes a procedure for receiving learning data including a learning purpose, a procedure for executing learning based on the learning data, and a procedure for executing learning on a computer that learns the control of an operation related to a predetermined task. The procedure for outputting the learning result according to the procedure to be performed is to execute the first learning for achieving the initial stage of the learning purpose, and based on the result of the first learning, the learning is performed. The second learning is executed to learn the control leading to the state in which the operation cannot be continued, and based on the result of the second learning, the control leading to the state in which the operation cannot be continued is excluded to achieve the learning purpose. 3 A program that includes procedures to perform learning.

Further, the device according to one aspect of the present invention is a device that executes a predetermined task, and is a first sensor that senses information necessary for the device to perform an operation for executing the task, an actuator, and a device using the actuator. A second sensor that senses the state change of the above, a control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor, and a storage unit that stores the learning result performed by the above learning device. The control unit determines the control amount according to the sensor values output from the first sensor and the second sensor based on the learning result stored in the storage unit.

According to the present invention, it is possible to provide a technique for shortening the time required for a learning device to achieve a learning object without human intervention.

It is a block diagram which shows the schematic structure of the learning apparatus in 1st Embodiment. It is a schematic diagram which shows the course in which the vehicle controlled by the learning device in 1st Embodiment automatically travels. It is a flowchart which shows the outline of the process of the learning apparatus in 1st Embodiment. It is a block diagram which shows the detailed structure of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a flowchart which shows the details of the process of the learning apparatus in 1st Embodiment. It is a figure which shows an example of the hardware composition of the learning apparatus in 1st Embodiment. It is a block diagram which shows the schematic structure of the learning apparatus in 2nd Embodiment.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same elements are designated by the same reference numerals, and duplicate description will be omitted. Further, the following embodiments are examples for explaining the present invention, and the present invention is not intended to be limited only to the embodiments thereof. Furthermore, the present invention can be modified in various ways as long as it does not deviate from the gist thereof.

<1. System overview>
The outline of the system in this embodiment will be described with reference to FIGS. 1 to 3.
FIG. 1 is a block diagram showing a schematic configuration of the learning device 1 according to the present embodiment. The learning device 1 learns a predetermined task. The learning device 1 according to the present embodiment is mounted on an automatic driving control vehicle (hereinafter, also simply referred to as “vehicle”) 90 as an example, and controls the vehicle 90 for automatically traveling on a predetermined course (see FIG. 2). To learn. At this time, learning data is given to the learning device 1 by, for example, an operator or the like. The learning data is, for example, data including the following learning objectives and learning requirements.

(Learning purpose)
・ Complete 10 laps of the course within the specified time to reach the goal.
(Learning requirements)
・ Do not go out of the course ・ Clockwise in the lap direction ・ Goal ・ At the initial stage level, “Go around the course once”

The task is an operation related to learning (the "operation related to learning" in the present embodiment is various controls required for automatic traveling of the vehicle 90. It is considered that the operation is executed by the vehicle 90 by the various controls. It may be achieved.), And in this embodiment, it is to go around the course. In addition, the purpose of learning is the level at which the task should be achieved, and in the present embodiment, as described above, "to finish 10 laps of the course within a predetermined time". Then, in the present embodiment, it is considered that the learning requirement is that the task can be performed in the learning at the initial stage level.

Further, in the following description, the learning device 1 will be described as being composed of a computer such as a PC (Personal Computer) or a server device, but the present invention is not limited to this, and for example, any set having a processor, RAM, and ROM. It may be realized by a built-in device. Further, the configuration implemented in each device is not limited to the configuration realized by software. Any configuration included in each device may be a configuration realized by hardware. For example, the neural network 22 described later may be configured by an electronic circuit such as a custom LSI (Large-Scale Integration) or an FPGA (Field-Programmable Gate Array).

As shown in FIG. 1, the learning device 1 includes a control unit 10, a machine learning unit 20, an motion classification unit 30, and a storage unit 40.

The control unit 10 is connected to the control sensor 91, the actuator 92, and the state detection sensor 93 provided outside the learning device 1 in the vehicle 90. The control unit 10 controls the actuator 92 in response to the outputs from the control sensor 91 and the state detection sensor 93 to automatically drive the vehicle 90.

The control sensor 91 is a group of sensors for performing automatic driving control of the vehicle 90. For example, the control sensor 91 is composed of an in-vehicle camera, an external obstacle detection sensor such as a laser, a road surface condition detection sensor, and the like. On the other hand, the state detection sensor 93 is a group of sensors that detect the control state of the vehicle 90 that is automatically traveling. For example, the state detection sensor 93 is composed of a vibration sensor, a noise sensor, a fuel consumption detection sensor, a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and the like.

The actuator 92 is controlled by the control unit 10 to automatically drive the vehicle 90. The actuator 92 is composed of, for example, an accelerator actuator, a brake actuator, a steering actuator, and the like. The accelerator actuator controls the driving force of the vehicle by controlling the throttle opening degree according to the control signal from the control unit 10. The brake actuator controls the braking force on the wheels of the vehicle by controlling the operation amount of the brake pedal in response to the control signal from the control unit 10. The steering actuator controls the driving of the steering assist motor of the electric power steering system in response to the control signal from the control unit 10 to control the steering action of the vehicle.

Next, the procedure for the learning device 1 to perform learning will be roughly described with reference to FIG. The details of the processing of each step will be described later. FIG. 3 is a flowchart showing an outline of a processing flow when the learning device 1 performs learning. First, as the initial stage of learning (S1), learning is performed for the purpose of being able to perform a task (that is, being able to perform an operation that satisfies the learning requirements of the initial stage). In the learning device 1 in the present embodiment, as an initial stage, "a goal is to go around the course once" is given as a learning requirement.

After clearing the purpose of the initial stage level, the operation classification (S2) is performed next. At this stage, by analyzing the learning content performed in the initial learning stage of S1, the task is divided into a plurality of parts based on a predetermined parameter (hereinafter, the divided task is also referred to as a "scene") and divided. In each of the performed scenes, a process for specifying an operation (hereinafter, also referred to as "partial operation") performed in the scene among a series of operations related to the task is performed. Predetermined parameters for dividing a task are, for example, the displacement amount of the operation related to the learning of the task, the environment for executing the operation related to the learning of the task (elapsed time from the start time of the task, and the start location of the task). Position, etc.). In the present embodiment, the position from the start location of the task (environment for executing the operation related to the learning of the task) is used as a predetermined parameter. That is, in the present embodiment, the learning device 1 divides the task into scenes based on the position on the course, and a series of movements related to learning are performed based on the movements performed in course units corresponding to the divided scenes. It is classified into scenes. Learning can be made more efficient by learning in sub-motion units classified according to the situation. In the present embodiment, the efficiency of learning may mean, for example, shortening the time required from the start of learning to the achievement of the learning purpose.

When the movements are classified, as the next step, learning of control (S3) leading to a state in which learning cannot be continued is performed for each classified partial movement. Here, the learning continuation impossible state means a state in which the task cannot be continued. For example, when the learning in the learning device 1 is the control of a predetermined device, it means that the operation of the predetermined device to be controlled is stopped or the predetermined device fails and becomes inoperable. In the present embodiment, the state in which learning cannot be continued is, for example, a state in which the student goes off course, crashes into a wall or the like and cannot move, or breaks down. By learning the control leading to the learning continuation impossible state in advance, it is possible to avoid falling into the learning continuation impossible state in learning the optimum control in a later step. This makes it possible to perform learning more efficiently.

At the final stage of learning (S4), learning is optimized. At this stage, learning is performed in which the partial movements learned by classifying each scene are combined, and then the movements are optimally performed from the start to the end. In the present embodiment, as the final stage of learning, learning to complete 10 laps of the course within a predetermined time is performed.

<2. Detailed processing>
Next, the details of the processing of the learning device 1 in each step will be described with reference to FIGS. 4 to 8. FIG. 4 is a block diagram showing a detailed configuration of the learning device 1 according to the present embodiment. As shown in FIG. 4, the machine learning unit 20 includes a learning data input / output unit 21, a neural network 22, and a learning result output unit 23. Further, the motion classification unit 30 is composed of a control data extraction unit 31 and an motion classification result extraction unit 32.
In the following, the details of the processing of each part will be described for each step of FIG.

(2-1. Initial stage of learning)
FIG. 5 is a flowchart showing a detailed processing flow in the initial stage of learning of S1 shown in FIG. First, in the initial stage of learning (first learning), the learning data input / output unit 21 receives the learning data (S101). The learning data is, for example, data including the above-mentioned learning purpose and learning requirement.

In the next step (S102), machine learning is performed. In the present embodiment, since the conditions for limiting the individual control operations are not specified in advance, the learning device 1 itself learns the control operations. Specifically, the control unit 10 sets a random control amount with respect to the actuator 92 and operates the actuator 92. At this time, since the vehicle 90 cannot naturally travel along the course, the vehicle 90 will travel randomly while going off the course. The control unit 10 reads an output (hereinafter, also referred to as “sensor value”) from the control sensor 91 and the state detection sensor 93 for a randomly given control amount, and stores these data (control amount and sensor value). It is stored in the part 40. The neural network 22 refers to the storage unit 40, reads the stored control amount and the sensor value, and learns the control operation adapted to the learning requirement by deep learning (S102).

In the learning requirements, "to go around the course and reach the goal" is set as the purpose of the initial stage level. Therefore, in the learning device 1, it is determined that the machine learning has reached the initial stage level when it is determined that the goal has been reached by going around the course once based on the output from the control sensor 91, for example (S103: Y). , Finish the initial learning.

(2-2. Classification of movement)
FIG. 6 is a flowchart showing a detailed processing flow in the operation classification of S2 shown in FIG. First, in performing the motion classification process, the control data extraction unit 31 determines the sensor value of the control sensor 91 at the end of the initial learning stage, the control amount of the actuator 92, and the sensor value of the state detection sensor 93. Extract from the storage unit 40 (S201). The control data extraction unit 31 inputs each extracted value to the neural network 22 as learning data.

Next, the neural network 22 performs machine learning based on the learning data input by the control data extraction unit 31 (S202). At this time, the neural network 22 classifies the orbiting motion into a predetermined number of scenes.

The classification process of the neural network 22 into the scene of the orbiting motion will be described in more detail. The neural network 22 classifies the orbiting motion into scenes based on the scene vector and the motion vector. The scene vector represents a scene of a task performed by the vehicle 90. The scene vector is acquired from, for example, a sensor value output by the control sensor 91 (for example, a position (or distance) from the start point and a direction from the start point). As an example, assuming x and y coordinates with the start point as the origin, the scene vector at the point l can be represented by _{(l x} , _y).

On the other hand, the motion vector represents the control state of the traveling vehicle 90. The motion vector is acquired from, for example, a sensor value (for example, speed, acceleration, angular velocity, angular acceleration, etc.) output by the state detection sensor 93. As an example, the motion vector at a certain point l is represented by _{(v l} , a _{l) using the velocity v and the acceleration a at the point l.}

The neural network 22 divides the task into scenes based on the scene vector (l _x , _y ), and based on the motion vector (v _l , a _l ), the motion to be learned in each divided scene. Learn classification. As a result, the learning device 1 can learn the partial motion according to the scene by determining which scene it is currently in. As an example, the neural network 22 grasps the acceleration, deceleration, direction change, etc. of the motion of the vehicle 90 by paying attention to the change point of the motion vector in addition to the position represented by the scene vector, and is a series based on the change point. It is possible to classify the movements of the above into movements according to the scene. Further, for example, the neural network 22 can learn the classification of motions based on the similarity of motion vectors.

In the example of the course shown in FIG. 2, the task is divided into scenes corresponding to the five courses A to O. The partial operations classified into each scene are as follows, for example.
Scene A: First straight partial operation (for example, control of deceleration timing, running position, etc. when approaching the next first corner)
Scene a: Partial movement of the first corner (for example, control of steering wheel operation at a corner, acceleration timing when entering the second straight, etc.)
Scene c: Second straight partial operation (for example, control of deceleration timing, running position, etc. when approaching the next second corner)
Scene d: Partial movement of the second corner (for example, control of steering wheel operation at a corner, acceleration timing when entering the third straight, etc.)
Scene e: Partial movement of the third straight (for example, control of acceleration when entering the first straight)

It is preferable that the neural network 22 can rearrange the divided scenes according to the order of progress.

The motion classification result extraction unit 32 extracts the classification of the partial motion learned by the neural network 22 and stores it in the storage unit 40 (S203).

(2-3. Learning of control leading to a state in which learning cannot be continued)
FIG. 7 is a flowchart showing a detailed processing flow in the control learning (second learning) leading to the learning continuation impossible state of S3 shown in FIG. First, the learning data input / output unit 21 refers to the storage unit 40, selects one of the partial operations classified in the processing of S2, and controls the actuator 92 required for the partial operation. Is extracted. Further, the learning data input / output unit 21 executes control with the controlled amount extracted by referring to the storage unit 40, and as a result, whether or not the learning cannot be continued is determined based on, for example, the output from the state detection sensor 93. To judge. The learning data input / output unit 21 reads out the extracted control amount and information on whether or not the learning cannot be continued as a result, and gives it to the neural network 22 as learning data. The neural network 22 performs learning by deep learning based on the given learning data (S301).

At this time, the learning result output unit 23 can output the learning result of the control leading to the learning continuation impossible state. As a result, the neural network 22 can perform additional learning by accepting as learning data the control that has reached the state where learning cannot be continued from another learning device 1'having, for example, having the same configuration (S302). This makes it possible to perform more efficient learning. Efficient learning means, for example, learning in which the time required from the start of learning to the achievement of the learning purpose is short. The process of S302 is not an essential process.

The learning device 1 performs the processing of S301 (and S302) for all the classified partial operations (S303).

Although not essential, the learning device 1 can learn the control leading to the inability to continue learning for all the classified partial movements, and then perform learning again through a series of movements (S304). This makes it possible to perform faster lap control.

As described above, the learning device 1 according to the present embodiment can learn the classified partial motions by first learning the control leading to the learning continuation impossible state, thereby avoiding the control in the subsequent learning. become. As a result, more efficient learning can be performed.

(2-4. Optimization learning)
FIG. 8 is a flowchart showing a detailed processing flow in the optimization learning (third learning) of S4 shown in FIG. In the optimized learning, by optimizing the learning performed in the steps up to S3, the learning purpose given as the learning data at the start of the learning (in the present embodiment, "10 laps of the course within a predetermined time". To achieve the goal ”). In the optimized learning, the learning is performed excluding the control that leads to the learning continuation impossible state learned in S3. At this time, the learning data input / output unit 21 refers to the storage unit 40 and extracts the learning data (set by the operator) input in the initial stage of learning (S1 in FIG. 3). Further, the learning data input / output unit 21 further refers to the storage unit 40 and extracts the state of the neural network 22 after learning the control leading to the learning continuation impossible state. The learning data input / output unit 21 sets these extracted data in the control unit 10.

The control unit 10 outputs a control amount for the actuator 92 based on the set data described above, and acquires sensor values of the control sensor 91 and the state detection sensor 93 for the control amount. The control unit 10 stores the given control amount and the sensor value output to the control amount in the storage unit 40.

The neural network 22 reads out the control amount and the sensor value stored in the control unit 10 in the above process, and performs learning by deep learning (S401). As a result, the neural network 22 makes the control operation adapting to the learning requirement more efficient from the start to the end of the operation (that is, from the start to the goal of the course) in the state where the control leading to the learning continuation impossible state is learned. You can learn well. The process of S401 is repeated until the entire learning is optimized (S402). The result of the optimization learning is extracted by the learning result output unit 23 and stored in the storage unit 40. As a result, in the optimized learning, the learning can be performed by excluding the control leading to the state in which the learning cannot be continued.

As described above, according to the learning device 1 according to the present embodiment, the learning device 1 itself can classify the movements related to learning into partial movements and perform learning. As a result, individual optimization can be performed for each classified operation, so that learning can be performed more efficiently (that is, in a shorter period of time). Further, according to the learning device 1 according to the present embodiment, when learning the partial motion, first, the control leading to the learning continuation impossible state is learned. As a result, the person can efficiently perform learning without setting detailed conditions for each operation in advance.

(Hardware configuration)
An example of the hardware configuration in the case where the learning device 1 described above is realized by the computer 800 will be described with reference to FIG. It should be noted that the configuration of each device can be realized by dividing it into a plurality of devices.

As shown in FIG. 9, the computer 800 includes a processor 801 and a memory 803, a storage device 805, an input interface unit (input I / F unit) 807, a data interface unit (data I / F unit) 809, and a communication interface unit (communication). The I / F section) 811 and the display device 813 are included.

The processor 801 controls various processes in the computer 800 by executing a program stored in the memory 803. For example, when the processor 801 executes the program stored in the memory 803, the control unit 10, the machine learning unit 20, the operation classification unit 30, and the like of the learning device 1 can be realized.

The memory 803 is, for example, a storage medium such as a RAM (Random Access Memory). The memory 803 temporarily stores the program code of the program executed by the processor 801 and the data required when the program is executed.

The storage device 805 is, for example, an auxiliary storage device such as a hard disk drive (HDD) or a solid state drive, or a non-volatile storage medium such as a flash memory. The storage device 805 stores an operating system and various programs for realizing each of the above configurations. Such programs and data are referred to by the processor 801 by being loaded into the memory 803 as needed. For example, the above-mentioned storage unit 40 is realized by the storage device 805.

The input I / F unit 807 is a device for receiving input from an administrator. Specific examples of the input I / F unit 807 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, and the like. The input I / F unit 807 may be connected to the computer 800 via an interface such as USB (Universal Serial Bus).

The data I / F unit 809 is a device for inputting data from the outside of the computer 800. Specific examples of the data I / F unit 809 include a drive device for reading data stored in various storage media. It is also conceivable that the data I / F unit 809 is provided outside the computer 800. In that case, the data I / F unit 809 is connected to the computer 800 via an interface such as USB.

The communication I / F unit 811 is a device for performing data communication via the Internet N by wire or wirelessly with an external device of the computer 800. It is also conceivable that the communication I / F unit 811 is provided outside the computer 800. In that case, the communication I / F unit 811 is connected to the computer 800 via an interface such as USB.

The display device 813 is a device for displaying various information. Specific examples of the display device 813 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display device 813 may be provided outside the computer 800. In that case, the display device 813 is connected to the computer 800 via, for example, a display cable or the like.

[Second Embodiment]
In the first embodiment, an example in which the learning device 1 is used for the automatic driving control vehicle 90 has been described. However, the device to which the learning device 1 is applied is not limited to the example shown in the first embodiment, and can be applied to various devices. In this embodiment, an example applied to the control of a robot whose task is to perform a pick-and-place operation will be described. In the second embodiment, the differences from the first embodiment will be mainly described.

First, the difference between the system configuration according to the present embodiment and the first embodiment will be described with reference to FIG. The configuration of the learning device 1 is the same as that of the first embodiment. On the other hand, regarding the configuration outside the learning device 1, in the present embodiment, the control sensor 91'is composed of a group of sensors for performing a pick-and-place operation. Specifically, it is composed of a work detection sensor (image sensor), a robot gripping force sensor, and the like. Further, the control sensor 91'has an image recognition algorithm and can recognize the shape of the workpiece to be gripped. The other configurations outside the learning device 1 are the same as those in the first embodiment.

Next, the difference between the learning according to the present embodiment and the learning according to the first embodiment will be described.
The pick-and-place operation, which is a task according to the present embodiment, refers to an operation performed by the following procedure.
1. 1. Recognize and grip the work shape.
2. Lift the gripped work.
3. 3. The lifted work is moved to a predetermined position according to the shape of the work.
4. Stack each work shape in the cylinder.

Further, in the learning of robot control according to the present embodiment, the learning objectives and learning requirements given are as follows.

(Learning purpose)
-From a container in which workpieces having three different shapes (for example, three types of cylindrical workpiece, square prism workpiece, and triangular prism workpiece) are stacked separately, the work shape can be obtained within a predetermined time by a pick-and-place operation. Ten workpieces are stacked in a cylinder (circular, quadrangular, triangular) having an entrance according to the above.
(Learning requirements)
・ Do not place workpieces in any position other than the specified position. ・ Stack 10 workpieces in a cylinder for each workpiece shape. ・ At the initial level, "stack one workpiece in a cylinder with an appropriate workpiece shape."

In the present embodiment, the task is to stack the workpieces in a cylinder according to the shape. Further, in the present embodiment, in the first embodiment, the pick-and-place motion to be learned divides the task into scenes based on the course on which the vehicle 90 travels, and classifies the partial motions based on the scene. In the same procedure, the pick-and-place operation may be classified into partial operations according to the situation. For example, in the present embodiment, the task corresponds to the scene corresponding to the movement of gripping the work, the scene corresponding to the movement of carrying the work, and the movement of stacking the work based on the displacement amount of the movement related to the learning of the task. The scene is divided into. Pick-and-place movements are classified into partial movements according to the divided scenes.

Further, in the present embodiment, the state in which learning cannot be continued means, for example, a state in which the work cannot enter the cylinder. Therefore, in the learning stage of the control leading to the state in which learning cannot be continued, the control to be learned is, for example, as follows.
・ Wrong place of placement (the shape of the work and the shape of the entrance of the cylinder are different)
・ The direction in which the workpieces are stacked is incorrect (the orientation of the workpiece and the orientation of the cylinder are different).

In the learning device 1 according to the present embodiment, by learning in advance the control leading to the above-mentioned learning continuation impossible state, the shape of the work and the shape of the cylinder are appropriately recognized, and the orientation when gripping the work is learned in advance. can do. As a result, in the final stage of learning, it is possible to avoid a state in which learning cannot be continued, so that learning efficiency can be further improved. That is, the time required to achieve the learning purpose can be further shortened.
Other configurations are the same as those in the first embodiment.

The embodiment of the present invention has been described above. It should be noted that the present embodiment is for facilitating the understanding of the present invention, and is not for limiting and interpreting the present invention. Further, the present invention can be changed or improved without deviating from the gist thereof. For example, each step in the above-mentioned processing flow can omit a part of each step within a range that does not cause a contradiction in the processing contents, or can arbitrarily change the order of each processing step or execute them in parallel.

In the above-described embodiment, an example in which the system according to the present invention is used to manage the ability acquired by the machine by AI technology such as deep learning has been described, but the present invention is not limited to this and covers a wide range of fields. Can be applied. For example, it can be applied to distinguish between good and bad products, various industrial fields such as food, mechanical parts, chemical products and chemicals, fishery field, agriculture field, forestry field, service industry, medical and health field. Further, the present invention may be applied to a case where AI technology is applied to a product in the embedded field, a system utilizing IT technology such as a social system, big data analysis, classification processing in a wide range of control devices, and the like.

In addition, in this specification, a "part", a "means", and a "procedure" do not simply mean a physical configuration, but also include a case where the processing performed by the "part" is realized by software. Further, even if one "part", "means", "procedure" or process performed by the device is executed by two or more physical configurations or devices, two or more "parts", "procedures" or devices The processing to be performed may be performed by one physical means or device.

In addition, some or all of the above embodiments may be described as in the following appendix, but are not limited to the following.
(Appendix 1)
With at least one hardware processor
The hardware processor
Accepts learning data including learning purposes,
Based on the learning data, the learning is executed and
The learning result by the neural network is output, and
Performing the learning
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control to reach the state where the operation related to the learning cannot be continued is executed. Then, based on the result of the second learning, the third learning for achieving the learning purpose is included by excluding the control leading to the state in which the learning cannot be continued.
(Appendix 2)
By at least one or more hardware processors
It ’s a learning step,
Steps to accept learning data including learning purpose,
Based on the learning data, the steps to execute the learning and
The step of outputting the learning result by the neural network and
And
The step of executing the learning is
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control to reach the state where the operation related to the learning cannot be continued is executed. Then, based on the result of the second learning, a learning method including a step of executing the third learning for achieving the learning purpose by excluding the control leading to the inability to continue.

1 Learning device 10 Control unit 20 Machine learning unit 21 Learning data input / output unit 22 Neural network 23 Learning result output unit 30 Motion classification unit 31 Control data extraction unit 32 Motion classification result extraction unit 40 Storage unit 90 Automatic driving control vehicle 91 For control Sensor 92 Actuator 93 State detection sensor

Claims (8)

A learning device that learns the control of movements related to a predetermined task.
A learning data reception unit that accepts learning data including learning purposes,
A neural network that executes learning based on the training data,
An output unit that outputs the learning result of the neural network and
With
The neural network
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control to reach the state where the operation related to the learning cannot be continued is executed. Then, based on the result of the second learning, the third learning for achieving the learning purpose is executed by excluding the control leading to the inability to continue.
Learning device. The output unit
Output the result of the second learning,
The learning device according to claim 1. The learning device is
A learning device that learns the control of a series of movements related to a predetermined task.
The task is divided into a plurality of scenes, and in each of the divided scenes, a classification unit for specifying a partial operation performed in the scene among the series of operations is further provided.
The neural network executes the second learning and the third learning for each partial operation.
The learning device according to claim 1. It is an automatic driving control learning device that learns control about a series of movements related to automatic driving of a vehicle that goes around a predetermined course.
A learning data reception unit that receives learning data including a learning purpose for the purpose of going around the course a predetermined number of times within a predetermined time.
A neural network that executes learning based on the training data,
An output unit that outputs the learning result of the neural network and
With
The neural network
The first learning for achieving one lap of the course is executed, and based on the result of the first learning, the second learning for learning the control leading to the state in which the operation related to the learning cannot be continued is executed. Then, based on the result of the second learning, the third learning for achieving the learning purpose is executed by excluding the control leading to the state of being unable to continue.
Automatic driving control learning device. A robot control learning device that learns control of a series of operations related to a task of grasping a predetermined work and stacking it in a placement place according to the shape of the work.
A learning data reception unit that receives learning data including a learning purpose for the purpose of stacking a predetermined number of the works in a predetermined place within a predetermined time.
A neural network that executes learning based on the training data,
An output unit that outputs the learning result of the neural network and
With
The neural network
The first learning for achieving the stacking of the work in the previously described place is executed, and based on the result of the first learning, the control leading to the state in which the operation related to the learning cannot be continued is learned. The second learning is executed, and based on the result of the second learning, the third learning for achieving the learning purpose is executed by excluding the control leading to the inability to continue.
Robot control learning device. It is a learning method for learning the control of an operation related to a predetermined task executed by a computer provided with a control unit.
The control unit
Steps to accept learning data including learning purpose,
Based on the learning data, the steps to execute the learning and
A step of outputting the learning result by the step of executing the learning and a step of outputting the learning result
And
The step of executing the learning is
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control to reach the state where the operation related to the learning cannot be continued is executed. Then, based on the result of the second learning, the third learning for achieving the learning purpose is executed by excluding the control leading to the non-continuable state, including the step.
Learning method. To a computer that learns to control movements related to a given task
Procedure for accepting learning data including learning purpose,
A procedure for executing learning based on the learning data, and a procedure for outputting the learning result by the means for executing the learning.
To execute,
The procedure for performing the learning is
The first learning for achieving the initial stage of the learning purpose is executed, and based on the result of the first learning, the second learning for learning the control to reach the state where the operation related to the learning cannot be continued is executed. Then, based on the result of the second learning, the third learning for achieving the learning purpose is executed by excluding the control leading to the non-continuable state, including the procedure.
program. A device that performs a given task
A first sensor that senses information necessary for the device to perform a task,
Actuator and
A second sensor that senses the state change of the device by the actuator, and
A control unit that controls the actuator based on the sensor values output from the first sensor and the second sensor.
A storage unit that stores the learning results performed by the learning device according to any one of claims 1 to 3.
With
The control unit
Based on the learning result stored in the storage unit, the control amount according to the sensor values output from the first sensor and the second sensor is determined.
apparatus.

Images