JP2020179438A - Computing system and machine learning method - Google Patents

Abstract

To generate a model for holding an object of various shapes.SOLUTION: A computing system learns a control algorism by which a holding device holds an object, and executes first learning processing which generates a first model for outputting an object hold success probability, and second learning processing which generates a second model for outputting object data of the object which is deformed from a shape of a reference object. In the second learning processing, object data of the first object, which is deformed from the shape of the reference object, is generated on the basis of the second model, a holding point of the first object is determined, a plurality of sets of the object data of the first object and the holding point of the first object are input in the first model, and the second model is updated on the basis of a comparison result of a maximum value of the holding success probability of the first object output from the first model, and a target probability.SELECTED DRAWING: Figure 1

Description

The present invention relates to a machine learning technique for generating a control model of a device that grips an object.

With the sophistication of machine learning technology, the development of algorithms (models) for controlling automobiles, robots, etc. is progressing. For example, the technique described in Patent Document 1 is known as a technique for developing an algorithm for controlling a device for gripping an object (grasping device).

In Patent Document 1, "The measurement feature identification unit 52 estimates the ease of grasping a part of the work measured by the sensor 2 based on the past extraction success / failure information of the DB 1, and the gripping operation calculation unit 53, The opening / closing operation calculation unit 54 preferentially does not grasp an object that is difficult to grasp, but when there is no other object that is likely to be grasped, adjust at least one of the opening / closing amount, the operating speed, and the gripping force of the hand. , Perform a more polite take-out operation ”is described. Further, Patent Document 1 describes that the ease of grasping a work is estimated by using a classifier such as a neural network.

In the prior art, a classifier is generated using learning data about an object of a known shape. Therefore, it is possible to predict whether or not an object having a known shape can be gripped, but it is not possible to predict whether or not an object having an unknown shape can be gripped.

Japanese Unexamined Patent Publication No. 2013-52490

Hiroharu Kato, Yoshitaka Ushiku, Tatsuya Harada, "Neural 3D Mesh Renderer", [online], November 20, 2017, [Search March 25, 2019], Internet <https://arxiv.org/abs/1711.07566 ＞

In order to improve the accuracy of the classifier, it is necessary to learn using learning data on objects of various shapes.

The present invention provides a technique for automatically generating learning data for objects having various shapes and efficiently improving the accuracy of the classifier using the learning data.

A typical example of the invention disclosed in the present application is as follows. That is, it is a computer system in which a gripping device learns a control algorithm for gripping an object, and the computer system includes at least one computer having a calculation device and a storage device connected to the calculation device, and at least the above. One computer uses object data including information on the shape of the object, a gripping point as a reference of a position for gripping the object, and learning data including information indicating the success or failure of gripping the object. The first learning process for generating the first model that outputs the gripping success probability, the second learning process for generating the second model that outputs the object data of the object whose shape of the reference object is deformed, and the second learning process. Is executed, and in the second learning process, the at least one computer generates the object data of the first object in which the shape of the reference object is deformed based on the second model, and the first object of the first object. The gripping point is determined, the object data of the first object and a plurality of sets of gripping points of the first object are input to the first model, and the gripping success probability of the first object output from the first model. The second model is updated based on the difference between the maximum value and the target probability of.

According to the present invention, learning data related to objects having various shapes is automatically generated according to the learning state of the classifier (first model), and the learning process using the learning data is executed to obtain the classifier. The accuracy can be improved efficiently. Issues, configurations and effects other than those mentioned above will be clarified by the description of the following examples.

It is a figure which shows the configuration example of the computer system of Example 1. FIG. It is a figure which shows an example of the model of the learning target of Example 1. FIG. It is a figure which shows the hardware configuration of the computer of Example 1. FIG. It is a figure which shows an example of the setting screen presented by the computer system of Example 1. FIG. It is a flowchart explaining an example of the process executed by the 1st learning part of Example 1. FIG. It is a flowchart explaining an example of the 1st learning process executed by the 1st learning part of Example 1. FIG. It is a flowchart explaining an example of the process executed by the 2nd learning part of Example 1. FIG. It is a flowchart explaining an example of the 2nd learning process executed by the 2nd learning part of Example 1. FIG. It is a flowchart explaining an example of control of the gripping apparatus of Example 1. FIG. It is a flowchart explaining an example of the process executed by the 2nd learning part of Example 2. It is a flowchart explaining an example of the 2nd learning process executed by the 2nd learning part of Example 2. FIG. It is a figure which shows the configuration example of the computer system of Example 3. It is a flowchart explaining an example of the process executed by the 1st learning part of Example 3. FIG. It is a flowchart explaining an example of the process executed by the 2nd learning part of Example 3. FIG.

First, the outline of the present invention will be described. In the present invention, a control algorithm (model) of a gripping device is learned by applying GAN (Generative Adversarial Networks). In GAN, a Generator that generates an image and a Discriminator that identifies whether or not the image is generated by the Generator are learned.

The computer system of the present invention is an object that generates object data including information on the shape of an object of an arbitrary shape and a probability model that calculates the probability of gripping an object of an arbitrary shape at an arbitrary gripping point (grasping success probability). Learn each with a data generation model.

In the learning of the probability model, the probability model is trained so that the maximum value of the gripping success probability of an object of an arbitrary shape becomes larger than the threshold value (target probability) by using the object data generated by the object data generation model. This makes it possible to efficiently learn a probabilistic model for grasping objects having various shapes.

In the learning of the object data generation model, the object data generation model is learned so that the difference between the maximum values of the target probability and the gripping success probability becomes small. That is, the object data generation model is updated so as to match the progress of learning of the probability model. As a result, it is possible to efficiently generate object data such that the gripping success probability is close to the target probability. Therefore, more efficient learning of the probabilistic model can be realized.

Hereinafter, examples of the present invention will be described with reference to the drawings. However, the present invention is not construed as being limited to the contents of the examples shown below. It is easily understood by those skilled in the art that a specific configuration thereof can be changed without departing from the idea or purpose of the present invention.

In the configurations of the invention described below, the same or similar configurations or functions are designated by the same reference numerals, and duplicate description will be omitted.

The notations such as "first", "second", and "third" in the present specification and the like are attached to identify the components, and do not necessarily limit the number or order.

The position, size, shape, range, etc. of each configuration shown in the drawings and the like may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not limited to the position, size, shape, range, etc. disclosed in the drawings and the like.

FIG. 1 is a diagram showing a configuration example of the computer system of the first embodiment. FIG. 2 is a diagram showing an example of a model to be learned according to the first embodiment. FIG. 3 is a diagram showing a hardware configuration of the computer 100 of the first embodiment.

First, the model learned by the computer system of the first embodiment will be described with reference to FIG.

The gripping device 200 grips the object 210 in a work space such as a warehouse or a factory. The gripping device 200 includes a control device 201 and an arm 202. The gripping device 200 may include a sensor such as a camera and a moving device such as a tire and a motor.

The control device 201 controls the entire gripping device 200. The control device 201 includes a controller, a drive device, and the like (not shown). The arm 202 grips the object according to the control by the control device 201. The arm 202 may have a force sensor, a tactile sensor, and the like. The present invention is not limited to the form of the arm 202. Any form may be used as long as the object 210 can be gripped.

Here, the outline of the processing in the gripping work of the gripping device 200 will be described.

The control device 201 acquires an image of the space in which the object 210 exists from a sensor or the like. The control device 201 selects the gripping point 220 of the object 210 based on the stochastic model. Here, the gripping point 220 is, for example, the coordinates that serve as a reference for the position where the arm 202 grips the object 210.

The control device 201 generates a trajectory plan to the grip point 220. Here, the trajectory plan is information on the moving trajectory of the arm 202 and the gripping device 200 for moving the arm 202 to the gripping point 220.

The control device 201 generates and outputs control information for controlling the operation of the gripping device 200 based on the trajectory plan. The gripping device 200 moves so that the arm 202 grips the object 210 at the gripping point 220 based on the control information.

The computer system of the first embodiment executes a learning process for generating a probabilistic model, which is a control algorithm for the gripping device 200 to grip an object. Returning to the description of FIG.

The computer system is composed of a plurality of computers 100-1 and 100-2. The computers 100-1 and 100-2 are connected to each other either directly or via a network.

The computer 100-1 executes a learning process for generating a probability model for calculating the probability that an object 210 having an arbitrary shape can be gripped at an arbitrary gripping point 220 (grasping success probability).

The probability model is a model for calculating the gripping success probability from the object data including the information about the shape of the object 210 and the input data including the gripping point 220. The computer 100-1 may input the object data itself into the probability model, or may input the feature amount of the object data extracted by using VAE (Variational AutoEncoder) or the like into the probability model.

The computer 100-2 executes a learning process for generating an object data generation model that generates the object data 170 for learning. Reference object data 101 used for generating learning object data 170 is input to the computer 100-2. The reference object data 101 is data relating to a reference object. The reference object data 101 and the learning object data 170 are, for example, image data generated by rendering a polygon mesh.

The object data generation model is a model that accepts the reference object data 101 as input data and generates the learning object data 170.

Here, the hardware configurations of the computers 100-1 and 100-2 will be described with reference to FIG. FIG. 3 describes the hardware configuration of the computer 100-1. The hardware configuration of computer 100-2 is the same as that of computer 100-1.

The computer 100-1 has a processor 300, a main storage device 301, a sub storage device 302, a network interface 303, and an IO interface 304. Each piece of hardware is connected to each other via an internal bus.

The hardware configuration of the computer 100-1 shown in FIG. 2 is an example and is not limited to this. For example, the computer 100-1 may have hardware (not shown), or may not have some hardware such as the sub storage device 302 and the IO interface 304.

The processor 300 executes a program stored in the main storage device 301. When the processor 300 executes processing according to a program, it operates as a functional unit (module) that realizes a specific function. In the following description, when the process is described with the functional unit as the subject, it is shown that the processor 300 is executing the program that realizes the module.

The main storage device 301 is a storage device such as a DRAM (Dynamic Random Access Memory), and stores a program executed by the processor 300 and information used by the program. The main storage device 301 is also used as a work area used by the program. The programs and information stored in the main storage devices 301 of the computers 100-1 and 100-2 will be described later.

The program and information may be stored in a device or device other than the main storage device 301, such as the sub storage device 302. In this case, the processor 300 reads the program and information from the device or device and loads it into the main storage device 301.

The sub-storage device 302 is a storage device such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), and permanently stores data.

The network interface 303 is an interface for communicating with an external device such as a terminal 320 via the network 310.

The IO interface 304 is an interface for connecting to the input device 330, the output device 331, and the like. The input device 330 is a keyboard, a mouse, a touch panel, and the like, and the output device 331 is a display, a printer, and the like.

Returning to the description of FIG. Next, the programs and information stored in the main storage devices 301 of the computers 100-1 and 100-2 will be described.

The main storage device 301 of the computer 100-1 stores the program that realizes the first learning unit 110 and the probability model management information 130.

The probability model management information 130 is information for managing the probability model. The probabilistic model management information 130 includes at least parameters that define the probabilistic model. The probability model management information 130 may include information regarding the structure of the probability model and the like. For example, when the probability model is a neural network, the probability model management information 130 may include information regarding the structure of the hierarchy and the like.

The first learning unit 110 executes a learning process (first learning process) for generating a probability model. The first learning unit 110 includes a gripping probability calculation unit 111, a parameter update unit 112, and a simulation unit 113.

The gripping probability calculation unit 111 calculates the gripping success probability of the object at the gripping point based on the set of the gripping point and the object data and the probability model defined by the probability model management information 130. The parameter update unit 112 updates the parameters that define the probability model. That is, the parameter update unit 112 updates the probability model.

The simulation unit 113 generates the learning data 150 and the probability data 160 by executing the simulation using the learning object data 170. As the simulation, for example, a physical simulation can be considered. Here, the learning data 150 includes the learning object data 170 and the gripping points (coordinates). The probability data 160 includes a gripping point and a gripping success probability.

The main storage device 301 of the computer 100-2 stores the program for realizing the second learning unit 120 and the object data generation model management information 140.

The object data generation model management information 140 is information for managing the object data generation model. The object data generation model management information 140 includes at least parameters that define the object data generation model. The object data generation model management information 140 may include information regarding the structure of the object data generation model and the like. For example, when the object data generation model is a neural network, the object data generation model management information 140 may include information regarding the structure of the hierarchy and the like.

The second learning unit 120 executes a learning process (second learning process) for generating an object data generation model. The second learning unit 120 includes an object data generation unit 121, a parameter update unit 122, and a grip point data generation unit 123.

The object data generation unit 121 generates learning object data 170 based on the reference object data 101 and the object data generation model defined by the object data generation model management information 140. The object data generation unit 121 may internally generate the learning object data 170 by iterative processing including the data generated by itself. The parameter update unit 122 updates the parameters that define the object data generation model. That is, the parameter update unit 122 updates the object data generation model.

The gripping point data generation unit 123 determines the gripping point of the object corresponding to the learning object data 170, and generates the learning object data 170 and the evaluation input data 180 including the gripping point.

Regarding each of the functional units of the computers 100-1 and 100-2, a plurality of functional units may be combined into one functional unit, or one functional unit may be divided into a plurality of functional units for each function. Good. Further, each functional unit may be realized by using a virtual computer running on one computer.

A part of the functional units of the first learning unit 110 may be included in the second learning unit 120, or a part of the functional units of the second learning unit 120 may be included in the first learning unit 110. .. For example, the second learning unit 120 may include the simulation unit 113.

The solid line in FIG. 1 indicates the input / output of data in the first learning process, and the dotted line in FIG. 1 indicates the input / output of data in the second learning process. In this specification, it is described as hostile learning that the first learning process and the second learning process are executed in a circular manner.

FIG. 4 is a diagram showing an example of the setting screen 400 presented by the computer system of the first embodiment.

The setting screen 400 is a screen for making settings necessary for the learning process executed by the first learning unit 110 and the second learning unit 120, and is displayed on the terminal 320 or the output device 331. The setting screen 400 may be presented by either the first learning unit 110 or the second learning unit 120. Further, a functional unit different from the first learning unit 110 and the second learning unit 120 may present the setting screen 400.

The setting screen 400 includes a reference object data column 401, a probability model column 402, an object data generation model column 403, a learning count column 404, a learning count column 405, a target probability column 406, a setting button 410, and a start button 411.

The reference object data field 401 is a field for setting the reference object data 101 to be input to the second learning unit 120. The probability model column 402 is a column for setting the probability model management information 130 for storing the probability model. The object data generation model column 403 is a column for setting the object data generation model management information 140 for storing the object data generation model.

For example, a file path or the like is set in the reference object data column 401, the probability model column 402, and the object data generation model column 403.

The learning number column 404 is a column for setting an upper limit value (upper limit value of the learning number) of the update number of the probability model. The learning number column 405 is a column for setting an upper limit value (upper limit value of the learning number) of the update number of the object data generation model. The target probability column 406 is a column for setting the target probability.

The setting button 410 is an operation button for setting the values set in each column on the computers 100-1 and 100-2. When the user operates the setting button 410, a setting instruction including the values set in each field is transmitted to the computer system.

At this time, the first learning unit 110 registers the initial probability model in the probability model management information 130, and stores the values set in the learning number of times column 404 and the target probability column 406 in the work area. Further, the second learning unit 120 registers the initial object data generation model in the object data generation model management information 140, and stores the values set in the reference object data 101 and the target probability column 406 in the work area.

The start button 411 is an operation button for inputting a learning start instruction to the computer system. The learning start instruction is transmitted to either computer 100-1 or computer 100-2.

FIG. 5 is a flowchart illustrating an example of processing executed by the first learning unit 110 of the first embodiment.

When the first learning unit 110 receives the learning start instruction, the first learning unit 110 starts the process described below. The learning start instruction is transmitted from the input device 330, the terminal 320, or the second learning unit 120. The input device 330 and the terminal 320 transmit a learning start instruction via the setting screen 400.

First, the first learning unit 110 acquires the learning object data 170 (step S101).

Specifically, the first learning unit 110 transmits an instruction to generate the learning object data 170 to the second learning unit 120. The object data generation unit 121 of the second learning unit 120 generates the learning object data 170 based on the reference object data 101 and the object data generation model management information 140, and transmits the learning object data 170 to the first learning unit 110. To do.

Next, the first learning unit 110 generates learning data 150 and probability data 160 using the acquired learning object data 170 (step S102). Specifically, the following processing is executed.

The simulation unit 113 arranges an object corresponding to the learning object data 170 in the virtual space. The position and orientation of the object in the virtual space can be set arbitrarily. The simulation unit 113 determines a gripping point of the object, and simulates whether or not the object can be gripped based on the gripping point. The simulation unit 113 may execute the simulation a plurality of times for the same gripping point.

The simulation unit 113 generates learning data 150 including the learning object data 170 and the gripping point, and transmits the learning data 150 to the gripping probability calculation unit 111.

The simulation unit 113 calculates the gripping success probability for each gripping point based on the simulation result. In the following description, the gripping success probability calculated by the simulation unit 113 is also described as a comparison probability.

The simulation unit 113 generates probability data 160 including a gripping point and a probability for comparison, and transmits the probability data 160 to the parameter update unit 112. The above is the description of the process of step S102.

Next, the first learning unit 110 executes the first learning process using the learning data 150 and the probability data 160 (step S103). The details of the first learning process will be described later with reference to FIG.

Next, the first learning unit 110 determines whether or not to end the hostile learning (step S104).

For example, the first learning unit 110 determines that the hostile learning ends when the number of executions of the first learning process is larger than the threshold value.

When it is determined that the hostile learning is to be continued, the first learning unit 110 transmits a learning start instruction to the second learning unit 120 (step S105), and then ends the process. When it is determined that the hostile learning is finished, the first learning unit 110 ends the process.

The simulation unit 113 may generate a set of object data, grip points, and simulation results as learning data 150 and output them to the grip probability calculation unit 111 and the parameter update unit 112.

FIG. 6 is a flowchart illustrating an example of the first learning process executed by the first learning unit 110 of the first embodiment.

The gripping probability calculation unit 111 calculates the gripping success probability of the object at an arbitrary gripping point by inputting the learning data 150 into the probability model (step S201). The gripping probability calculation unit 111 transmits the gripping success probability of each gripping point to the parameter updating unit 112.

Next, the parameter update unit 112 updates the probability model based on the gripping success probability and the comparison probability of each gripping point (step S202).

For example, the parameter updating unit 112 updates the parameter (probability model management information 130) that defines the probability model so that the difference between the gripping success probability and the comparison probability of each gripping point becomes small.

Next, the parameter update unit 112 determines whether or not to end the learning of the probability model (step S203).

For example, when the update count of the probability model is larger than the value in the learning count column 404, the parameter update unit 112 determines that the learning of the probability model is completed. Further, the parameter update unit 112 inputs the test input data to the grip probability calculation unit 111, and ends the learning of the probability model when the maximum value of the grip success probability output from the grip probability calculation unit 111 is larger than the target probability. Then it is determined.

If it is determined that the learning of the probabilistic model is to be continued, the first learning unit 110 returns to step S201 and executes the same process. When it is determined that the learning of the probabilistic model is finished, the first learning unit 110 ends the first learning process.

FIG. 7 is a flowchart illustrating an example of processing executed by the second learning unit 120 of the first embodiment.

When the second learning unit 120 receives the learning start instruction, the second learning unit 120 starts the process described below. The learning start instruction is transmitted from the input device 330, the terminal 320, or the first learning unit 110.

First, the second learning unit 120 generates the evaluation input data 180 (step S301). Specifically, the following processing is executed.

The object data generation unit 121 generates a polygon mesh of an object having an arbitrary shape based on the reference object data 101 and the object data generation model. The object data generation unit 121 generates image data by rendering a polygon mesh. The object data generation unit 121 generates learning object data 170 including image data and transmits it to the gripping point data generation unit 123.

As a rendering method, for example, the method described in Non-Patent Document 1 is used. The present invention is not limited to the rendering method.

The gripping point data generation unit 123 determines a plurality of gripping points of the object corresponding to the learning object data 170, and generates the learning object data 170 and the evaluation input data 180 including the gripping points. In addition, one evaluation input data 180 is generated for a set of learning object data 170 and one gripping point. The above is the description of the process of step S301.

Next, the second learning unit 120 transmits the evaluation input data 180 to the first learning unit 110, and acquires the reference probability data 190 (step S302).

When the gripping probability calculation unit 111 of the first learning unit 110 receives the evaluation input data 180, it calculates the gripping success probability of the object at each gripping point, and obtains the reference probability data 190 including the calculated gripping success probability. It is transmitted to the second learning unit 120. One reference probability data 190 is generated for one evaluation input data 180.

Next, the second learning unit 120 executes the second learning process (step S303). The details of the second learning process will be described later with reference to FIG.

Next, the second learning unit 120 determines whether or not to end the hostile learning (step S304).

For example, the second learning unit 120 determines that the hostile learning ends when the number of executions of the second learning process is larger than the threshold value.

When it is determined that the hostile learning is finished, the second learning unit 120 ends the process.

When it is determined that the hostile learning is continued, the second learning unit 120 transmits a learning start instruction to the first learning unit 110 (step S305). After that, the second learning unit 120 ends the process.

FIG. 8 is a flowchart illustrating an example of the second learning process executed by the second learning unit 120 of the first embodiment.

The parameter update unit 122 updates the object data generation model based on the gripping success probability and the target probability of each gripping point (step S401).

Specifically, the parameter updating unit 122 updates the parameter (object data generation model management information 140) that defines the object data generation model so that the difference between the maximum value of the gripping success probability and the target probability becomes small. For example, the parameter that defines the object data generation model at random (object data generation model management information 140) may be updated, or the user may manually update the parameter. The above-mentioned update method is an example and is not limited to this. By updating the object data generation model in this way, it is possible to efficiently generate object data such that the maximum value of the gripping success probability output from the probability model is close to the target probability.

Next, the parameter update unit 122 determines whether or not to end the learning of the probabilistic model (step S402).

For example, when the update count of the object data generation model is larger than the value in the learning count column 405, the parameter update unit 122 determines that the learning of the object data generation model is completed. Further, the parameter update unit 122 inputs the test input data to the gripping probability calculation unit 111, and ends the learning of the probability model when the maximum value of the gripping success probability output from the gripping probability calculation unit 111 is larger than the target probability. Then it is determined.

When it is determined that the learning of the object data generation model is continued, the second learning unit 120 returns to step S401 and executes the same process. When it is determined that the learning of the object data generation model is finished, the second learning unit 120 ends the second learning process.

Next, an example of control of the gripping device 200 based on the probability model will be described. FIG. 9 is a flowchart illustrating an example of control of the gripping device 200 of the first embodiment.

The control device 201 acquires image data of the space (working space) in which the object 210 exists (step S501).

Next, the control device 201 calculates the distribution of the gripping success probability in the target space by using the image data and the probability model (step S502).

Specifically, the control device 201 inputs the image data and the coordinates into the probability model, and calculates the gripping success probability of each coordinate. The control device 201 may execute a process of extracting an object included in the image data or the like by using a known image process.

Next, the control device 201 determines the gripping point of the object 210 based on the distribution of the gripping success probability in the work space, and generates a trajectory plan of the determined gripping point (step S503).

Next, the control device 201 controls the arm 202 based on the trajectory plan (step S504). As a result, the gripping device 200 grips the object 210.

As described above, according to the first embodiment, in the first learning process, a stochastic model for grasping an object having various shapes is generated by using the object data generated based on the object data generation model. it can. Further, by updating the object data generation model based on the difference between the maximum value of the gripping success probability and the target probability, it is possible to efficiently generate the object data so that the maximum value of the gripping success probability is close to the target probability. As a result, more efficient learning of the probabilistic model can be realized.

In the second embodiment, the processing executed by the second learning unit 120 is partially different. Hereinafter, Example 2 will be described with a focus on the differences from Example 1.

The configuration of the computer system of the second embodiment is the same as that of the first embodiment. The hardware configuration of the computer 100 is the same as that of the first embodiment. The software configuration of the computer 100 of the second embodiment is the same as that of the first embodiment.

However, the second learning unit 120 of the second embodiment manages the progress flag. One of "0", "1", and "2" is set in the progress flag. It is assumed that the initial value of the progress flag is set to "0". “1” indicates a state in which the learning of the probabilistic model is stagnant, and “2” indicates a state in which the learning of the probabilistic model is in progress. “0” indicates a state that does not correspond to any of the above.

In the second embodiment, the learning of the second learning process is adjusted according to the progress of the learning of the probabilistic model. As a result, efficient learning of the probabilistic model can be realized.

The process executed by the first learning unit 110 of the second embodiment is the same as that of the first embodiment. In the second embodiment, the processing executed by the second learning unit 120 is partially different. FIG. 10 is a flowchart illustrating an example of processing executed by the second learning unit 120 of the second embodiment.

The processing of steps S301 and S302 is the same as that of the first embodiment. After the process of step S302 is executed, the second learning unit 120 determines whether or not the learning of the probabilistic model is stagnant (step S351). The following methods can be considered as the determination method.

(Determination Method 1) The second learning unit 120 calculates the average value of the gripping success probability from the gripping success probability of each gripping point, and determines whether or not the average value is smaller than the threshold value. When the average value of the gripping success probability is smaller than the threshold value, the second learning unit 120 determines that the learning of the probability model is stagnant. Note that statistical values other than the average value may be used.

(Determination method 2) The second learning unit 120 manages the maximum value of the gripping success probability as a history, and calculates the rate of change based on the previous maximum value and the current maximum value. The second learning unit 120 determines whether or not the difference between the maximum value of the gripping success probability and the target probability is smaller than the first threshold value and the rate of change is smaller than the second threshold value. When the above conditions are satisfied, the second learning unit 120 determines that the learning of the probabilistic model is stagnant.

When it is determined that the learning of the probabilistic model is stagnant, the second learning unit 120 sets the progress flag to "1" (step S352) and executes the second learning process (step S303).

When it is determined that the learning of the probabilistic model is not stagnant, the second learning unit 120 executes the second learning process (step S303).

When it is determined in step S304 that the hostile learning is not completed, the second learning unit 120 determines whether or not the learning of the probabilistic model is progressing (step S353).

For example, the second learning unit 120 determines that the learning of the probability model is progressing when the number of times that the maximum value of the gripping success probability becomes larger than the target probability is larger than the threshold value.

When it is determined that the learning of the probabilistic model is not progressing, the second learning unit 120 transmits a learning start instruction to the first learning unit 110 (step S305). After that, the second learning unit 120 ends the process.

When it is determined that the learning of the probabilistic model is progressing, the second learning unit 120 sets the progress flag to "2" (step S354) and transmits a learning start instruction to the first learning unit 110 (step). S305). After that, the second learning unit 120 ends the process.

The second learning process is also different in the second embodiment. FIG. 11 is a flowchart illustrating an example of the second learning process executed by the second learning unit 120 of the second embodiment.

The second learning unit 120 determines whether or not the progress flag is “0” (step S451).

If it is determined that the progress flag is "0", the second learning unit 120 proceeds to step S453.

When it is determined that the progress flag is not "0", the second learning unit 120 updates the target range (step S452).

Specifically, the second learning unit 120 changes the variable α and the variable β. The variable α and the variable β are variables for designating the range of similarity between the reference object and the object generated based on the updated object data generation model. It is assumed that the second learning unit 120 holds the initial values of the variable α and the variable β. The initial values of the variable α and the variable β may be set via the setting screen 400.

The second learning unit 120 updates the variables α and β based on the difference between the maximum value of the gripping success probability and the target probability. Specifically, it is set as follows.

When the progress flag is "1", the second learning unit 120 sets a value larger than the previous variables α and β based on the difference. As a result, the object data generation model can be updated so that the object data of the object whose change amount from the reference object is smaller than that of the object generated based on the object data generation model before the update is generated.

When the progress flag is "2", the second learning unit 120 sets values smaller than the previous variables α and β based on the difference. As a result, the object data generation model can be updated so that the object data of the object whose amount of change from the reference object is larger than that of the object generated based on the object data generation model before the update is generated.

By executing the above-mentioned processing, the difference between the maximum value of the gripping success probability and the target probability can be propagated to the update of the object data generation model. The above is the description of the process of step S452.

In step S453, the second learning unit 120 sets the target range (step S453).

Next, the parameter update unit 122 updates the object data generation model (step S454). The method of updating the object data generation model is the same as that of the first embodiment.

Next, the object data generation unit 121 generates the learning object data 170 based on the updated object data generation model (step S455).

Next, the parameter update unit 122 calculates the degree of similarity between the reference object and the object corresponding to the learning object data 170 (step S456).

Next, the parameter update unit 122 determines whether or not to end the learning of the object data generation model (step S457). Specifically, the following processing is executed.

The parameter update unit 122 determines whether or not the similarity is within the target range. Specifically, the parameter update unit 122 determines whether or not the similarity is equal to or greater than the variable α and equal to or less than the variable β. When the similarity is within the target range, the parameter update unit 122 determines that the learning of the object data generation model is completed.

When it is determined that the similarity is out of the target range, the parameter update unit 122 determines whether or not the update count of the object data generation model is larger than the value in the learning count column 405.

When the update count of the object data generation model is larger than the value in the learning count column 405, the parameter update unit 122 determines that the learning of the object data generation model is completed. When it is determined that the update count of the object data generation model is equal to or less than the value in the learning count column 405, the parameter update unit 122 determines that the learning of the object data generation model is continued. The above is the description of the process of step S457.

When it is determined that the learning of the object data generation model is continued, the second learning unit 120 returns to step S454 and executes the same process.

When it is determined that the learning of the object data generation model is finished, the second learning unit 120 sets the progress flag to "0" (step S458), and then ends the second learning process.

When the value of the variable α or the variable β is reduced, the object data generation model is updated to generate the object data of the object having a large amount of change from the reference object. When the variable α or the variable β is increased, the object data generation model is updated to generate the object data of the object whose change amount from the reference object is small.

Adjusting the variable β corresponds to adjusting the diversity of objects. Adjusting the variable α corresponds to suppressing the generation of an object with a very large amount of change. The adjustment of the width of the target range corresponds to the adjustment of the amount of change in the object data generation model.

When the object data of an object having a complicated shape is input as the learning data, the learning of the probabilistic model may be stagnant. Therefore, the computer system of the second embodiment updates the object data generation model so that an object having a simple shape is generated when the learning of the probability model is stagnant. That is, the object data generation model is updated so that the amount of change in shape from the reference object is smaller than the object generated from the object data generation model before the update. By inputting the object data of an object having a simple shape, it is possible to prevent the learning of the probabilistic model from stagnation.

On the other hand, when the learning of the probabilistic model is progressing, the second learning unit 120 updates the object data generation model so that an object having a more complicated shape is generated. That is, the object data generation model is updated so that the amount of change in shape from the reference object is larger than that of the object generated from the object data generation model before the update. This makes it possible to efficiently generate a probabilistic model for grasping an object having a more complicated shape.

According to the second embodiment, it is possible to prevent the learning of the probabilistic model from stagnation and realize efficient learning.

In the third embodiment, when the stagnation of the learning process of the probability model is detected, the first learning unit 110 executes the re-learning process of the probability model based on the instruction of the second learning unit 120. Hereinafter, the third embodiment will be described with a focus on the differences from the first embodiment.

FIG. 12 is a diagram showing a configuration example of the computer system of the third embodiment.

The configuration of the computer system of the third embodiment is the same as that of the first embodiment. The hardware configuration of the computer 100 is the same as that of the first embodiment. The software configuration of the computer 100-2 of the third embodiment is the same as that of the first embodiment.

In the third embodiment, the history management information 1200 is stored in the main storage device 301 of the computer 100-1. Other configurations are the same as in the first embodiment.

The history management information 1200 is information for managing the history of the probability model management information 130. In the history management information 1200, the probability model management information 130 and the update date and time are managed in association with each other.

In the third embodiment, the processes executed by the first learning unit 110 and the second learning unit 120 are partially different.

FIG. 13 is a flowchart illustrating an example of processing executed by the first learning unit 110 of the third embodiment.

The first learning unit 110 determines whether or not the received instruction is a re-learning start instruction (step S151).

If it is determined that the received instruction is a learning start instruction, the first learning unit 110 proceeds to step S101.

When it is determined that the received instruction is the re-learning start instruction, the parameter update unit 112 of the first learning unit 110 updates the probability model based on the history stored in the history management information 1200 (step S152). After that, the first learning unit 110 proceeds to step S101.

For example, the parameter update unit 112 reads the history of the probability model having the latest update date and time, and updates the probability model. The above-mentioned update method is an example, and is not limited to this. For example, the probability model may be updated based on the history of the probability model before a certain period.

The processing from step S101 to step S105 of Example 3 is the same as that of Example 1.

FIG. 14 is a flowchart illustrating an example of processing executed by the second learning unit 120 of the third embodiment.

The processes of steps S301 to S302 and steps S303 to S305 of the first embodiment are the same as the processes of the first embodiment. The process of step S351 is the same as the process of the second embodiment.

If it is determined in step S351 that the learning of the probability model is not stagnant, the second learning unit 120 proceeds to step S303.

When it is determined in step S351 that the learning of the stochastic model is stagnant, the second learning unit 120 outputs a re-learning start instruction to the first learning unit 110 (step S361). After that, the second learning unit 120 ends the process.

According to the third embodiment, when the learning of the probabilistic model is stagnant, the first learning unit 110 returns to the past probabilistic model and executes the learning process. As a result, learning stagnation can be avoided.

The present invention is not limited to the above-mentioned examples, and includes various modifications. Further, for example, the above-described embodiment describes the configuration in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to add, delete, or replace a part of the configuration of each embodiment with other configurations.

Further, each of the above configurations, functions, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. The present invention can also be realized by a program code of software that realizes the functions of the examples. In this case, a storage medium in which the program code is recorded is provided to the computer, and the processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the program code itself and the storage medium storing the program code itself constitute the present invention. Examples of the storage medium for supplying such a program code include a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, an SSD (Solid State Drive), an optical disk, a magneto-optical disk, a CD-R, and a magnetic tape. Non-volatile memory cards, ROMs, etc. are used.

In addition, the program code that realizes the functions described in this embodiment can be implemented in a wide range of programs or script languages such as assembler, C / C ++, perl, Shell, PHP, Python, and Java (registered trademark).

Further, by distributing the program code of the software that realizes the functions of the examples via the network, it is stored in a storage means such as a hard disk or memory of a computer or a storage medium such as a CD-RW or a CD-R. , The processor provided in the computer may read and execute the program code stored in the storage means or the storage medium.

In the above-described embodiment, the control lines and information lines show what is considered necessary for explanation, and do not necessarily indicate all the control lines and information lines in the product. All configurations may be interconnected.

100 Computer 101 Reference object data 110 First learning unit 111 Gripping probability calculation unit 112, 122 Parameter updating unit 113 Simulation unit 120 Second learning unit 121 Object data generation unit 123 Gripping point data generation unit 130 Probability model management information 140 Object data generation Model management information 150 Learning data 160 Probability data 170 Learning object data 180 Evaluation input data 190 Reference probability data 200 Gripping device 201 Control device 202 Arm 210 Object 220 Gripping point 300 Processor 301 Main storage device 302 Sub storage device 303 Network interface 304 IO interface 310 Network 320 Terminal 330 Input device 331 Output device 400 Setting screen 1200 History management information

Claims (8)

A computer system in which a gripping device learns a control algorithm for gripping an object.
The computer system includes an arithmetic unit and at least one computer having a storage device connected to the arithmetic unit.
The at least one computer
Output the gripping success probability of the object by using the object data including the information about the shape of the object, the gripping point which is the reference of the gripping position of the object, and the learning data including the information indicating the success or failure of gripping the object. First learning process to generate the first model to be
The second learning process for generating the second model that outputs the object data of the object whose shape of the reference object is deformed is executed.
The at least one computer is used in the second learning process.
Based on the second model, the object data of the first object obtained by deforming the shape of the reference object is generated.
The gripping point of the first object is determined,
The object data of the first object and a plurality of sets of grip points of the first object are input to the first model.
A computer system characterized in that the second model is updated based on the difference between the maximum value of the gripping success probability of the first object and the target probability output from the first model. The computer system according to claim 1.
The at least one computer is used in the second learning process.
When the learning of the first model is stagnant, the second model is updated so that the object data of the object whose shape change amount from the reference object is smaller than that of the first object is generated.
When the learning of the first model is progressing, the second model is updated so that the object data of the object whose shape change from the reference object is larger than that of the first object is generated. Computer system. The computer system according to claim 2.
The at least one computer is used in the second learning process.
A target range of similarity between the shape of the object corresponding to the object data generated based on the second model and the shape of the reference object is set.
The difference between the maximum value of the gripping success probability of the first object and the target probability becomes small, and the shape and reference of the second object corresponding to the object data generated based on the updated second model. A computer system characterized in that the second model is updated so that the degree of similarity with the shape of an object is within the target range. The computer system according to claim 1.
Holds history management information for managing the update history of the first model,
The at least one computer is used in the second learning process.
When the maximum value of the gripping success probability of the first object output from the first model is equal to or less than the target probability, the statistical value of the gripping success probability of the first object and the maximum value of the gripping success probability of the first object. Calculate one of the changes in the value as an evaluation value,
Based on the evaluation value, it is determined whether or not the learning of the first model is stagnant.
A computer system characterized in that when it is determined that learning of the first model is stagnant, the first model is updated using the history management information. A machine learning method performed by a computer system that learns a control algorithm for a gripping device to grip an object.
The computer system is composed of an arithmetic unit and at least one computer having a storage device connected to the arithmetic unit.
The machine learning method is
The at least one computer uses object data including information on the shape of the object, a gripping point as a reference of a position for gripping the object, and learning data including information indicating success or failure of gripping the object. The step of executing the first learning process for generating the first model that outputs the gripping success probability of the object,
A step of executing a second learning process for generating a second model for outputting the object data of an object whose shape of the reference object is deformed based on the object data of the reference object is included.
The second learning process is
A first step in which the at least one computer generates the object data of the first object obtained by deforming the shape of the reference object based on the second model.
A second step in which the at least one computer determines the gripping point of the first object,
A third step in which the at least one computer inputs the object data of the first object and a plurality of sets of grip points of the first object into the first model.
Machine learning comprising a fourth step of updating the second model based on the difference between the maximum value of the gripping success probability of the first object and the target probability output from the first model. Method. The machine learning method according to claim 5.
The fourth step is
When the learning of the first model is stagnant, the second is such that at least one computer generates the object data of an object whose shape change amount from the reference object is smaller than that of the first object. Steps to update the model and
When the learning of the first model is in progress, the second is such that at least one computer generates the object data of an object whose shape change from the reference object is larger than that of the first object. A machine learning method characterized by including steps to update the model. The machine learning method according to claim 6.
The fourth step is
A step in which the at least one computer sets a target range of similarity between the shape of an object corresponding to the object data generated based on the second model and the shape of the reference object.
The at least one computer makes a difference between the maximum value of the gripping success probability of the first object and the target probability small, and corresponds to the object data generated based on the updated second model. (Ii) A machine learning method comprising a step of updating the second model so that the similarity between the shape of the object and the shape of the reference object is within the target range. The machine learning method according to claim 5.
The computer system holds history management information for managing the update history of the first model.
The machine learning method is
A step in which the at least one computer calculates either a statistical value of the gripping success probability of the first object or a change amount of the maximum value of the gripping success probability of the first object as an evaluation value.
A step in which the at least one computer determines whether or not the learning of the first model is stagnant based on the evaluation value.
A machine characterized in that, when it is determined that the learning of the first model is stagnant, the at least one computer includes a step of updating the first model using the history management information. Learning method.

Images