JP2024101163A - Information processing method, information processing device, robot system, control method of robot system, article manufacturing method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology to improve control accuracy for a control object using a learned model.

SOLUTION: An information processing method executes processing for acquiring first data by operating an object corresponding to a control object in an environment different from an operation environment of the control object, and acquiring a learned model by machine learning using the first data. The information processing method executes processing for acquiring second data for causing the control object to be operated in the operation environment using the learned model, and determination processing for causing the control object to be operated in the operation environment using the second data and determining the result. The information processing method executes learning processing (steps S31-S35) for acquiring third data on an operation in which the result in the determination processing is determined to be successful, and causing the learned model to perform learning by machine learning using the third data.

SELECTED DRAWING: Figure 13

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an information processing method, an information processing device, a robot system, a control method for a robot system, a method for manufacturing an article, a program, and a recording medium.

Traditionally, imitation learning has been known as a method for machine learning a robot control method, in which a controlled object imitates an imitation object through machine learning by moving the object. Imitation learning allows a robot to learn with a small amount of data, and can make the robot perform actions that would be difficult to do through teaching at a production site. As a technique for improving the learning accuracy in imitation learning, a technique has been disclosed in which data is selected at the stage of collecting data for training a machine learning model, and only data desirable for learning is selectively collected (see Patent Document 1).

JP 2021-107970 A

In imitation learning, when it is desired to learn an action more easily than learning a control method using a robot, or when it is desired to collect a large amount of data, data may be collected from a system other than the robot that is the actual controlled object, such as human actions or simulations. In such cases, differences arise between the person or simulation that is the imitation object from which the action data is collected, and the data obtained from the robot that is the controlled object, and this becomes noise in learning. This poses the problem of a decrease in the accuracy of control of the controlled object using a trained model obtained by machine learning from data containing noise.

The present invention aims to provide a technology that can improve the accuracy of controlling the controlled object.

A first aspect of the present invention is an information processing method that executes a process of acquiring first data by operating an object corresponding to a control object in an environment different from the operating environment of the control object, and acquiring a trained model by machine learning using the first data, a process of acquiring second data for operating the control object in the operating environment using the trained model, a determination process of operating the control object in the operating environment using the second data and determining the result, and a learning process of acquiring third data about an operation whose result is determined to be successful in the determination process, and training the trained model by machine learning using the third data.

A second aspect of the present invention is an information processing device including an information processing unit, characterized in that the information processing unit executes a process of acquiring first data by operating an object corresponding to the control object in an environment different from the operating environment of the control object, and acquiring a trained model by machine learning using the first data, a process of acquiring second data for operating the control object in the operating environment using the trained model, a determination process of operating the control object in the operating environment using the second data and determining the result, and a learning process of acquiring third data about an operation whose result is determined to be successful in the determination process, and training the trained model by machine learning using the third data.

The present invention can improve the accuracy of control of the controlled object.

1 is a conceptual diagram of a robot motion learning device according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a recording device included in the robot motion learning device according to the embodiment of the present invention. FIG. 2 is a control block diagram of a control device provided in the robot motion learning device according to the embodiment of the present invention. FIG. 4 is an explanatory diagram of a correspondence relationship between an imitation target area and a control target area according to the embodiment of the present invention. FIG. 2 is an explanatory diagram of a learning device according to an embodiment of the present invention. 10 is a flowchart showing the flow of a learning process executed by the information processing device according to the embodiment of the present invention. FIG. 2 is a diagram showing an example of a controlled task executed by a controlled robot according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a controlled task executed by a controlled robot according to an embodiment of the present invention. FIG. 4 is an explanatory diagram of a determination unit according to the embodiment of the present invention. FIG. 4 is an explanatory diagram of a GUI displayed on a display unit according to an embodiment of the present invention. 11 is an explanatory diagram of a detailed setting window displayed on a display unit according to an embodiment of the present invention. FIG. 10 is a flowchart showing the flow of a determination process executed by the information processing device according to the embodiment of the present invention. 10 is a flowchart showing the flow of a relearning process executed by the information processing device according to the embodiment of the present invention.

The following describes in detail the embodiment of the present invention with reference to the drawings. Note that the configuration shown below is merely an example, and the details of the configuration may be modified as appropriate by the parties concerned without departing from the spirit of the invention. Also, the numerical values used in the following description are for reference only and are merely examples.

<Robot system configuration>
Fig. 1 is a conceptual diagram showing a robot motion learning method as a robot system according to this embodiment. Fig. 2 is a conceptual diagram of a recording device 400 included in a robot motion learning device 100. The robot motion learning device 100 is composed of the recording device 400, an area to be imitated 200, an area to be controlled 300, a learning device 1 that generates robot motions, and a judging section 2 that judges whether the motion generated by the learning device 1 is successful or not.

In the robot motion learning device 100, first, the action that the subject 201 to be imitated actually wants the controlled robot 301 to perform in the area to be imitated 200 is executed as the task to be imitated 202. The action of the subject 201 to be imitated is detected by the imitated action sensor 203. The imitated action sensor 203 detects the action data (second data) of the subject 201 to be imitated accompanying the action, such as hand position information, the state of the task to be imitated 202, and the state of the environment in the area to be imitated 200, and records them in the recording device 400 as initial collection data 401.

In the robot motion learning device 100, a series of steps is repeated multiple times to detect the motion of the subject 201 to be imitated, which is executed as the task 202 to be imitated, using the imitated motion sensor 203, and then the learning device 1 performs an initial learning process. The initial learning process is a process of acquiring a learned model that can output information on the next motion to be performed in a certain situation, using as input the recorded hand position of the subject 201 to be imitated, the state of the task 202 to be imitated, and the state of the environment within the area 200 to be imitated.

After the initial learning process is completed, the learning device 1 executes the inference process. The inference process is a process in which the operation data of the controlled robot 301, the state of the controlled task 302, and the environmental state within the controlled area 300 detected by the control operation sensor 303 are input, and the operation data to be performed next by the controlled robot 301 is output. The controlled robot 301 generates an operation by reproducing the operation data output as a result of the inference process.

The judgment unit 2 receives as input the current operation data of the controlled robot 301 during or after the completion of the operation, the state of the controlled task 302, and the value detected by the control operation sensor 303 regarding the environmental state within the controlled area 300, and judges whether the task based on the generated operation has been successful. Details of the judgment method used by the judgment unit 2 will be described later.

If the task based on the generated action is judged to be successful, the judgment unit 2 performs a storage process to store the action data input at the time of judgment and the value detected by the control action sensor 303 as success data (third data) 402 in the recording device 400. At this time, the judgment unit 2 labels and stores the initial collection data 401 and the success data 402 so that they can be clearly distinguished inside the recording device 400, as shown in FIG. 2.

After completing the series of operations up to the time when the success data 402 is stored in the recording device 400, the learning device 1 adds the success data 402 stored in the recording device 400 as a new input and performs a re-learning process. By performing the re-learning process, the robot motion learning device 100 can gradually close the gap between the object to be imitated and the environment surrounding the object to be imitated, and the object to be controlled and the environment surrounding the object to be controlled. This allows the robot motion learning device 100 to improve the accuracy of control of the control target robot 301, which is the object to be controlled, and to increase the success rate of the desired task.

<Control block diagram of the robot motion learning device>
Fig. 3 is a control block diagram of the robot motion learning device 100. As shown in Fig. 2, the information processing device 700 of the robot motion learning device 100 has a CPU 701 which is a type of processor and serves as an information processing unit. The CPU 701 also functions as a learning device 1 and a determination unit 2.

The information processing device 700 also has a ROM 702, a RAM 703, and a HDD 704 as a recording unit. Also, as devices for operating the system, a display 707 as a display unit, and a keyboard 708 and a mouse 709 as input units are connected. The information processing device 700 is also connected to the imitation motion sensor 203, the control motion sensor 303, and the controlled robot 301 on the imitation target side, and is capable of exchanging data with these.

The HDD 704 is configured to be able to read and write data and to retain records. Initially collected data 401, successful data 402, and a learned model 706 generated by the learning device 1 through learning are stored in the HDD 704. The HDD 704 also stores a program 705 for calling up the learned model 706. When the robot motion learning device 100 wants to operate the controlled robot 301, it can execute the program 705 to obtain data detected by the control motion sensor 303 and transmit the motion data to the controlled robot 301. This HDD 704 constitutes the recording device 400 shown in FIG. 1.

<Relationship between objects to be imitated and objects to be controlled>
Fig. 4 is a diagram showing the correspondence between objects to be imitated and objects to be controlled. As shown in Fig. 4, an area to be imitated 200 includes an actor to be imitated 201, which is an object to be imitated, a task to be imitated 202, an imitated action sensor 203, and a light 204 that illuminates the area to be imitated 200. An area to be controlled 300 includes a robot to be controlled 301, which is an object to be controlled, a task to be controlled 302, a control action sensor 303, and a light 304 that illuminates the area to be controlled 300.

In the robot motion learning device 100, first, a task to be imitated 202 is placed in an area to be imitated 200. The task to be imitated 202 is placed in the area to be imitated 200 and is the same as a control target task 302 that is actually desired to be performed by a control target robot 301. In other words, the control target task 302 is placed in the area to be controlled 300 and is the same as the task to be imitated 202.

The task to be imitated 202 may be any task that changes from an initial state to a desired state through the operation of a robot, such as picking and placing a workpiece, assembling parts into an industrial product, or folding a towel.

The imitation action sensor 203 is composed of, for example, a camera, a thermometer, a microphone, a tactile sensor, a torque sensor, an acceleration sensor, etc., and detects the state of the task to be imitated 202, the movement of the subject to be imitated 201, and the state of the environment within the area to be imitated 200. The imitation action sensor 203 may be a component possessed by the subject to be imitated 201. The environment of the area to be imitated 200 includes the brightness, installed objects, background, temperature, sounds, etc. within the area to be imitated 200.

The subject 201 to be imitated performs the task 202 to be imitated. The subject 201 to be imitated may be the same robot as the control target robot 301, which is the controlled object, a different robot, or a human. In other words, the subject 201 to be imitated may be any object that corresponds to the control target robot 301, which is the controlled object. The subject 201 to be imitated also has components that correspond to those possessed by the control target robot 301 (for example, a component that grasps a workpiece corresponding to the robot hand 301a).

When the subject of the action to be imitated 201 is the same or a different robot as the robot to be controlled 301, the worker operates the subject of the action to be imitated 201 to perform the task using a joystick, direct teaching, point teaching, master-slave robot, etc. When the subject of the action to be imitated 201 is a human, the human needs to wear a tool or the like that can observe the action data such as the hand position and tactile values required to control the robot to be controlled 301.

The imitation motion sensor 203 detects the motion data of the subject 201 to be imitated and the state of the environment within the area to be imitated 200 at a preset sampling period while the subject 201 to be imitated is performing an action. The sampling period at which the imitation motion sensor 203 detects the motion data and the state of the environment is set to a period (e.g., 200 Hz) that is sufficiently short for the motion of the subject 201 to be imitated to be reproduced by the controlled robot 301.

The series of actions of the subject 201 to be imitated and detection by the imitation action sensor 203 constitutes one sample of the initially collected data 401 in this embodiment. The learning device 1 may acquire only one sample of the initially collected data 401 and learn from it, or may acquire any number of samples of the initially collected data 401 and then learn from it.

The control motion sensor 303 is identical to the imitation motion sensor 203, and is installed in the controlled area 300 so as to have the same relative position as the imitation motion sensor 203 with respect to the imitation area 200. In other words, if the imitation subject 201 has the imitation motion sensor 203, the control motion sensor 303 is configured to be possessed by the controlled robot 301. The control motion sensor 303 is composed of, for example, a camera, a thermometer, a microphone, a tactile sensor, a torque sensor, an acceleration sensor, etc., and detects the state of the controlled task 302, the movement of the controlled robot 301, and the state of the environment within the controlled area 300.

The environment of the controlled area 300 includes brightness, objects, background, temperature, sounds, etc. within the controlled area 300. The environment within the area to be imitated 200 and the environment within the controlled area 300 do not need to correspond to each other, and there may be differences between the environment within the area to be imitated 200 and the environment within the controlled area 300. In this embodiment, for example, the illuminance of the light 204 provided within the area to be imitated 200 is different from the illuminance of the light 304 provided within the controlled area 300. In other words, the subject to be imitated 201 is configured to operate in an environment different from the operating environment of the robot to be controlled 301.

The controlled robot 301 is a robot manipulator having at least one or more drive units and a robot hand 301a, and at least one or more drive units can be controlled by the operation data output from the learning device 1. The controlled robot 301 is installed at a position in the controlled area 300 where the learned operation using the imitation subject 201 described below can be realized, and the robot hand 301a is moved to an operation start position. The operation start position of the robot hand 301a is a relative position in the controlled area 300 that has the same positional relationship as the operation start position of the imitation subject 201 with respect to the imitation subject area 200 when the initial collection data 401 is collected.

After the robot hand 301a of the controlled robot 301 moves to the operation start position, the control operation sensor 303 detects the current state of the controlled task 302, the operation of the controlled robot 301, and the state of the environment within the controlled area 300. The various detected data are input to the learned learning device 1.

The learning device 1 outputs operation data (e.g., the opening and closing degree of the robot hand 301a, etc.) that the controlled robot 301 should perform next based on the various current data input and the various past data input up to that point. The controlled robot 301 operates based on the operation data (e.g., command values for the tip position of the robot hand 301a, etc.) output from the learning device 1.

In the controlled area 300, various data detected by the control operation sensor 303 is input to the learning device 1, and the controlled robot 301 is operated based on the operation data output from the learning device 1 for a preset number of steps. Note that the preset number of steps is preferably the same as the number of steps used to acquire the initial collection data 401, but may be a number of steps longer than that.

<Learning process>
Next, a learning process executed by the learning device 1 using the initially collected data 401 as teacher data (first data) will be described. Fig. 5 is a diagram for explaining machine learning executed by the learning device 1. The learning device 1 of this embodiment is configured by machine learning such as LSTM (Long Short Term Memory).

As shown in Figure 5, LSTM is a supervised learning method that predicts future results from current and past sampling data. For example, LSTM takes n sampling data points of hand position, tactile values, surrounding environment, and task state from any time t = t to t = t + n as input. Then, LSTM takes into account the time-series relationships between the input data and learns the hand position, tactile values, surrounding environment, and task state for the future time t = t + n + 1.

The learning device 1 executes an inference process using a trained model generated by machine learning using the LSTM. The learning device 1 performs an inference process using the motion data of the controlled robot 301, the state of the controlled task 302, and the environmental state in the controlled area 300 detected by the control motion sensor 303 as inputs, and outputs motion data that the controlled robot 301 should perform next.

The data input to the LSTM may be one type of data, or may be a combination of hand position and tactile values. In this embodiment, the LSTM is given as a specific example, but the algorithm used may be a Neural Network, an RNN (Recurrent Neural Network), a Transformer, or the like, as long as it is capable of predicting time-series data.

<Flow of processing executed in the learning process>
6 is a flowchart showing a flow of processing executed in the robot motion learning device 100 when the CPU 701 functioning as the learning device 1 performs machine learning using the initial collected data 401. In the robot motion learning device 100, first, the subject 201 to be imitated executes the task 202 to be imitated (S1). Next, in the robot motion learning device 100, motion data accompanying the motion of the subject 201 to be imitated, the state of the task 202 to be imitated detected by the imitated motion sensor 203, and the state of the environment in the area 200 to be imitated are acquired (S2). In this processing, the CPU 701 saves the motion data accompanying the motion of the subject 201 to be imitated, the state of the task 202 to be imitated detected by the imitated motion sensor 203, and the state of the environment in the area 200 to be imitated as the initial collected data 401 in the HDD 704.

Next, the CPU 701 determines whether the number of samples of the initially collected data 401 used in the machine learning is sufficient (S3). In this process, the CPU 701 determines whether a preset number of samples of the initially collected data 401 have been collected. If the CPU 701 determines that the number of samples of the initially collected data 401 does not reach the preset number of samples (No), the process returns to step S1, and the processes of steps S1 to S3 are repeated until the preset number of samples is reached. Here, the preset number of samples is, for example, 20 to 100 samples, but as described above, it may be 1 sample, and no specific, clear standard is set.

If it is determined in the processing of step S3 that the number of samples of the initially collected data 401 has reached a preset number of samples (Yes), the CPU 701 executes machine learning to generate a trained model 706 (S4). In this processing, the CPU 701 executes machine learning using LSTM with the initially collected data 401 as input, and acquires the trained model 706. This enables the CPU 701 to operate the controlled robot 301 using the operation data at each time point output by the inference processing using the trained model 706.

The processing of steps S1 to S4 constitutes a process of operating the subject 201 to be imitated in an environment different from the operating environment of the robot 301 to be controlled, acquiring initial collected data 401, and acquiring a trained model 706 by machine learning using the initial collected data 401.

<Determination section>
Next, a detailed description will be given of the determination by the determination unit 2. In the robot motion learning device 100, after the control target robot 301 has completed a prescribed number of steps of motion, one or more of the motion data of the control target robot 301, the environment in the control target area 300, and the state of the control target task 302 are input to the determination unit 2. The determination unit 2 uses the various input data to determine whether the control target robot 301 has succeeded in the control target task 302.

As a specific example of the determination method, a pick-and-place task in which the controlled robot 301 grasps a workpiece, carries it to a goal point, and places it there, as shown in FIG. 7, is the controlled task 302. In the example shown in FIG. 7, the robot hand 301a of the controlled robot 301 grasps a workpiece 501 placed at a start point, and carries it along an arbitrary route to the goal point 502 and places it there.

The goal point 502 is set on a weighing scale 503, which is an example of a control operation sensor 303, and when the controlled robot 301 correctly transports and places the workpiece 501, the weight of the workpiece 501 is measured by the weighing scale 503. Therefore, the user can set a threshold value for the measurement value of the weighing scale 503 that corresponds to the actual control environment, and set it so that the controlled task 302 is determined to have been successful when a weight equal to or greater than the threshold is detected. In other words, the judgment unit 2 judges that the controlled task 302 is successful when a weight equal to or greater than the threshold is detected, by the user setting a threshold value for the measurement value of the weighing scale 503 that corresponds to the actual control environment.

In this way, the judgment unit 2 judges success or failure using a threshold based on the environment of the controlled area 300. This allows the robot motion learning device 100 to judge success or failure through simple processing such as comparing the results of the motion of the controlled robot 301 with the threshold, thereby reducing the processing load in the judgment by the judgment unit 2.

Next, as a specific example of the judgment method, a case will be described in which the controlled robot 301 shown in FIG. 8 is a controlled task 302 of pulling a cable, and the success or failure of the controlled task 302 is judged using machine learning. In the example shown in FIG. 8, the controlled robot 301 is tasked with pulling a cable 511 around a workpiece 512 from a start point 513, passing through the periphery of relay points 514 and 515, and inserting the tip of the cable 511 into a goal point 516.

For this reason, in the example shown in FIG. 8, even if cable 511 is inserted into goal point 516, if cable 511 passes through the inner circumference of relay points 514 and 515, controlled task 302 will not be successful. One method for determining the success or failure of such a task is to determine the relative positional relationship between workpiece 512 and cable 511 from their appearance after the operation of pulling cable 511 is completed, and the determination can be automated by using machine learning in determination unit 2.

Figure 9 is a diagram for explaining a case where the judgment is automated using machine learning in the judgment unit 2. In the example shown in Figure 9, the judgment unit 2 is configured by machine learning such as a VAE (Variational Auto Encoder). A VAE has the property that when unknown data is input, it cannot output the data in its original state.

As shown in FIG. 9, during learning, the VAE inputs only successful external images of the workpiece 512 and cable 511 after the completion of the imitation target task 202, which has the same content as the controlled task 302, and learns to output the same data as the input. This allows latent features of the external image when the task is successful to be extracted within the algorithm. Note that the input data is not limited to image data, and may include motion data of the imitation target actor 201 performing the imitation target task 202, and environmental information within the imitation target area 200 detected by the imitation motion sensor 203.

When the VAE determines whether the controlled task 302 is successful, the judgment unit 2 inputs an external image of the controlled task 302 after the controlled robot 301 has completed its operation. Since the VAE learns using only data from when the imitation task 202 is successful as input, when the VAE inputs an external image from when the controlled task 302 is successful, it can compress the input external image internally and then restore it to its original state. On the other hand, when the VAE inputs an external image from when the controlled task 302 fails, it cannot compress it and then restore it to its original state.

Therefore, the judgment unit 2 can determine the success or failure of the operation of the controlled robot 301 by taking the difference between the input image and the output image of the VAE as the degree of abnormality and setting a threshold value for the degree of abnormality. Note that, when the judgment unit 2 learns the VAE using data other than images, it can calculate the degree of abnormality by inputting data similar to the data used for learning at the time of judgment. Also, in this embodiment, the VAE is given as a specific example, but the algorithm used can be any algorithm that allows unsupervised learning that learns only data of successful cases, and may be a GAN (Generative Adversarial Network), etc.

<Operation process of the controlled object>
Next, a description will be given of the judgment of the motion of the control target robot 301. In the robot motion learning device 100, after the control target robot 301 completes a prescribed number of steps of motion, one or more of the motion data of the control target robot 301, the environment in the control target area 300, and the state of the control target task 302 are input to the judgment unit 2.

Sampling data such as the operation data of the controlled robot 301, the environment in the controlled area 300, and the state of the controlled task 302 input to the judgment unit 2 must be data that is set to be input in advance. For example, if it is set to judge the success or failure of an operation only from an image of the imitation target task 202 at a certain time after the operation is completed, only an image of the controlled task 302 at a certain time after the controlled robot 301 operates is input to the judgment unit 2.

The judgment unit 2 determines whether the input data satisfies a preset condition, and if the input data satisfies the condition, judges that the operation was successful. If it is judged that the operation was successful, the judgment unit 2 performs a storage process to store the operation data input at the time of the judgment and the value detected by the control operation sensor 303 as success data 402 in the recording device 400. In other words, if the judgment unit 2 judges that the operation was successful, it stores the operation data output in the inference process and the value detected by the control operation sensor 303 when the controlled robot 301 operates based on the operation data as success data 402 in the recording device 400.

On the other hand, if it is determined that the movement has failed, the determination unit 2 discards the movement data input at the time of the determination and the value detected by the control movement sensor 303 without storing them in the recording device 400. In this way, the robot movement learning device 100 can alleviate the constraints on the storage capacity of the recording device 400 by discarding the movement data input when the movement has failed and the value detected by the control movement sensor 303.

After the operation and judgment process is completed, the robot operation learning device 100 may proceed to a re-learning process described below to perform re-learning process of the learning device 1, or may repeat the operation process of operating the controlled object without performing the re-learning process.

<Display section>
The setting of judgment conditions for the operation of the control target robot 301 and confirmation of the judgment results can be performed using a GUI (Graphical User Interface) 600 shown in Fig. 10. The GUI 600 is displayed on a display 707.

When setting judgment conditions for the operation of the controlled robot 301 or checking the judgment results, a judgment display screen 600a is displayed on the GUI 600. The judgment display screen 600a displays an execution ID display box 601, a success/fail judgment box 602, an input image display box 603, a judgment result image display box 604, a parameter monitor 605, and a setting window 607. The judgment display screen 600a also displays a detailed settings button 612, an add setting item button 613, and a reset setting item button 614.

The setting window 607 displays a condition list 608 that displays a list of judgment conditions. The condition list 608 displays a condition label setting box 609 for setting a label to confirm the contents of the condition, a threshold setting box 610 for setting a threshold, and a magnitude relationship setting box 611 that indicates the magnitude relationship that must be satisfied with respect to the threshold.

If you want to add a setting item, you can add a condition by selecting the add setting item button 613. You can delete the corresponding condition by selecting the delete condition button 615 linked to each condition. Also, you can delete all conditions by selecting the reset setting item button 614.

The user can operate the keyboard 708 and mouse 709 to input any value into each box displayed in the setting window 607 to set conditions. When multiple conditions are set, the final success/failure determination is determined by the logical product of all the conditions. In addition, the user can independently set a logical expression that must be satisfied by the multiple conditions, using the detailed setting window 620 (see FIG. 11) that is displayed by operating the detailed setting button 612.

As shown in FIG. 11, the detailed settings window 620 displays a condition list 623 in which the same conditions as those in the settings window 607 are entered. The detailed settings window 620 also displays a logical formula setting box 621, a term number setting box 622, a done button 624, an add setting item button 625, a reset setting item button 626, and an erase condition button 627. Note that the add setting item button 625 has the same function as the add setting item button 613, the reset setting item button 626 has the same function as the reset setting item button 614, and the erase condition button 627 has the same function as the erase condition button 615.

The condition list 623 is synchronized with the condition list 608 in the setting window 607 when the detailed settings button 612 is selected. The user can set any condition by editing the condition list 623 while adding or removing conditions using the add setting item button 625, reset setting item button 626, and delete condition button 627.

The user can also set the detailed relationships between multiple conditions by setting an arbitrary term number in the term number setting box 622 and inputting the logical expression that each condition must satisfy in the logical expression setting box 621.

After completing the condition settings, by selecting the done button 624, the condition list 608 displayed in the setting window 607 is synchronized with the condition list 623, the display of the detailed setting window is closed, and the judgment display screen 600a is displayed.

When the operation of the controlled robot 301 is started after the settings are completed, the execution ID display box 601 displays the execution ID that is automatically assigned for each operation. By checking the execution ID displayed in the execution ID display box 601, the user can identify which operation is currently being performed. Here, an operation is a series of operations from when the controlled robot 301 starts to when the controlled task 302 ends, and one operation ends one controlled task 302.

In the parameter monitor 605, data (parameters) obtained from the results of the operation of the controlled robot 301 with respect to the set conditions are displayed in a parameter list 606. In the parameter list 606, boxes that display data that meets the conditions are displayed lit, and boxes that display data that do not meet the conditions are displayed unlit.

The success/failure judgment box 602 displays the overall success/failure judgment result by the judgment unit 2. The success/failure judgment box 602 is initially displayed as off, and if it is determined that the controlled robot 301 has failed to operate, the words "Judgment NG" are displayed as lit, indicating that the judgment result is a failure. On the other hand, if it is determined that the controlled robot 301 has succeeded in operating, the success/failure judgment box 602 remains off.

When an image is used as input to the judgment unit 2, the input image is displayed in the input image display box 603. When an image is used as input to the judgment unit 2 and the judgment result is obtained as an image, the judgment result image can be displayed in the judgment result image display box 604. When the judgment result is a failure, the image displayed in the judgment result image display box 604 is displayed as a heat map in which the failed areas are highlighted in color.

<Flow of processing executed in the operation process of the controlled object>
FIG. 12 is a flowchart showing the flow of processing executed when the controlled robot 301 is operating in the controlled area 300.

In the robot motion learning device 100, first, the conditions for the CPU 701 functioning as the judgment unit 2 to judge the success or failure are set using the GUI 600 (S11). In this process, the CPU 701 sets the conditions for judging the success or failure of the control target task 302 executed by the control target robot 301 through the user's operation using the GUI 600 and the input units of the keyboard 708 and mouse 709.

Next, the CPU 701 detects the current state of the controlled robot 301 using the control operation sensor 303 (S12). Next, the CPU 701 inputs the data acquired in the process of step S12 to the learning device 1, and operates the controlled robot 301 according to various data output by the inference process using the learned model 706 of the learning device 1 (S13). In this process, the CPU 701 operates the controlled robot 301 based on the operation data of the controlled robot 301 output by the above-mentioned inference process, the environment in the controlled area 300, and the state of the controlled task 302. The processes of steps S12 and S13 constitute a process of acquiring operation data for operating the controlled robot 301 in the environment in the controlled area 300 using the learned model 706.

Next, the CPU 701 determines whether the control target task 302 has been completed (S14). In this process, if the CPU 701 determines that the control target task 302 has not been completed (No), the CPU 701 returns to step S12 and repeats steps S12 to S14 until the control target task 302 is completed. On the other hand, if the CPU 701 determines that the control target task 302 has been completed (Yes), the CPU 701 proceeds to step S15.

In the process of step S15, the CPU 701 judges whether the controlled task 302 is successful and displays the result of the judgment (S15). In this process, the CPU 701 uses the judgment conditions set in the process of step S11 to determine whether the data corresponding to the input conditions satisfies the conditions. The CPU 701 also performs a process to display the result of the judgment on the GUI 600 of the display 707.

By displaying the judgment conditions and the judgment results on the GUI 600, the robot motion learning device 100 allows the user to easily check whether the result of the motion of the controlled robot 301 was judged appropriately and whether the result was successful. The processes of steps S11 and S15 constitute a process of displaying the judgment conditions of the judgment process and the judgment results on the display 707.

Next, the CPU 701 determines whether the control target task 302 has been successful (S16). In this process, the CPU 701 determines whether the control target task 302 has been determined to be successful in the process of step S15, and if it has been determined that the control target task 302 has been successful (Yes), the process proceeds to step S18. On the other hand, if it has been determined that the control target task 302 has failed (No), the CPU 701 proceeds to step S17. The processes of steps S15 and S16 constitute a determination process that operates the control target robot 301 in the environment of the control target area 300 using the operation data and determines the result.

In the processing of step S17, the CPU 701 discards the various data output by performing inference processing from the learned model 706 during the execution of the current controlled task 302 (S17), and returns the processing to step S12.

In the process of step S18, the CPU 701 performs an inference process using the learned model 706 when executing the current control target task 302, and stores the various data output as success data 402 in the recording device 400 (S18). This process of step S18 constitutes a storage process that acquires success data 402 for an operation whose result is determined to be successful in the determination process.

Next, the CPU 701 determines whether the number of samples of the success data 402 stored in the recording device 400 has reached a preset number of samples (S19). In this process, if it is determined that the preset number of samples has been reached (Yes), the CPU 701 ends the operation of the controlled robot 301. On the other hand, if it is determined that the preset number of samples has not been reached (No), the CPU 701 returns the process to step S12.

As a result, in the robot motion learning device 100, the processes of steps S12 to S19 are repeated until the number of samples of the successful data 402 stored in the recording device 400 reaches the preset number of samples.

<Relearning process>
Next, re-learning is performed by the learning device 1 using the success data 402 determined by the determination unit 2 to be a successful operation. In the re-learning, the learning device 1 determines which data to use for the re-learning from the initial collected data 401 and the success data 402 stored in the recording device 400, depending on the number of samples of the success data 402 stored in the recording device 400.

<Flow of processing executed in the re-learning process>
13 is a flowchart showing the flow of a re-learning process executed when the CPU 701 functioning as the learning device 1 re-learns the trained model 706 using the success data 402. The CPU 701 first determines whether or not a sufficient number of samples of the success data 402 have been acquired (S31). In this process, the CPU 701 determines whether or not the number of samples of the success data 402 stored in the recording device 400 is a sufficient number of samples. Here, the sufficient number of samples is, for example, the same number of samples as the initial collected data 401 collected in the learning process.

If it is determined in the process of step S31 that the number of samples of the successful data 402 is sufficient (Yes), the CPU 701 performs re-learning by machine learning using the successful data 402 stored in the recording device 400 (S32). In this process, the CPU 701 performs machine learning using the successful data 402 stored in the recording device 400, and does not use the initially collected data 401. In the process of step S32, the CPU 701 performs machine learning using the same algorithm as the machine learning performed in the learning process, obtains a trained model 706, and ends the re-learning process.

In this way, when the number of samples of the successful data 402 is equal to or greater than the total number of samples of the initially collected data 401 as a predetermined number of samples, the CPU 701 performs re-learning using the successful data 402 equal to or greater than the predetermined number of samples as learning data. The robot motion learning device 100 acquires a learned model 706 that has been re-learned using only the successful data 402 obtained from the motion of the controlled robot 301. Then, the robot motion learning device 100 can improve the accuracy of control of the controlled robot 301 by using the learned model 706 acquired by re-learning.

If it is determined in the process of step S31 that the number of samples of successful data 402 is not sufficient (No), the CPU 701 determines whether the number of samples of successful data 402 that have been acquired is equal to or greater than half the number of samples of initially collected data 401 (S33). In this process, the CPU 701 determines whether the number of samples of successful data 402 stored in the recording device 400 is equal to or greater than half the number of samples of initially collected data 401 acquired in the learning process. Note that the amount of half is not absolute, and can be any number that the user desires as long as the total number of samples of all of the successful data 402 and part of the initially collected data 401 reaches a sufficient amount for learning.

If it is determined in the process of step S33 that the number of samples of the successful data 402 is equal to or greater than half the number of samples of the initially collected data 401 (Yes), the CPU 701 proceeds to step S34. On the other hand, if it is determined that the number of samples of the successful data 402 is less than half the number of samples of the initially collected data 401 (No), the CPU 701 proceeds to step S35.

In the process of step S34, the CPU 701 performs re-learning by machine learning using all of the successful data 402 stored in the recording device 400 and the same number of samples of the initial collected data 401 as the successful data 402 (S34). In this process, the CPU 701 performs machine learning using the successful data 402 stored in the recording device 400 and the same number of samples of the initial collected data 401 as the successful data 402, obtains a trained model 706, and ends the re-learning process.

By executing the process of step S34, the robot motion learning device 100 reduces the relative data difference in the data samples used in re-learning. As a result, the robot motion learning device 100 is expected to improve the success rate of the controlled task 302 when the controlled robot 301 is operated using various data output by performing inference processing from the re-learned learned model 706.

In the process of step S35, the CPU 701 performs re-learning by machine learning using all of the successful data 402 stored in the recording device 400 and all of the initially collected data 401 stored in the recording device 400 (S35). In this process, the CPU 701 performs machine learning using all of the successful data 402 and the initially collected data 401 stored in the recording device 400, obtains a trained model 706, and ends the re-learning process.

By executing the process of step S35, the number of data samples used in re-learning in the robot motion learning device 100 becomes greater than when learning is performed using only the initially collected data 401. As a result, the robot motion learning device 100 can be expected to improve the generalization performance of the re-learned trained model 706.

The processing of steps S31 to S35 constitutes a learning process in which the trained model 706 is trained by machine learning using the successful data 402.

<Summary of this embodiment>
As described above, the robot motion learning device 100 of the present embodiment operates the control target robot 301 and the corresponding imitation target actor 201 in the imitation target area 200 different from the control target area 300, and acquires the initial collected data 401. The robot motion learning device 100 performs machine learning using the acquired initial collected data 401 as teacher data, acquires a learned model 706, and acquires motion data for operating the control target robot 301 in the environment of the control target area 300 using the learned model 706. The robot motion learning device 100 acquires success data 402 for motions determined to be successful as a result of operating the control target robot 301 in the environment of the control target area 300 using the acquired motion data. Then, the robot motion learning device 100 re-learns the learned model 706 by machine learning using the acquired success data 402.

With this configuration, the robot motion learning device 100 can acquire a learned model 706 that can achieve the controlled task 302 with a sufficiently high success rate. The robot motion learning device 100 re-learns the learned model 706 using the initial collected data 401 obtained from the environment of the imitation target area 200, using the successful data 402. This allows the robot motion learning device 100 to improve the accuracy of control of the controlled robot 301 using the learned model 706.

<Modification>
In this embodiment, when saving the successful data 402 determined to be successful as a result of the determination process, the CPU 701 is configured not to perform any particular processing on the initial collection data 401, but is not limited to this. When saving the successful data 402 in the recording device 400, the CPU 701 may be configured to discard the number of samples of the initial collection data 401 corresponding to the number of samples of the successful data 402.

By configuring in this manner, the robot motion learning device 100 can prevent an increase in the amount of sample data stored in the recording device 400 by discarding at least a portion of the initially collected data 401 based on the addition of acquired successful data 402.

In addition, the CPU 701 of this embodiment is configured to re-learn using all of the successful data 402 and all of the initially collected data 401 when the number of samples of the successful data 402 is less than half the number of samples of the initially collected data 401, but is not limited to this. For example, the CPU 701 may be configured to re-learn using all of the successful data 402 and at least a portion of the initially collected data 401. Specifically, when 100 samples of the initially collected data 401 and 20 samples of the successful data 402 are stored in the recording device 400, re-learning may be performed using 20 samples of the successful data 402 and 80 samples of the initially collected data 401.

In addition, in this embodiment, the robot motion learning device 100 has a display 707 as a display unit, and a keyboard 708 and a mouse 709 as input units, each of which is configured separately, but is not limited to this. The robot motion learning device 100 may have a touch panel in which the display unit and the input unit are integrated. When configured in this way, the robot motion learning device 100 can perform various settings and processes corresponding to various buttons that are arranged on the GUI 600 displayed on the touch panel when the user presses the buttons.

The information processing method and information processing device of the present invention can be applied to software design and program creation for various machines and equipment, such as industrial robots, service robots, and processing machines that operate under computer numerical control, in addition to production equipment. For example, machines and equipment that can automatically perform movements such as stretching, bending, stretching, moving up and down, moving left and right, or rotating, or a combination of these movements, based on information from a recording device provided in the control device.

Furthermore, the present invention also includes a control method for a robot system that executes the above-mentioned information processing method and includes a robot manipulator as a controlled object.Furthermore, the present invention also includes a manufacturing method for an article using a robot system that operates by executing the above-mentioned information processing method.Furthermore, the present invention also includes a program that can execute the above-mentioned information processing and a computer-readable recording medium that stores the program.

Also, the present invention can be realized by supplying a program that realizes one or more of the functions of the above-mentioned embodiments to a system or device via a network or a recording medium (storage medium), and having one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.

The present invention is not limited to the embodiments described above, and many modifications are possible within the technical concept of the present invention. For example, the above-described different embodiments and/or modifications may be combined. The effects described in the embodiments are merely a list of the most favorable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments and/or modifications.

The disclosure of this embodiment includes the following configuration:

(Method 1)
A process of acquiring first data by operating an object corresponding to a control object in an environment different from an operating environment of the control object, and acquiring a trained model by machine learning using the first data;
A process of acquiring second data for operating the control object in the operating environment using the trained model;
a determination process of operating the controlled object in the operating environment using the second data and determining a result;
acquiring third data about the operation determined to be successful in the determination process, and performing a learning process of learning the trained model by machine learning using the third data;
23. An information processing method comprising:

(Method 2)
The learning process includes learning the learned model by machine learning using the third data and at least a portion of the first data.
2. The information processing method according to claim 1,

(Method 3)
the learning process discarding at least a portion of the first data based on the addition of the acquired third data;
3. The information processing method according to claim 1 or 2.

(Method 4)
When a number of samples of the third data is equal to or greater than a predetermined number of samples, the learning process uses the third data equal to or greater than the predetermined number of samples as learning data.
4. The information processing method according to any one of Methods 1 to 3.

(Method 5)
The determination process is performed by machine learning.
5. The information processing method according to any one of Methods 1 to 4.

(Method 6)
The determination process determines whether the process is successful or not by using a threshold value based on the operating environment.
6. The information processing method according to any one of methods 1 to 5.

(Method 7)
Execute a process of displaying the judgment conditions and the judgment results of the judgment process on a display unit.
7. The information processing method according to any one of methods 1 to 6.

(Method 8)
execute a process of discarding the second data, the second data being determined to have been operated unsuccessfully in the determination process;
8. The information processing method according to any one of methods 1 to 7.

(Method 9)
The object has a component corresponding to a component of the control target object.
9. The information processing method according to any one of methods 1 to 8.

(Configuration 10)
An information processing device including an information processing unit,
The information processing unit,
A process of acquiring first data by operating an object corresponding to a control object in an environment different from an operating environment of the control object, and acquiring a trained model by machine learning using the first data;
A process of acquiring second data for operating the control object in the operating environment using the trained model;
a determination process of operating the controlled object in the operating environment using the second data and determining a result;
acquiring third data about the operation determined to be successful in the determination process, and performing a learning process of learning the trained model by machine learning using the third data;
23. An information processing apparatus comprising:

(Configuration 11)
The information processing device according to configuration 10,
The control object includes a robot manipulator.
A robot system comprising:

(Configuration 12)
The robot manipulator has at least one or more actuators,
The information processing device acquires, as the second data, a command value for driving the driving unit.
12. The robot system according to claim 11 .

(Configuration 13)
The robot manipulator has at least one sensor;
The information processing device performs a determination of the determination process using a value detected by the sensor.
13. The robot system according to configuration 11 or 12.

(Method 14)
An information processing unit executes an information processing method according to any one of methods 1 to 9,
The control object includes a robot manipulator.
A method for controlling a robot system comprising:

(Method 15)
14. Manufacturing an article using the robot system according to any one of configurations 11 to 13.
A method for producing an article.

(Configuration 16)
A program for causing a computer to execute the information processing method according to any one of Methods 1 to 9.

(Configuration 17)
A computer-readable recording medium storing the program according to configuration 16.

201...Object (subject to be imitated): 301...Control object, robot manipulator (robot to be controlled): 301a...Component (robot hand): 303...Sensor (control operation sensor): 401...First data (initial collected data): 402...Third data: 700...Information processing device: 701...Information processing unit (CPU): 706...Trained model: 707...Display unit (display)

Claims (17)

A process of acquiring first data by operating an object corresponding to a control object in an environment different from an operating environment of the control object, and acquiring a trained model by machine learning using the first data;
A process of acquiring second data for operating the control object in the operating environment using the trained model;
a determination process of operating the controlled object in the operating environment using the second data and determining a result;
acquiring third data about the operation determined to be successful in the determination process, and performing a learning process of learning the trained model by machine learning using the third data;
23. An information processing method comprising: The learning process includes learning the learned model by machine learning using the third data and at least a portion of the first data.
2. The information processing method according to claim 1, the learning process discarding at least a portion of the first data based on the addition of the acquired third data;
2. The information processing method according to claim 1, When a number of samples of the third data is equal to or greater than a predetermined number of samples, the learning process uses the third data equal to or greater than the predetermined number of samples as learning data.
2. The information processing method according to claim 1, The determination process is performed by machine learning.
2. The information processing method according to claim 1, The determination process determines whether the process is successful or not by using a threshold value based on the operating environment.
2. The information processing method according to claim 1, Execute a process of displaying the judgment conditions and the judgment results of the judgment process on a display unit.
2. The information processing method according to claim 1, execute a process of discarding the second data, the second data being determined to have been operated unsuccessfully in the determination process;
2. The information processing method according to claim 1, The object has a component corresponding to a component of the control target object.
2. The information processing method according to claim 1, An information processing device including an information processing unit,
The information processing unit,
A process of acquiring first data by operating an object corresponding to a control object in an environment different from an operating environment of the control object, and acquiring a trained model by machine learning using the first data;
A process of acquiring second data for operating the control object in the operating environment using the trained model;
a determination process of operating the controlled object in the operating environment using the second data and determining a result;
acquiring third data about the operation determined to be successful in the determination process, and performing a learning process of learning the trained model by machine learning using the third data;
23. An information processing apparatus comprising: The information processing device according to claim 10,
The control object includes a robot manipulator.
A robot system comprising: The robot manipulator has at least one or more actuators,
The information processing device acquires, as the second data, a command value for driving the driving unit.
The robot system according to claim 11 . The robot manipulator has at least one sensor;
The information processing device performs a determination of the determination process using a value detected by the sensor.
The robot system according to claim 11 . An information processing unit executes the information processing method according to any one of claims 1 to 9,
The control object includes a robot manipulator.
A method for controlling a robot system comprising: Manufacturing an article using the robot system according to claim 11.
A method for producing an article. A program for causing a computer to execute the information processing method according to any one of claims 1 to 9. A computer-readable recording medium storing the program according to claim 16.

Images