JP2023183271A - Processing device, robot control system, and machine learning method - Google Patents

Abstract

To provide a processing device, a robot control system, and a machine learning method, which can reduce temporal variations in teaching data that impede motion learning.SOLUTION: A processor (CPU (23)) of a processing device (computer 20) segments teaching data for a robot (robot device 1) into the same kinds of motion of the robot (motion segmenting unit 41). The processor performs correction for equalizing the speed or timing of each of the same kind of motion of the robot to the plurality of segmented teaching data (speed adjusting unit 42, timing adjusting unit 43). The processor combines the plurality of corrected teaching data (data processing unit 45). The processor performs machine learning by using the combined teaching data (machine learning device 34).SELECTED DRAWING: Figure 4

Description

The present invention relates to a processing device, a robot control system, and a machine learning method.

In order to respond to the rapid changes in the business environment and the diversification of people's needs, high-mix, low-volume production systems are attracting attention. Automation using robots can be considered as a means of increasing production efficiency, but controlling robots requires enormous programming costs and high specialized knowledge, and the number of man-hours required to introduce them is a problem. Therefore, machine learning is being used for all robot functions in order to reduce the man-hours required to introduce robots. For example, in object recognition, many studies have been reported that use machine learning to quantitatively calculate the characteristics of an object from contact information with a robot and image information in order to estimate the type, position, and orientation of the object.

On the other hand, in motion generation, many studies have been reported that apply machine learning to autonomous control of robots in order to change the environment and realize motions that are difficult to describe with programs. Reinforcement learning is an example of autonomous control technology. Because the robot acquires the optimal motion to accomplish a task through trial and error, there is no need to explicitly teach the motion. However, it takes a huge amount of trial and error (learning time) to obtain the optimal motion.

Imitation learning is an autonomous control technology that requires little learning time. Imitation learning consists of three phases: teaching, learning, and execution. In the teaching phase, the teacher teaches the desired motion by operating the robot, and at this time collects and records time-series information output by the encoder, temperature sensor, vision sensor, ultrasonic sensor, etc. installed on the robot. . Instructions for teaching robots include remote control using a controller, direct teaching by touching the robot, and methods using control programs.

The time series information obtained in the teaching phase is called teaching data. The type of teaching data depends on the motion generation method. For example, when generating a motion according to the position of a robot or an object, the teaching data includes the position of the robot, joint angle information, and the like. Furthermore, when generating actions in response to visual information, images of the robot and the work environment may be included. Further, when generating a motion according to the state of the target object, the position and posture information of the target object may be included. Further, for example, when a motion is generated according to the speed of the robot, the teaching data includes the speed of the robot, joint angular velocity information, and the like. Further, for example, when generating a motion according to the force of the robot, the teaching data includes tactile information of the robot.

In the learning phase, learning data is created based on the teaching data collected in the teaching phase, and the learning data is used to train the motion generation model. The teaching data may be used as is as the learning data. Further, in order to remove noise, defects, error values, etc. contained in the teaching data, learning data may be created by converting the teaching data. Converting teaching data to learning data is called preprocessing. Preprocessing includes data cleaning, which removes data that includes missing values and outlier values, and normalization, which converts data values so that they fall within a specified range. By updating the weights of the machine learning model using learning data, when a certain value is input to the model, a desired value is output. For example, when joint angle information and images at a certain time are input, the model is trained to predict joint angle information and images at the next time.

In the execution phase, the task is executed by generating motions using the motion generation model obtained in the learning phase and the robot's sensor data. For example, assume that in the learning phase, a machine learning model is trained to predict joint angle information and an image at the next time when joint angle information and an image at a certain time are input. By using this machine learning model in the execution phase, it is possible to autonomously generate movements by inputting joint angle information and images at each time and using the output joint angle information as control command values for the robot. I can do it.

In order to improve the motion generation accuracy of imitation learning, the following imitation learning is described in Patent Document 1. This Patent Document 1 states that "imitation learning is performed based on observation information grasped by the operator during the model operation."

Japanese Patent Application Publication No. 2021-10984

By the way, an issue with imitation learning is that it is difficult to simultaneously reduce the introduction man-hours (teaching/learning phase man-hours) and acquire generalized behavior. For example, in program-based teaching, since the robot is operated by computer control, it is possible to obtain more uniform teaching data compared to when the robot is operated by a human, making it easier to learn the relationships between teaching data and improving generalization performance. do. However, since it is necessary to program the robot's movements one by one, the number of man-hours required for introduction is large.

Teaching methods that require less man-hours to introduce than program-based methods include a method in which a person remotely controls a robot using a controller, a method in which a robot is operated by direct touch, and the like. These teaching methods require less man-hours to introduce because there is no need to program the robot's movements one by one.

However, when a robot is moved by a human, it is difficult to learn the relationships between the teaching data because there are variations in time such as movement speed and timing for each teaching data compared to program-based teaching methods. It is. In order to ignore the influence of this variation in the time direction, a huge amount of learning data is required and the number of teachings increases.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a processing device, a robot control system, and a machine learning method that can reduce temporal variations in teaching data that impede motion learning. Note that although the techniques described in Patent Document 1 have in common that learning data is manipulated to improve motion generation accuracy, they do not mention reducing variations in the time direction.

In order to achieve the above object, a processing device according to an example of the present invention segments teaching data to a robot for each motion of the same type of the robot, and divides the teaching data for the robot into segments of the same type of motion of the robot. The present invention includes a processor that performs correction to align the speed or timing of the operations of the , synthesizes the plurality of corrected teaching data, and performs machine learning using the synthesized teaching data.

According to the present invention, it is possible to reduce temporal variations in teaching data that impede motion learning. Problems, configurations, and effects other than those described above will be made clear by the following description of the embodiments.

1 is a schematic diagram showing a configuration example of a robot control system to which the present invention is applied. FIG. 2 is a block diagram showing an example of the hardware configuration of a computer that implements a motion planning section of the robot control system. FIG. 2 is a block diagram showing an example of a functional configuration of a computer included in a motion planning section of the robot control system. FIG. 2 is a block diagram illustrating an example of a functional configuration of a consistency verification device included in a motion planning section in the first embodiment of the present invention. FIG. 3 is an explanatory diagram of an example of a method for realizing speed adjustment in the first embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of a functional configuration of a machine learning device included in a motion planning section in the first embodiment of the present invention. 7 is a flowchart illustrating an example of the operation of the consistency verification device included in the operation planning section in the first embodiment of the present invention. FIG. 3 is an explanatory diagram of a reaching task in order to show an example of the effect of the consistency verification device included in the motion planning unit in the first embodiment of the present invention. FIG. 3 is a diagram showing the relationship between the time during a teaching operation and the position of the hand in the Y-axis direction. FIG. 7 is a diagram illustrating a reaching task when there is variation in teaching data in the time direction. FIG. 7 is a diagram showing the relationship between time and hand position in the Y-axis direction during a teaching operation with variations in the time direction. FIG. 8 is an explanatory diagram when the consistency verification device is applied to the reaching tasks shown in FIGS. 8C and 8D. FIG. 7 is a block diagram illustrating an example of the internal configuration of a consistency verification device included in a motion planning section in a second embodiment of the present invention. 12 is a flowchart illustrating an example of the operation of the consistency verification device included in the operation planning section in the second embodiment of the present invention. It is a schematic diagram showing an example of composition of a robot control system in a 3rd embodiment of the present invention. FIG. 3 is a block diagram showing an example of a functional configuration of a computer included in a motion planning section of a robot control system according to a third embodiment of the present invention. FIG. 7 is a block diagram showing an example of the hardware configuration of a computer that implements a motion planning section of a robot control system in a third embodiment of the present invention. FIG. 3 is a block diagram showing an example of the functional configuration of a screening device included in a motion planning section of a robot control system according to a third embodiment of the present invention. FIG. 7 is a block diagram showing an example of the functional configuration of a screen operation unit included in the robot control system according to a third embodiment of the present invention. FIG. 12 is a block diagram showing an example of the functional configuration of a motion planning section included in the robot control system according to a fourth embodiment of the present invention. FIG. 7 is a block diagram showing an example of a functional configuration of a motion parameter adjustment device included in a motion planning section of a robot control system according to a fourth embodiment of the present invention.

The first to fourth embodiments will be described below with reference to the drawings. The present embodiment relates to a robot motion generation technology that learns time-series data of a robot and autonomously generates motions. The present embodiment aims to achieve both reduction in introduction man-hours and acquisition of generalized motion by reducing variations in the time direction of teaching data that impede motion learning.

(First embodiment)
First, a robot control system according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 3.

FIG. 1 shows a configuration example of a robot control system to which the present invention is applied. The robot control system 100 shown in FIG. 1 includes a robot device 1, a sensor data acquisition section 2, a motion planning section 3, and a control section 4. Although a single-arm robot arm is shown as a specific example of the robot device 1, the configuration of the robot device 1 is not limited and may be, for example, a double-arm arm. Alternatively, it may be a moving device such as legs, crawlers, wheels, or propellers.

The sensor data acquisition unit 2 acquires sensor data output from the robot device 1. The sensor data includes sensor data of the robot device 1 and the environment acquired during work.

The motion planning section 3 uses the data obtained by the sensor data acquisition section 2 as teaching data to learn and generate a motion in the following three steps. First, learning data is created by performing preprocessing to suppress variations in teaching data. Next, a motion generation model is constructed by performing motion learning based on the training data. Finally, a motion command value is generated using the constructed motion generation model.

The control unit 4 provides the command value calculated by the motion planning unit 3 to the robot device 1, thereby driving the actuator mounted on the robot device 1.

[Hardware configuration of motion planning section]
Next, the hardware configuration of the motion planning unit 3 included in the robot control system 100 will be described with reference to FIG. 2. Here, the hardware configuration of the computer that implements the motion planning section 3 will be explained.

FIG. 2 is a block diagram showing an example of the hardware configuration of a computer that implements the motion planning section 3. As shown in FIG. The illustrated computer 20 is an example of hardware that constitutes a computer used by the sensor data acquisition section 2, motion planning section 3, and control section 4. For example, a personal computer can be used as the calculator 20.

The computer 20 includes a ROM (Read Only Memory) 22, a CPU (Central Processing Unit) 23, a RAM (Random Access Memory) 24, a nonvolatile storage 25, and an input/output interface 26, which are connected to a bus 21. A network interface 27 is provided.

The ROM 22 records software program codes that implement the functions of the sensor data acquisition section 2, motion planning section 3, and control section 4 according to the present embodiment.

The CPU 23 serves as an arithmetic processing unit that reads out software program codes that implement the functions of the sensor data acquisition section 2, motion planning section 3, and control section 4 according to the present embodiment from the ROM 22, loads the corresponding programs into the RAM 24, and executes them. Function. In the RAM 24, values of variables, parameters, etc. generated during arithmetic processing by the CPU 23 are temporarily written. The values of variables, parameters, etc. written in the RAM 24 are read out by the CPU 23 as appropriate. Although a CPU is used as the arithmetic processing unit, other processors such as an MPU (Micro Processing Unit) may also be used.

The nonvolatile storage 25 is an example of a recording medium, and can store data used by a program, data obtained by executing the program, and the like. For example, the nonvolatile storage 25 stores learning data, motion generation models, etc., which will be described later. Further, an OS (Operating System) and programs executed by the CPU 23 may be recorded in the nonvolatile storage 25. As the nonvolatile storage 25, a magnetic recording medium, an optical recording medium, a semiconductor recording medium, etc. can be adopted.

The input/output interface 26 is an interface for communicating signals and data from each sensor and each actuator included in the robot control system 100. The input/output interface 26 may have the function of an A/D (Analog/Digital) converter and/or a D/A converter (not shown) that processes input signals or output signals. The sensor data herein includes information obtained not only from each sensor but also from each actuator.

As the network interface 27, for example, a NIC (Network Interface Card) or a modem is used. The network interface 27 is configured to be capable of transmitting and receiving various data to and from an external device via a communication network such as a LAN or the Internet, or a dedicated line to which a terminal is connected. For example, by using the network interface 27, it is possible to send and receive various data to and from the robot device 1.

[Functional configuration of motion planning section]
Next, the functional configuration of the motion planning section 3 included in the robot control system 100 will be described with reference to FIG. 3. FIG. 3 shows an example of the functional configuration of the motion planning section 3 that implements the present invention. In the motion planning unit 3 shown in FIG. 3, the sensor data storage device 31 is a device that records the sensor data acquired by the sensor data acquisition unit 2 as time-series data. The consistency verification device 32 is a device that converts the time series data recorded in the sensor data storage device 31 and outputs learning data. The learning data storage device 33 is a device that records the learning data output from the consistency verification device 32. The machine learning device 34 is a device that learns learning data recorded in the learning data storage device 33 and outputs a command value to the control unit 4.

Next, the configuration of the consistency verification device 32 of the motion planning section 3 will be explained using FIG. 4. In the consistency verification device 32, the motion segmentation unit 41 is a device that divides sensor data into groups of motions. Methods for realizing motion segmentation include segmentation based on speed changes, segmentation based on clustering, and segmentation based on deep learning.

Segmentation based on velocity changes is a method based on the robot's operating speed and can be used when the data characteristics are simple. Segmentation by clustering can be used even when the taught data characteristics are complex, but it is necessary to set the number of segments in advance. Segmentation using deep learning can be used even when the taught data characteristics are complex, and there is no need to set the number of segments in advance. On the other hand, the implementation cost and calculation cost are higher than segmentation based on speed changes or segmentation based on clustering.

The speed adjustment unit 42 adjusts the movement speed in order to reduce variations in movement speed between teaching data that impede acquisition of generalized movement. As an example of a method for implementing the speed adjustment section, an algorithm will be described in which the speeds of the teaching data are made equal by quantizing the teaching data and removing noise.

FIG. 5 is a diagram showing the procedure and effects of this algorithm. Graph g5A shows data of different moving speeds, with the horizontal axis representing time and the vertical axis representing the hand position in a certain axis (hereinafter referred to as the x-axis) in the world coordinate system. Graph g5B represents the result of quantizing the two teaching data shown in graph g5A to convert the hand position value into a discrete approximate value. As shown in graph g5B, by quantizing, the amount of change in the teaching data per time becomes constant.

Note that when the amount of change per time in the teaching data is larger than the quantization width, the amount of change after quantization does not necessarily become a constant value. Therefore, the size of the quantization width needs to be equal to or larger than the amount of change in teaching data per time. In other words, after the speed adjustment, the speed is the same as the original teaching data or becomes larger than the original teaching data.

However, by performing upsampling as described in Modification 2, it is possible to adjust the speed to a smaller speed than the original teaching data. The enlarged display 5C shows that by performing quantization in one direction of the hand, a time when the hand position does not change, that is, a time when the speed is zero, occurs. Graph g5D represents the result of noise removal for the quantized teaching data shown in graph g5B. Noise removal refers to removing the time when the velocity occurs after quantization, regarding it as noise. As shown by the graph g5D, it can be seen that the hand speeds of the two teaching data are the same. It can also be seen that the final hand position remains the same as before the speed adjustment. From the above, it can be seen that speed adjustment can be achieved by performing quantization and noise removal.

The timing adjustment unit 43 shown in FIG. 4 adjusts the motion start and end timings in order to reduce variations in the timings of motion start and end, etc. between teaching data, which hinders the acquisition of generalized motions. An example of a method for realizing timing adjustment is to increase or decrease the stationary time of the robot device 1. For example, in order to align the operation start times of the robot device 1 in all the teaching data, calculate the average value of the operation start times of all the teaching data, and set the operation start time of each teaching data to be equal to the average value. It is conceivable to increase or decrease the stationary time.

The dynamics adjustment unit 44 adjusts the dynamics in the sensor data in order to prevent the generated motion from becoming unstable due to excessive magnitudes and changes in dynamics such as acceleration, jerk, and torque in the teaching data after speed adjustment. Adjust dynamics such as acceleration, jerk, and torque. One example of a method for realizing dynamics adjustment is to apply a moving average filter to the hand position data.

The data processing unit 45 converts the data output from the dynamics adjustment unit 44 into learning data so that the data can be learned by the deep learning model. Specifically, learning data is constructed by synthesizing each adjusted segment for each teaching data.

Next, the configuration of the machine learning device 34 will be explained using FIG. 6. In the machine learning device 34, the machine learning model definition unit 61 defines the structure and parameters of the machine learning model.

The learning unit 62 uses the learning data accumulated in the learning data storage device 33 to update the weights of the machine learning model defined in the machine learning model definition unit 61, so that when a certain value is input to the model, a desired value is obtained. Make the value output. For example, if the robot's joint angles at a certain time are input, predicted values of the robot's joint angles at the next time are output.

The learned weight storage unit 63 stores the weight parameters of the learned model.

The inference unit 64 constructs a motion generation model by using the machine learning model read from the machine learning model definition unit 61 and the weight parameters of the model read from the learned weight storage unit 63, and uses the same sensor data as during learning. The motion is generated by inputting it into the model.

For example, suppose that during learning, a model is constructed such that when a robot's joint angles at a certain time are input, a predicted value of the robot's joint angles at the next time is output. In this case, when generating motion, by inputting the joint angles at each time into the model, the joint angles at the next time can be obtained, and the robot's motion can be generated by using the obtained joint angles as command values. .

[Example of operation of motion planning unit 3]
Next, the order in which each function of the motion planning section 3 described above operates will be explained. The operation order can be broadly divided into three phases: teaching, learning, and execution. In the teaching phase, movements are taught by operating the robot device 1, and sensor data obtained during the teaching phase is recorded in the sensor data storage device 31 using the sensor data acquisition unit 2.

In the learning phase, the consistency verification device 32 is used to convert the sensor data of the sensor data storage device 31 into learning data, and the machine learning model defined by the machine learning model definition unit 61 is used for learning. Get the weight. The consistency verification device 32 converts sensor data through three steps: motion segmentation, speed adjustment, and timing adjustment. These three steps will be explained with reference to FIG.

FIG. 7 is a flowchart showing an example of the operation of the consistency verification device 32. After the start of preprocessing (S1), first, motion segmentation is performed for all teaching data (S2). In motion segmentation, the motion segmentation unit 41 is used to divide sensor data in the time direction, thereby dividing the sensor data into groups of motions. After motion segmentation, speed adjustment (S5), timing adjustment (S6), and dynamics adjustment (S7) are performed for each segment (S3, S4). The data processed by the timing adjustment unit 43 is converted into learning data using the data processing unit 45 (S8), and the preprocessing is ended (S9).

As described above, by processing the teaching data with the consistency verification device 32, learning data with reduced temporal variation between the teaching data can be obtained. Note that it is not always necessary to perform both speed adjustment and timing adjustment, and it may be necessary to perform only one of them. For example, if only the speed of the teaching operation is different, only the speed needs to be adjusted, so timing adjustment is not necessary. Further, if only the timing of the teaching operation is different, only the timing needs to be adjusted, so speed adjustment is not necessary.

The learning data constructed by the consistency verification device 32 is input to the learning section 62 of the machine learning device 34. The learning unit 62 calls the machine learning model from the machine learning model definition unit 61 and performs learning based on the learning data. The motion generation model obtained as a result of learning is stored in the learned weight storage section 63.

In the execution phase, the task is executed by generating motion using the motion generation model obtained in the learning phase. The inference unit 64 of the machine learning device 34 is used to calculate the command value to the robot device 1.

[Example of effect of motion planning section]
As a verification of the effectiveness of the motion planning unit 3, how the variation in the time direction between teaching data is reduced by operating the motion planning unit 3 is shown in FIGS. 8 (8A to 8D) and FIG. 9. and explain. FIG. 8 (8A to 8D) shows a reaching task in which the hand of the robot arm is moved linearly to a target hand position. The influence of variations in the time direction in the teaching data on the reaching task will be explained using FIGS. 8 (8A to 8D).

First, using FIGS. 8A and 8B (graph g8B), the reason why a task is successful when there is no temporal variation in the teaching data will be explained. The hand depicted in FIG. 8A shows the position of the robot hand at the initial time. Moreover, p, q, and r drawn in FIG. 8A each represent the target hand position. X and Y in FIG. 8A are the names of coordinate axes set in the task environment, and here the X and Y axes indicate position [cm]. Here, let us assume that motions for target hand positions p and r are taught. In FIG. 8A, arrows (solid lines) from the hand toward p and r represent the trajectory of the hand during the teaching motion.

FIG. 8B (graph g8B) is a diagram showing the relationship between the time during the teaching operation and the hand position in the Y-axis direction. As shown in graph g8B, the teaching motions for the target hand positions p and r have the same hand speed and reaching start/end time, and there is no variation in the time direction. In the motion learning phase, by learning the teaching motion for the target hand positions p and r, a motion generation model is obtained that outputs a command value to the robot arm at the next time when current sensor data is input.

Next, with reference to FIG. 8A and graph g8B, consider the motion generated for the target hand position q in the motion execution phase. In the motion execution phase, the target hand position q is generalized to the motion generation model constructed in the motion learning phase by learning between two points, so that it is located exactly in the middle of the target hand positions p and r. . At this time, the hand position generated for q passes exactly between the hand positions of the teaching motion for p and r at each time. In FIG. 8A and graph g8B, the motion generated with respect to q is shown by a broken line, and the hand is stable without wobbling in the Y-axis direction. From this, it can be seen that the reaching task is successful when there is no variation in the teaching data in the time direction.

Next, using FIG. 8C and FIG. 8D (graph g8D), the reason why the task fails when there is variation will be explained. FIG. 8C shows the state of the reaching task when there is variation in the teaching data in the time direction. Each symbol in the figure represents the same meaning as in FIG. 8A.

FIG. 8D (graph g8D) is a diagram showing the relationship between time and hand position in the Y-axis direction during the teaching operation. There are variations in the time direction in the teaching operations for the target hand positions p and r. Specifically, the movement speed of the teaching operation of the target hand position r is faster than that of the target hand position p. Furthermore, the reaching start time (double-dashed line) and the reaching completion time (dotted-dotted line) are also early.

Next, consider the motion generated for the target hand position q with reference to FIG. 8C and graph g8D. As described above, due to the positional relationship between p, q, and r, the motion generated for q passes exactly in the middle of the taught motion for p and r. In FIG. 8C and graph g8D, the action generated for q is shown by a dashed line. Referring to the motion generated for q, the hand is largely shaken in the Y-axis direction. From this, it can be seen that the reaching task will fail if there is variation in the teaching data in the time direction.

Next, using FIG. 9, it will be explained that a generalized operation can be obtained by using the consistency verification device 32 even when there is variation in the teaching data in the time direction. FIG. 9 is an explanatory diagram when the consistency verification device 32 is applied to the reaching tasks shown in FIGS. 8C and 8D. Since the same reference numerals in FIG. 9 are the same parts as those in FIGS. 8C and 8D, a repeated explanation will be omitted.

Graph g9A is teaching data for target hand positions p and r shown in graph g8D. As described above, there are variations in the teaching data for the target hand positions p and r in the time direction.

Graph g9B is a graph obtained by segmenting the teaching motions for target hand positions p and r shown in graph g9A. The dashed dotted line in graph g9B represents the segmented time. As shown in graph g9B, the operation is divided into reaching start time and reaching end time. For example, as shown in graph g9B, in order to detect the reaching start time, the motion segmentation unit 41 performs segmentation based on speed changes to find the time when the hand starts moving from a stationary state. Good.

Graph g9C is an explanatory diagram for aligning the hand speeds for the hand motion shown in graph g9B. As shown in graph g9C, the hand speed of the arm from the reaching start time to the reaching completion time may be adjusted to be equal to the average value of the hand speeds in the two teaching data + α (constant). More preferably, the hand speed of the arm from the reaching start time to the reaching completion time is made equal to the average value.

Graph g9D is an explanatory diagram for arranging the timing of hand motions for the hand motions shown in graph g9C. As shown in graph g9D, the reaching start time and the reaching completion time are made equal between the teaching data.

Graph g9E shows the motion generated for target hand position q in FIG. 8A during the execution phase. Since the target hand position q is located exactly between the target hand positions p and r, the motion generated for q also passes exactly between the taught motions for p and r. Referring to the motion generated for q, the hand does not shake in the Y-axis direction and is stable.

<Modification 1>
By the way, in this embodiment, the motion segmentation unit 41 divides sensor data into groups of motions. However, if the data characteristics are complex, segmentation may not work well. One possible solution to this problem is to use Recurrent Dropout. Recurrent Dropout is the process of learning while randomly inactivating nodes in a recurrent neural network. By using Recurrent Dropout, it is expected that generalization performance against temporal variations in training data will be improved.

<Modification 2>
By the way, in this embodiment, the speed adjustment section 42 quantizes the sensor data. However, if the amount of change per unit time in sensor data is larger than the quantization width (in other words, if you want to increase the speed after conversion), the amount of change per unit time in sensor data after quantization exceeds the quantization width. As a result, the amount of change per time after quantization is not equal and failure to equalize the speed occurs.

In order to avoid this, the speed adjustment unit 42 needs to perform upsampling before quantization to reduce the amount of change in the sensor per unit time. Upsampling means increasing the sampling frequency of time series data. Methods for implementing upsampling include linear interpolation, nearest neighbor interpolation, spline interpolation, and the like. However, it is known that these interpolation methods cannot be applied to image data, and a separate technique called frame interpolation is required. Details will be explained in the second embodiment.

The main features of this embodiment can be summarized as follows.

The processor (CPU 23) of the processing device (computer 20) segments the teaching data to the robot (robot device 1) for each similar type of robot motion (motion segmentation unit 41). The processor (CPU 23) corrects the plurality of segmented teaching data to align the speed or timing of the same type of motion of the robot (speed adjustment section 42, timing adjustment section 43). The processor (CPU 23) synthesizes the plurality of corrected teaching data (data processing unit 45). The processor (CPU 23) performs machine learning using the combined teaching data (machine learning device 34).

This reduces variations in teaching data in the time direction. As a result, the introduction man-hours (teaching/learning phase man-hours) are reduced, and the relationships between teaching data can be learned more easily, improving generalization performance.

The processor (CPU 23) performs correction for smoothing the dynamics values of the same type of motion of the robot on the plurality of segmented teaching data (dynamics adjustment unit 44). For example, values of dynamics (acceleration, jerk, torque, etc.) are smoothed by taking a moving average. Thereby, the operation of the robot can be stabilized.

The teaching data includes, for example, position information indicating the position of the robot, joint angle information indicating the joint angle, speed information indicating the speed of the robot, joint angular velocity information indicating the joint angular velocity of the robot, and sensor values of a tactile sensor provided in the robot. The tactile information includes at least one of tactile information indicating the tactile information. In this embodiment, the teaching data includes position information indicating the position of the robot (the position of the gripper). Thereby, machine learning can be performed using the state of the robot as teaching data.

Further, the teaching data may include at least one of position information indicating the position of the object on which the robot works, and posture information indicating the posture of the object. Thereby, machine learning can be performed using the state of the object as teaching data.

The dynamics include, for example, at least one of acceleration, acceleration change, jerk, jerk change, torque, and torque change. This allows the robot to move smoothly.

The processor (CPU 23) quantizes the plurality of segmented teaching data and eliminates noise, thereby making the speeds of the robot's similar motions uniform (FIG. 5). Thereby, variations in the teaching data in the time direction can be quickly reduced.

The processor (CPU 23) corrects the plurality of segmented teaching data by increasing or decreasing the robot's rest time to align the timing of the robot's similar motions (FIG. 9). Thereby, variations in the teaching data in the time direction can be easily reduced.

The robot control system 100 includes a processing device (computer 20) and a robot (robot device 1). The processor (CPU 23) uses the learned machine learning model to generate a command value for the robot's operation (machine learning device 34). The robot operates according to command values. This allows the robot to be autonomously controlled.

(Second embodiment)
Next, as a second embodiment of the present invention, a case where sensor data includes an image will be described with reference to FIGS. 10 and 11.

[Functional configuration example]
FIG. 10 is a block diagram showing an example of the internal configuration of a consistency verification device included in the motion planning section in the second embodiment of the present invention. Note that in FIG. 10, the same reference numerals as in FIG. 4 indicate the same parts, so repeated explanation will be omitted. As shown in FIG. 10, the second embodiment divides sensor data by a motion segmentation unit 41, and uses a speed adjustment unit 42 and/or a timing adjustment unit 43 to reduce variations in the time direction. This is the same as the first embodiment in that the dynamics is adjusted by using the dynamics adjustment unit 44 and converted into learning data by the data processing unit 45. The difference from the first embodiment is that in the second embodiment, the consistency verification device 32 includes a frame interpolation unit 101. The frame interpolation unit 101 increases the frame rate of time-series data of images.

Examples of methods for implementing frame interpolation include frame interpolation based on optical flow and frame interpolation using deep learning. Frame interpolation based on optical flow consists of three steps: feature extraction, movement change calculation, and interpolated image generation. For example, consider two consecutive images out of time-series data of images. In feature extraction, feature quantities of two images are extracted. In the movement change calculation, attention is paid to the same feature quantity among the feature quantities calculated in the feature quantity extraction, and the movement change amount thereof is calculated. Note that this movement change of the feature amount is called optical flow. In interpolation image generation, an image between the two original images is estimated by moving pixels in the two original images based on the amount of change in movement.

Examples of frame interpolation using deep learning include models such as FLAVR and FILM. FLAVR is a deep learning model that enables highly accurate frame interpolation by performing optical flow and three-dimensional convolution calculations inside the model. Further, FILM is a deep learning model that takes into account the scale (enlargement or reduction) of an image. When time-series data of an image is input to FLAVR (or FILM), time-series data of an image after frame interpolation can be obtained.

[Operation example]
FIG. 11 is a flowchart showing an example of the operation of the consistency verification device 32 in the second embodiment of the present invention. As shown in FIG. 11, the second embodiment performs motion segmentation of sensor data (S2), performs speed adjustment (S6), timing adjustment (S7) for each segment, and performs teaching. This is the same as the first embodiment in that learning data is constructed by composing each adjusted segment for each data (S8, S9). Note that in FIG. 11, the same reference numerals as in FIG. 7 indicate the same parts, so repeated explanation will be omitted. The difference from the first embodiment is that in the second embodiment, frame interpolation (S13) is performed before speed adjustment (S6).

The main features of this embodiment can be summarized as follows.

The teaching data includes images of the robot (robot device 1) or the working environment.

The processor (CPU 23) performs frame interpolation of images included in the teaching data for the plurality of segmented teaching data, and then performs correction to equalize the speeds of the same type of motion of the robot (frame interpolation unit 101). Thereby, even if the teaching data includes images of the robot (robot device 1) or the working environment, variations in the teaching data in the time direction can be easily reduced.

(Third embodiment)
Next, as a third embodiment of the present invention, a case where only teaching data effective for improving generalization performance is extracted will be described with reference to FIGS. 12, 13, and 14.

FIG. 12 is a block diagram showing a configuration example of a robot control system 100 in the third embodiment of the present invention. Note that in FIG. 12, the same reference numerals as in FIG. 1 indicate the same parts, so repeated explanation will be omitted. The difference from the first embodiment is that in the third embodiment, the robot control system 100 includes a screen operation section 5. The screen operation section 5 can display the processing results of the motion planning section. Furthermore, various parameters in the motion planning section 3 are determined based on the operation input received from the user.

FIG. 13 is a block diagram showing a configuration example of the motion planning section 3 in the third embodiment of the present invention. Note that in FIG. 13, the same reference numerals as those in FIG. 3 indicate the same parts, so a repeated explanation will be omitted. The difference from the first embodiment is that in the third embodiment, the motion planning section 3 includes a screening device 131. The screening device 131 extracts only sensor data that is effective for improving generalization performance from among the sensor data stored in the sensor data storage device 31. Thereafter, the extracted teaching data is output to the consistency verification device 32 and the screen operation section 5.

[Hardware configuration example]
Next, the hardware configuration of the motion planning section 3 included in the robot control system 100 in the third embodiment of the present invention will be described with reference to FIG. 14. Note that in FIG. 14, the same reference numerals as those in FIG. 3 indicate the same parts, so a repeated explanation will be omitted. The difference from the first embodiment is that the third embodiment includes a video output interface 141.

As the video output interface 141, for example, VGA (Video Graphics Array), DVI (Digital Visual interface), HDMI (High-Definition Multimedia Interface, registered trademark), or Display Port is used. The video output interface 141 is configured to be able to transmit video to a display via a dedicated line or the like.

[Functional configuration example]
Next, an example of the functional configuration of the screening device 131 of the motion planning section 3 included in the robot control system 100 in the third embodiment of the present invention will be described with reference to FIG. 15.

FIG. 15 shows an example of the functional configuration of a screening device 131 that implements the present invention. Note that in FIG. 15, the same reference numerals as those in FIG. 13 indicate the same parts, so a repeated explanation will be omitted.

In the screening device 131 shown in FIG. 15, the grouping unit 151 groups the sensor data stored in the sensor data storage device 31 so that data with similar actions are grouped together. As a method of implementing the grouping section, for example, it is possible to divide sensor data of the same target hand position into groups.

The representative data calculation unit 152 calculates, for each group obtained by the grouping unit 151, time series data that represents the group (this is referred to as representative data). As a method for realizing the representative data calculation unit 152, for example, time series data obtained by calculating the median value at each time for all time series data in a group may be used as representative data. .

The outlier data detection unit 153 detects time series data (referred to as outlier data) that is not similar to the representative data calculated by the representative data calculation unit 152 for each group obtained by the grouping unit 151. As a method for implementing the outlier data detection section, it is possible to use, for example, DTW (Dynamic Time Warping) and IQR (Interquartile Range). DTW is an index of similarity between time series data. The possible values of DTW are 0 or more, and the closer it is to 0, the more similar the time series data are. First, the DTW series is obtained by calculating the DTW with the representative data for all the time series data in the group. Next, the IQR of the calculated DTW sequence is determined. IQR is an index that expresses the degree of dispersion of data, and is determined by (3rd quartile) - (1st quartile). Finally, among the time-series data, data whose DTW for the representative data is larger than (third quartile+α×IQR) is determined to be outlier data. The value of α is initially set to 1.5. Further, the value can also be changed by the user using the screen operation section 5.

The calculation result output unit 154 outputs sensor data other than the deviation data to the consistency verification device 32. Additionally, information on missing data is output to the screen operation unit 5.

Next, an example of the functional configuration of the screen operation unit 5 included in the robot control system 100 in the third embodiment of the present invention will be described with reference to FIG. 16.

FIG. 16 shows an example of the functional configuration of the screen operation section 5 that embodies the present invention. Note that in FIG. 16, the same reference numerals as in FIG. 1 indicate the same parts, so repeated explanation will be omitted.

In the screen operation unit 5 shown in FIG. 16, the operation input unit 161 is configured with an input device such as a mouse and a keyboard, and receives mouse input, keyboard input, etc. from the user.

The screen display section 162 is composed of, for example, a display, and visualizes information obtained from the operation input section 161 and the motion planning section 3.

The screen control unit 163 receives information from the motion planning unit 3 and the operation input unit 161. It also outputs information to the motion planning section 3 and screen display section 162.

The main features of this embodiment can be summarized as follows.

The processor (CPU 23) performs screening by removing data that deviates from the teaching data (screening device 131). Thereby, generalization performance can be further improved.

(Fourth embodiment)
Next, as a fourth embodiment of the present invention, a case will be described with reference to FIG. 17 in which an arbitrary motion speed and/or arbitrary force magnitude is realized when motion is generated.

FIG. 17 is a block diagram showing a configuration example of a motion planning section in the fourth embodiment of the present invention. Note that in FIG. 17, the same reference numerals as those in FIG. 3 indicate the same parts, so a repeated explanation will be omitted. The difference from the first embodiment is that in the fourth embodiment, the motion planning section 3 includes a motion parameter adjustment device 171. The motion parameter adjustment device 171 realizes an arbitrary motion speed and/or arbitrary force magnitude when generating a motion from the sensor data stored in the sensor data storage device 31.

[Hardware configuration of operating parameter adjustment device]
The hardware configuration of the motion planning unit 3 included in the robot control system 100 in the fourth embodiment of the present invention is the same as that in the third embodiment of the present invention, and therefore, a repeated explanation will be omitted.

[Example of functional configuration of operating parameter adjustment device]
Next, an example of the functional configuration of the motion parameter adjustment device 171 of the motion planning section 3 included in the robot control system 100 in the fourth embodiment of the present invention will be described with reference to FIG. 18.

FIG. 18 shows an example of the functional configuration of an operating parameter adjustment device 171 that embodies the present invention. Note that in FIG. 18, the same reference numerals as those in FIGS. 12 and 13 indicate the same parts, so a repeated explanation will be omitted.

In the operating parameter adjustment device 171 shown in FIG. 18, the operating parameter storage unit 181 stores values of parameters necessary for operating parameter adjustment. Types of parameters include, for example, the control cycle of the motion generation model, the range of current values, the range of torque sensor values, and the like. The value of the parameter can be changed by the user via the screen operation unit 5.

The motion parameter adjustment calculation section 182 adjusts the motion speed and/or the magnitude of force when generating motion, based on the parameter values stored in the motion parameter storage section 181. As a specific method of changing the motion speed in the motion parameter adjustment calculation section, for example, changing the control cycle when motion is generated can be considered. For example, if it is desired to increase the motion speed, the control cycle at the time of motion generation may be made smaller than the sampling cycle of the teaching data. As a specific method of changing the magnitude of force in the operation parameter adjustment calculation section, for example, changing the current value or torque sensor value may be considered. For example, if it is desired to reduce the magnitude of the force, the upper limit of the current value or torque sensor value may be made smaller than the teaching data.

The main features of this embodiment can be summarized as follows.

The processing device (computer 20) includes a storage device (operation parameter storage unit 181) that stores operation parameters indicating parameters for adjusting the operation of the robot (robot device 1) after machine learning. The storage device includes, for example, a RAM 24, a nonvolatile storage 25, and the like.

The processor (CPU 23) uses the learned machine learning model to generate command values for the operation of the robot (robot device 1) based on the operation parameters (machine learning device 34). This allows the robot's motion to be adjusted without having to perform teaching/learning again.

The operating parameter is, for example, the sampling period of teaching data. The processor (CPU 23) increases the speed of the robot's operation by making the control period of the robot (robot device 1) smaller than the sampling period in the command value, or increases the speed of the robot's operation by making the control period of the robot (robot device 1) larger than the sampling period in the command value. By doing so, the speed of the robot's motion is reduced (motion parameter adjustment device 171). Thereby, the speed of the robot's motion can be adjusted without having to perform teaching/learning again.

Further, the operation parameter is, for example, the maximum value M of the robot torque or a value correlated therewith in the teaching data. The processor (CPU 23) increases the force of the robot by making the upper limit of the torque of the robot (robot device 1) or a value correlated therewith (for example, the value of the driving current of an actuator) larger than the maximum value M in the command value. Alternatively, in the command value, the upper limit of the robot's torque or a value correlated therewith is made smaller than the maximum value M, thereby reducing the force of the robot. This allows the robot's force to be adjusted without having to perform teaching/learning again.

Note that the present invention is not limited to the embodiments described above, and includes various modifications. For example, the embodiments described above have been described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace some of the configurations of each embodiment with other configurations.

Further, each of the configurations, functions, processing units, etc. described above may be partially or entirely realized in hardware by designing, for example, an integrated circuit. As the hardware, a broadly defined processor device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) may be used.

Further, each component of the motion planning section 3 according to each embodiment described above may be implemented in the control section 4. Further, the processing performed by a certain processing section of the motion planning section 3 may be realized by one piece of hardware, or may be realized by distributed processing by a plurality of pieces of hardware.

In each of the embodiments described above, the sensor data acquisition section 2, the motion planning section 3, and the control section 4 are realized by one computer 20, as an example, but they may be realized by separate computers. By implementing the motion planning unit 3, which has a large calculation load, using a high-performance computer, the overall throughput can be improved. Note that the computers are connected to each other via a communication network such as a LAN or the Internet.

Note that the embodiment of the present invention may have the following aspects.

(1). This is a machine learning system that performs machine learning to generate robot motions, and includes an acquisition unit that acquires teaching data to the robot (time-series sensor information, etc.) and segmentation of the teaching data based on the robot's motion. A speed adjustment that corrects the teaching data so that the speeds of the same kind of motions are the same when the speeds of the same kind of motions among the segmented motions are different between the motion segmentation unit and different teaching data that have a common task. and a timing adjustment unit that corrects the teaching data so as to align the timings of the same type of movement when the timings of the same type of movement among the segmented movements are different between different teaching data having a common task; a dynamics adjustment unit that corrects the dynamics of the robot in the teaching data, and the teaching data in which at least one of the speed, the timing, and the dynamics has been corrected is used as learning data in the machine learning system. system.

(2). In (1), the machine learning system includes an acquisition unit that acquires the position of the robot, joint angle information, etc. as teaching data to the robot.

(3). In (2), the machine learning system is a machine learning system that includes an acquisition unit that acquires images of the robot and the work environment in addition to the robot's position and joint angle information.

(4). In (2), the machine learning system includes an acquisition unit that acquires position and posture information of a target object in addition to robot position and joint angle information.

(5). In (1), the machine learning system includes an acquisition unit that acquires robot speed, joint angular velocity information, etc. as teaching data to the robot.

(6). In (1), the machine learning system includes an acquisition unit that acquires tactile information of the robot as teaching data to the robot.

(7). In (1), the machine learning system includes a dynamics adjustment unit that adjusts acceleration and/or changes in acceleration among the dynamics of the robot.

(8). In (1), the machine learning system includes a dynamics adjustment unit that adjusts jerk and/or jerk change among the dynamics of the robot.

(9). In (1), the machine learning system includes a dynamics adjustment unit that adjusts torque or/and torque change among the dynamics of the robot.

(10). In (1), when the teaching data to the robot includes a camera image, the speed adjusting section performs frame interpolation between the teaching data and then adjusts the teaching data so as to equalize the speed of the same type of motion. A machine learning system that is characterized by correction.

(11). In (1), the machine learning system includes a screening device that extracts teaching data effective for improving motion generalization performance from acquired teaching data.

(12). In (1), the machine learning system is equipped with a motion parameter adjustment device that adjusts the control cycle of the robot in order to achieve an arbitrary motion speed when generating motion using a trained machine learning model. Machine learning system.

(13). In (1), the machine learning system is equipped with a motion parameter adjustment device that adjusts the torque of the robot in order to generate an arbitrary force when generating motion using a trained machine learning model. , machine learning systems.

According to (1) to (13), by reducing variations in the time direction between teaching data that impede motion learning, it is expected that both reduction of the introduction man-hours and acquisition of generalization performance can be achieved.

DESCRIPTION OF SYMBOLS 1... Robot device, 2... Sensor data acquisition part, 3... Motion planning part, 4... Control part, 5... Screen operation part, 22... ROM, 23... CPU, 24... RAM, 25... Non-volatile storage, 26... Input Output interface, 27... Network interface, 31... Sensor data storage device, 32... Consistency verification device, 33... Learning data storage device, 34... Machine learning device, 41... Motion segmentation unit, 42... Speed adjustment unit, 43... Timing adjustment section, 44... Dynamics adjustment section, 45... Data processing section, 61... Machine learning model definition section, 62... Learning section, 63... Learned weight accumulation section, 64... Inference section, 101... Frame interpolation section, 131... Screening device, 161... Operation input section, 151... Grouping section, 152... Representative data calculation section, 153... Outlier data detection section, 154... Calculation result output section, 161... Operation input section, 162... Screen display section, 163... Screen Control unit, 171... Operating parameter adjustment device, 181... Operating parameter storage unit, 182... Operating parameter adjustment calculation unit

Claims (15)

Segmenting the teaching data to the robot for each of the same types of movements of the robot,
correcting the plurality of segmented teaching data to align the speed or timing of the same type of motion of the robot;
Synthesizing the plurality of corrected teaching data,
A processing device including a processor that performs machine learning using the synthesized teaching data. The processing device according to claim 1,
The processor includes:
A processing device characterized in that the plurality of segmented teaching data are corrected to smooth values of dynamics of the same type of motion of the robot. The processing device according to claim 2,
The teaching data is
position information indicating the position of the robot; joint angle information indicating joint angles;
A processing device comprising at least one of speed information indicating a speed of the robot, joint angular velocity information indicating a joint angular velocity of the robot, and tactile information indicating a sensor value of a tactile sensor provided in the robot. The processing device according to claim 3,
The teaching data is
A processing device comprising an image of the robot or the working environment. The processing device according to claim 3,
The teaching data is
A processing device comprising at least one of position information indicating a position of an object on which the robot works, and posture information indicating an attitude of the object. The processing device according to claim 2,
The dynamics are
acceleration, acceleration change,
jerk, jerk change,
A processing device comprising at least one of torque and torque change. The processing device according to claim 4,
The processor includes:
A processing device characterized by performing frame interpolation of an image included in the plurality of segmented teaching data, and then performing correction to equalize the speeds of the same type of motion of the robot. The processing device according to claim 1,
The processor includes:
A processing device characterized in that screening is performed by removing data that deviates from the teaching data. The processing device according to claim 1,
comprising a storage device that stores motion parameters indicating parameters for adjusting the motion of the robot after machine learning;
The processor includes:
A processing device that generates command values for the robot's motion based on the motion parameters using a trained machine learning model. The processing device according to claim 9,
The operating parameter is a sampling period of the teaching data,
The processor includes:
In the command value, the speed of operation of the robot is increased by making the control period of the robot smaller than the sampling period, or in the command value, the control period of the robot is made larger than the sampling period. A processing device characterized by reducing the speed of robot movement. The processing device according to claim 9,
The operation parameter is the maximum value of the torque of the robot or a value correlated thereto in the teaching data,
The processor includes:
In the command value, the force of the robot is increased by making the upper limit of the torque of the robot or a value correlated therewith larger than the maximum value, or in the command value, the torque of the robot or a value correlated therewith is increased. A processing device characterized in that the force of the robot is reduced by making an upper limit of the robot smaller than the maximum value. The processing device according to claim 1,
The processor includes:
A processing device characterized by quantizing the plurality of segmented teaching data and removing noise to equalize the speeds of the same type of motion of the robot. The processing device according to claim 1,
The processor includes:
A processing device characterized in that the plurality of segmented teaching data are corrected by increasing or decreasing the stationary time of the robot so as to align the timings of the same type of motion of the robot. A robot control system comprising the processing device according to claim 1 and a robot,
The processor generates a command value for the robot's operation using a learned machine learning model,
The robot control system is characterized in that the robot operates according to the command value. a step of segmenting the teaching data to the robot for each of the same type of movements of the robot;
a step of correcting the plurality of segmented teaching data to align the speed or timing of the same type of motion of the robot;
a step of synthesizing the plurality of corrected teaching data;
performing machine learning using the synthesized teaching data;
machine learning methods, including;

Images