JP2021030393A - Robot control device - Google Patents

Abstract

To realize flexible control in accordance with a wide range of environments and situations while allowing individual management and correction of control of operations in accordance with the respective environments and situations.SOLUTION: A range of environments and situations in which control for enabling each control routine (hereafter referred to as operation control) for executing an operation to achieve the operation purpose is possible is described by a machine learning model. Appropriate operation control for a current situation is selected and executed according to a model output (e.g., whether it is within the range).SELECTED DRAWING: Figure 1

Description

The present invention relates to the control of a robot having some kind of sensor, sensing surrounding objects by the sensor, and interacting with other objects (including an object that can be said to be an "environment", for example, a wall or a floor). The present invention also relates to the configuration of a control program for management, evaluation verification, and improvement of robot motion control.

An environment in which robots (eg, autonomously controlled domestic robots, nursing robots, etc.) that perform operations that involve interaction with other objects in an environment with a degree of freedom (open environment) in which the environment and tasks are not fixed are placed. -It is necessary to operate flexibly according to the situation. In addition, the actions to be performed vary depending on the task.

At first glance, the robot arm in the factory workshop only needs to move a single path (path) for a single object (environment, situation, task is fixed), and it becomes a robot. The required requirements are different. In addition, when considering practical application, there are aspects such as evaluation verification and improvement.

As an example of movement, move an object placed in various places to another various place, grab an object from the side, move it while holding it, put it in another place, grab it and then turn it. There are things such as passing the stick provided in the hole that the object has, inserting the grasped object into the gap between the object and the object, stacking the object, and so on.

In addition, as an example of the environment / situation in which an operation is executed, there are various places where an object is placed, what kind of object exists around it, and whether an object physically interferes with another object. ..

As an example of considering surrounding objects, a system has been proposed that confirms the existence of surrounding objects (which space the object occupies) and confirms whether or not the robot physically interferes (for example, patent documents). 1).

Further, a system has been proposed in which sensor information is input to a machine learning model, the target device is controlled by the output of the machine learning model, and flexible control is realized (see, for example, Patent Document 2).

Japanese Unexamined Patent Publication No. 2005-81445 JP-A-2017-64910

The conventional system disclosed in Patent Document 1 described above determines whether or not a surrounding object occupies the space and operates in a region where no other object exists. Therefore, for example, actions involving physical interaction such as pushing away other objects and placing the objects in place, and changes in actions according to the type of object are not considered, and such actions are included. It is not possible to respond to the control.

Further, the system disclosed in Patent Document 2 can be flexibly controlled in a wide range of situations by controlling using a machine learning model, but management, evaluation verification, and modification of motion control for each individual situation. Etc. are difficult.

Further, in the above-mentioned conventional technique, in the correction of the operation control regarding the operation error in a certain situation, the influence of the correction on the operation of the operation control in another situation is not considered. The modification may affect the behavior of motion control in other situations. In a situation where the operation was originally established (a situation in which the operation achieving the purpose was achieved), the operation may be changed by the modification and the operation may not be established. Therefore, it is necessary to evaluate and verify again whether or not the operation is established in all situations.

An object of the present invention is to provide a robot control device capable of individually managing and modifying the control of movements according to each situation while realizing flexible control according to a wide range of environments and situations.

In order to achieve the above object, the invention according to claim 1 describes the current state (position, posture) of the physical characteristics including the physical structure and shape of the surrounding object or the target robot itself, and the characteristics that can be changed with time. ,, temperature, etc.), their relative relationship, combination, information (situation information), any situation information is input, and the presence or absence of operation establishment in the input situation of the given operation control (2) It is characterized by including a machine learning model that outputs a value (a value, a continuous value, or a multidimensional vector value), and an operation selection execution unit that selects and executes an operation control based on the value output by the model.

The invention according to claim 2 is a machine learning model that outputs the presence / absence or degree (binary value, continuous value, or multidimensional vector value) of the given motion control in the input situation. Each control is constructed individually, or even if it is a single model configuration, it does not affect the output of the presence or absence or degree of operation establishment in the operation control other than the operation control of the learning target, or it has a slight effect. It is characterized in that such learning is performed using a model capable of learning a model of motion control of an object by a method that does not give.

The invention according to claim 3 is characterized in that the input (situation information) of the machine learning model includes at least one of the position (absolute position, relative position), posture, shape, and type of the object. ..

The invention according to claim 4 obtains a situation data-establishment / non-establishment data pair generated from a design specification value of operation control, an operation result in an actual machine, a human rule of thumb, an operation result in a physical simulator, and the like. , It is characterized in that it is used as training data of a machine learning model.

The invention according to claim 5 is a condition that defines what kind of state in the motion control is the state in which the motion is established in the motion control, in addition to the motion control when the data is generated by using the physics simulator in the learning data generation unit. It has a function to change the environment (situation information) in various ways in the physics simulator, execute the given motion control, test whether the motion is established, and create a status data-establishment data pair based on the result. It is characterized by that.

The invention according to claim 6 is characterized in that, when testing the establishment of an operation with an object simulator, at least one environment (situation information) in which the given operation control is established is provided.

INDUSTRIAL APPLICABILITY The present invention provides a robot control device capable of individually managing and modifying the control of operations according to each situation while realizing flexible control according to a wide range of environments and situations.

More specifically, according to the first aspect of the present invention, it is possible to flexibly determine the situation in which the operation of the operation control is established and execute the operation appropriately. In other words, the boundaries of the range of situations in which motion control operations are established, which are high-dimensional and extremely complex with a wide variety of information and are difficult for humans to imagine or describe manually, are also machines. By using a learning model, it becomes easy to describe.

According to the invention of claim 2, in the machine learning model for determining the situation, the addition of the situation judgment regarding the new motion control or the modification of the situation judgment regarding the existing motion control is added to the situation judgment of the other motion control. It can be done without any influence.

According to the invention of claim 3, the same effect as that of the invention of claims 1 and 2 is exhibited.

According to the invention of claim 4, the same effects as those of the inventions of claims 1 and 2 are exhibited.

According to the invention of claim 5, it is possible to automatically construct or modify (learn or relearn) a model for determining a situation.

According to the invention of claim 6, when the model of the situation judgment is automatically constructed or modified (learning or re-learning), the given situation information is used as a starting point when scanning the situation to be tested. As a result, it is possible to efficiently test the range of situations that hold.

FIG. 1 is a diagram illustrating a basic concept of the present invention. FIG. 2 is a diagram showing details of the basic concept of the present invention. FIG. 3 is a block diagram showing a configuration of a robot control device according to an embodiment. FIG. 4 is a diagram illustrating preprocessing by the robot control device according to the embodiment. FIG. 5 is a diagram illustrating an operation (operation control) of the robot control device according to the embodiment. FIG. 6 is a diagram illustrating a process at the time of maintenance / evaluation verification of the robot control device according to the embodiment. FIG. 7 is a diagram showing a specific example of a machine learning model included in the robot control device according to the embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram illustrating a basic concept of the present invention. Here, the "range of environment / situation" when processing the target object with the robot arm is illustrated. In this figure, "NG" exemplifies an "environment / situation range" in which the robot arm cannot operate.

As shown in this figure, the present invention uses the concept of "a range of operating environments / situations that do not fail". That is, for each operation purpose and operation control, the "range of environment / situation that can be operated without failure" is automatically calculated, and each operation control is managed individually.

Here, the environment / situation means the type and placement of the target object and its surrounding objects. Since it consists of various types of objects, positional relationships, and situations, it is conceptually a multidimensional space. In addition, depending on the movement, the shape and posture of the robot itself are also included in the environment / situation. In addition, when considering the cooperative operation of a plurality of robots, the positional relationship / posture relationship of each robot is also included in the environment / situation.

In the present embodiment, the robot arm will be described as an example, but the present invention can be applied to all kinds of robots regardless of the shape and structure of the robot.

FIG. 2 is a diagram showing details of the basic concept of the present invention. As shown in this figure, in the present invention, the operation purpose is realized by combining a plurality of operation controls for each operation purpose. Each motion control is managed individually and is independent.

FIG. 2A exemplifies the conceptual space of the environment / situation for the operation purpose A (for example, moving an object). In this example, the operation control A1 is possible in the operable environment / situation range A1. Operation control A2 is possible in the operable environment / situation range A2. In the operable environment / situation range A3, the operation control A3 is possible.

FIG. 2B illustrates the conceptual space of the environment / situation for the operation purpose B (for example, changing the posture of an object). In this example, the operation control B1 is possible in the operable environment / situation range B1. Operation control B2 is possible in the operable environment / situation range B2. Operation control B3 is possible in the operable environment / situation range B3.

In the present invention, as shown in FIG. 2, one operation purpose is realized by combining one or more operation controls in consideration of a range that does not fail.

FIG. 3 is a block diagram showing a configuration of the robot control device 10 according to the embodiment. The robot control device 10 describes the target motion input unit 12, the current status detection unit 14, the motion control selection unit 16, the motion control execution unit 18, and the ML (machine learning) that describes the operable status range of the motion control of each target motion. ) Model 20, operation control set 22, target operation / operation control design specification value / actual machine operation value / empirical rule / operation control / establishment judgment standard / operation basic status, etc. It is composed of a data pair generation unit 26, an operation physical simulation unit 28, and an operationable status range confirmation unit 30. The robot control device 10 has a model (ML model 20) that describes an operable situation range for each motion control, and uses data samples for learning the model as motion control design specification values, actual machine motion values, empirical rules, and physics. Obtained from a simulator (physical simulation unit 28 of motion) or the like. This process is automatically performed regardless of the content of the operation control.

When a machine learning model is used for control in motion control given by various information input units 24, the ML model 20 may be integrated into the machine learning model. For example, one machine in which the input type of the machine learning model for motion control and the input type of the ML model 20 are common, and an output for establishment / rejection is provided separately from the output for motion control. For example, integration into a learning model.

Further, in the case of individual machine learning models for each motion control, the inputs and outputs of each machine learning model do not have to be in the same format.

In addition, the following two things can be said about the correspondence of the model output in the addition and modification of the motion in the configuration of a single model instead of the individual machine learning model for each motion control.

(1) "The model configuration is fixed and only the output value is changed, and it is a model that adds motion control and corrects the situation range, and does not affect the output value related to motion control other than the motion control to be learned. If you use a model that can learn the model related to the motion control of the target in a way that affects only a little, the output of the model is, for example, a multidimensional vector, and the addition of motion control is a new pattern. Learning is performed so that the output value of is output.

(2) "Change of model configuration This is a model that adds motion control and corrects the situation range by expanding and changing the output value, and affects the output value related to motion control other than the motion control of the learning target. When using a model that can learn a model related to the target motion control in a way that has little or no effect, for example, adding a new output calculation unit for motion control as a model configuration, etc. Add new motion control by changing the calculation structure.

In the following, for the sake of simplification of the explanation, an individual machine learning model for each motion control is assumed.

FIG. 4 is a diagram illustrating pre-processing by the robot control device 10 according to the embodiment. Here, among the block diagrams shown in FIG. 3, only the blocks related to the preprocessing are shown.

In order to learn the ML model 20 in the motion control to be used, the generation unit 26 of the situation data / establishment possibility data pair generates a large amount of situation data / establishment possibility data pairs, which are learning data. The generation is based on the design specification value of the operation control, the operation value in the actual machine, the empirical rule of the operation control creator, etc., and the situation data-establishment possibility data pair indicating whether or not the operation is established in what situation. The operation control is performed by putting the operation control and the criteria for determining whether or not the operation is established in various situations, and the basic (typical) situation when operating the operation in the physical simulation unit 28 of the operation. It is generated based on the result of operation in various situations.

FIG. 5 is a diagram illustrating an operation (operation control) of the robot control device 10 according to the embodiment. Here, among the block diagrams shown in FIG. 3, only the blocks related to the operation (motion control) of the robot control device 10 are shown.

The motion control selection unit 16 is added to the abstract model of the situation range (ML model 20 existing for the number of motion controls corresponding to the target motion) corresponding to the target motion input from the target motion input unit 12 in the current situation. The current status data obtained from the detection unit 14 is input, and the output of each ML model 20 is confirmed (the ML model 20 outputs an inferred value as to whether or not the corresponding operation control is possible).

The motion control selection unit 16 determines from the output value of the ML model 20 which motion control can be operated in the current situation (note that there is a case where a plurality of motion controls can be operated. When the ranges overlap. ), And select the motion control from the set 22 of the motion control. Then, the motion control execution unit 18 executes the selected motion control. That is, the current situation is put into the ML model 20, it is inferred whether or not it fails, and the motion control that appears if it does not fail is adopted and executed.

FIG. 6 is a diagram illustrating a process at the time of maintenance / evaluation verification of the robot control device 10 according to the embodiment. Here, among the block diagrams shown in FIG. 3, only the blocks related to maintenance / evaluation verification are shown.

The confirmation unit 30 of the range of the operable situation examines the ML model 20 (learned in the pre-processing) of each motion control, what kind of motion control is associated with a certain purpose motion in a certain situation, and what kind of motion control is associated with. Check if the motion control is also associated, create a new motion control if necessary, and map it according to the procedure of "pre-processing".

Further, when it is desired to improve the operation control in a certain purpose operation in a certain situation, the confirmation unit 30 of the range of the operable situation determines (inspects) the operation control associated with the situation, and in response to the operation control Make corrections. Since the motion controls are completely separated according to the possible range (that is, each motion control is independent), the modification does not affect the other motion controls.

In this way, when confirming whether or not the operation is appropriate in a certain situation (check 1), it is possible to confirm the output of the ML model 20 by the confirmation unit 30 in the range of the operable situation. Further, when it is desired to improve the control in a certain situation (check 2), the ML model 20 may be viewed by the confirmation unit 30 of the range of the operable situation, and the corresponding operation control may be corrected.

FIG. 7 is a diagram showing a specific example of the ML model 20 included in the robot control device 10 according to the embodiment. In the present embodiment, as the ML model 20, a DNN (Deep Neural Network) called Three Dimensional Convolutional Neural Network is used. The DNN inputs the presence / absence of an object at each position in the space, the type of the object, and the like, and outputs whether or not the operation is possible (OK / NG).

As described above, the robot control device 10 according to the present embodiment satisfies all of the following four necessary requirements 1 to 4 for realizing robot control.

-Requirement 1: Flexible control according to a wide range of environments and situations is possible.

This is possible because the robot control device 10 according to the present embodiment can combine a plurality of motion controls and there are no restrictions on the type of control method.

-Requirement 2: It is possible to control in consideration of physical phenomena (interference of the target object with other objects, such as pushing, friction, along, balance of the center of gravity, etc.).

This is possible because the robot control device 10 according to the present embodiment determines by physical simulation.

-Requirement 3: It is possible to clarify the range of environment / situation in which each control routine for executing an operation (hereinafter referred to as operation control) is established (that is, the operation is performed so as to achieve the purpose).

This is possible because the robot control device 10 according to the present embodiment models the operable situation range.

-Requirement 4: Each operation control can be tuned individually.

This is possible because the robot control device 10 according to the present embodiment handles the operation control individually for each possible situation range.

The present invention can be used as a robot control device, particularly as a robot control device that can individually manage and modify the control of movements according to each situation while realizing flexible control according to a wide range of environments and situations. ..

10 Robot control device 12 Purpose operation input unit 14 Current status detection unit 16 Operation control selection unit 18 Operation control execution unit 20 ML (machine learning) model 22 Operation control set 24 Various information input units 26 Status data / feasibility data Pair generation unit 28 Operation physics simulation unit 30 Operation status range confirmation unit

In order to achieve the above object, the invention according to claim 1 exists for each operation purpose as many as the number of operation controls corresponding to the operation purpose, and has a physical structure and shape of a surrounding object or the target robot itself. Of the information (situation information) of the physical characteristics including, the current state (position, posture, and temperature) of the characteristics that can be changed with time, their relative relationships, combinations, and information (situation information), any situation information is input and given. When a machine learning model that outputs an output value (binary value, continuous value, or multidimensional vector value) indicating whether or not the operation is established in the input situation, and the operation purpose and situation information are input. By giving the situation information to the machine learning model for the number of motion controls corresponding to the motion objectives, the output values for each of the motion controls corresponding to the motion objectives are acquired and acquired. An operation control selection unit that selects an operation control to be executed from the operation controls corresponding to the operation purpose based on an output value, and an operation control execution unit that executes the operation control selected by the operation control selection unit. The motion control is a control routine for achieving an operation purpose by sensing a surrounding object with a sensor and interacting with another object .

Invention according to claim 2, wherein the machine learning model, for each given operation control, or are constructed individually, be configured of a single model, the operation control other than the operation control to be learned It is characterized in that such learning is performed using a model capable of learning a model of a target motion control by a method that does not affect the output of the output value in the above.

According to a third aspect of the invention, the status information includes at least the position of the object (the absolute position, relative position), and wherein the posture, the shape, to be included any of the types.

Invention according to claim 4, design specification value of the operation control, the operation result of the actual machine, the human heuristics, or the operation results or we generate within physics simulator, status data - established feasibility data pairs and characterized by further comprising a learning data generating unit for generating as a learning data of the machine learning model.

Invention of claim 5, wherein the learning data generating unit, when generating, using the physical simulator, in addition to the operation control, what state in the operation control define whether the state is an operation established at that operating control The conditions to be specified are given, the environment (situation information) is changed in various ways in the physics simulator, the given operation control is executed, it is tested whether or not the operation is established, and the result is generated as the above-mentioned situation data-establishment possibility data pair. characterized in that it.

Invention according to claim 6, when testing the operation established by the physical simulator, the environment (status information) given operational control is established operation, at least one example given, starting the point, test It is characterized by scanning the environment (situation information).

Claims (6)

Information on the physical characteristics including the physical structure and shape of the surrounding object or the target robot itself, the current state (position, posture, temperature, etc.) of the characteristics that can change with time, their relative relationships, combinations, etc. A machine learning model that inputs arbitrary situation information (situation information) and outputs the presence or absence (binary value, continuous value, or multidimensional vector value) of the operation establishment in the input situation of the given operation control. ,
Based on the value output by the model, the operation selection execution unit that selects and executes the operation control,
A robot control device characterized by being equipped. A machine learning model that outputs the presence / absence or degree (binary value, continuous value, or multidimensional vector value) of the given motion control in the input situation is constructed individually for each given motion control. Or, even if it is a single model configuration, the operation control of the target is performed by a method that does not affect or slightly affects the output of the presence or absence or degree of operation establishment in the operation control other than the operation control of the learning target. The robot control device according to claim 1, wherein such learning is performed using a model capable of learning the model of. The robot control device according to claim 1 or 2, wherein the input (situation information) of the machine learning model includes at least one of the position (absolute position, relative position), posture, shape, and type of the object. The situation data-establishment / rejection data pair generated from the design specification value of the motion control, the motion result in the actual machine, the human empirical rule, the motion result in the physical simulator, etc. is used as the learning data of the machine learning model. , The robot control device according to claim 1, 2 or 3. When generating using a physics simulator in the learning data generation unit, in addition to the motion control, a condition that defines what kind of state is the state in which the motion is established in the motion control is given in the motion control, and the environment (in the physics simulator) Claims 1, 2, 3 or claim 1, which has a function of executing a given motion control by variously changing (situation information), testing whether or not the motion is established, and creating a status data-establishment / rejection data pair based on the result. 4. The robot control device according to 4. The robot control device according to claim 5, wherein at least one environment (situation information) in which the given motion control is established when the motion establishment is tested by the object simulator is given.