JP7378309B2 - working equipment - Google Patents

Description

The present invention relates to a working device.

Patent Document 1 discloses a system for automatically operating a robot arm. This system includes a plurality of imitation models that imitate the operation of a robot arm by an operator using machine learning, and a model selection unit that selects an imitation model to be used based on classification of data of the surrounding environment.

JP2018-206286A

Conventionally, there are automatic driving systems that predict certain conditions using predictive models and automate operations based on the predicted results. However, predictions made by conventional autonomous driving systems have only made simple motion predictions, such as predicting the motion of a single object. Therefore, in conventional automatic driving systems, it is difficult to handle multiple objects that interact with each other and change their positions.

An object of the present invention is to provide a work device that can automate operations on a plurality of objects.

(1)
A working device according to one embodiment of the present invention includes:
A working device that stores a plurality of objects in a container,
a movable part capable of operating the plurality of objects;
a state acquisition unit that acquires the states of the plurality of objects;
an operation determining unit that predicts a change in the state of the plurality of objects after the operation and determines an operation for the plurality of objects;
an operation control unit that causes the movable unit to perform the operation determined by the operation determination unit;
Equipped with
The operation determining unit uses a machine-learned prediction model that predicts the states of the plurality of objects after the operation by the movable part, including state changes due to interactions of the plurality of objects, and the prediction model. an evaluation processing unit that evaluates a prediction result, and determines an operation based on the prediction using the prediction model and the evaluation by the evaluation processing unit,
The evaluation processing unit evaluates the prediction result based on gaps between a plurality of objects in the container and the number of objects in the container.
(2)
A working device according to another aspect of the present invention includes:
A working device that transports earth and sand as multiple objects,
a movable part capable of operating the plurality of objects;
a state acquisition unit that acquires the states of the plurality of objects;
an operation determining unit that predicts a change in the state of the plurality of objects after the operation and determines an operation for the plurality of objects;
an operation control unit that causes the movable unit to perform the operation determined by the operation determination unit;
Equipped with
The operation determining unit uses a machine-learned prediction model that predicts the states of the plurality of objects after the operation by the movable part, including state changes due to interactions of the plurality of objects, and the prediction model. an evaluation processing unit that evaluates a prediction result, and determines an operation based on the prediction using the prediction model and the evaluation by the evaluation processing unit,
The evaluation processing unit evaluates the prediction result based on a comparison between the sediment shape of the prediction result and the target sediment shape.
(3)
A working device according to another aspect of the present invention includes:
A movable part that can operate on multiple objects,
a state acquisition unit that acquires the states of the plurality of objects;
an operation determining unit that predicts a change in the state of the plurality of objects after the operation and determines an operation for the plurality of objects;
an operation control unit that causes the movable unit to perform the operation determined by the operation determination unit;
a setting processing unit capable of setting target state data of the plurality of objects;
Equipped with
The operation determining unit uses a machine-learned prediction model that predicts the states of the plurality of objects after the operation by the movable part, including state changes due to interactions of the plurality of objects, and the prediction model. an evaluation processing unit that evaluates a prediction result, and determines an operation based on the prediction using the prediction model and the evaluation by the evaluation processing unit,
The evaluation processing unit evaluates the prediction result using the data of the target state.

According to the present invention, it is possible to provide a work device that can automate operations on a plurality of objects.

FIG. 1 is a block diagram showing a working device according to Embodiment 1 of the present invention. FIG. 3 is a diagram illustrating variables used for evaluation. 3 is a flowchart showing a procedure of work processing executed by a control unit. It is an explanatory diagram showing an example of the 1st operation and evaluation. It is an explanatory diagram showing an example of the 2nd operation and evaluation. FIG. 2 is a block diagram showing a working device according to Embodiment 2 of the present invention. FIG. 7 is a diagram illustrating automatic operation processing of the working device according to the second embodiment.

Embodiments of the present invention will be described in detail below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a working device according to Embodiment 1 of the present invention. In the first embodiment, the object to be operated is a workpiece such as an injection molded product or a glass. The work device 1 of the first embodiment is a device that automatically stores a plurality of workpieces in a container (box), and aims to efficiently store a large number of workpieces in the container.

As shown in FIG. 1, the work device 1 includes an imaging unit 3 for acquiring the positions of a plurality of workpieces, a movable part 2 such as a robot hand that can operate the workpieces, and a movable part 2 that moves the movable part 2 to perform operations. It also includes a control unit 10 that performs automatic operation. The photographing section 3 corresponds to an example of a state acquisition section according to the present invention.

The movable part 2 is capable of adding a workpiece to a container and moving a workpiece within the container.

The control unit 10 includes a storage unit that stores a control program, a CPU (Central Processing Unit) that executes the control program, and an I/O unit that inputs captured images from the imaging unit 3 and outputs control signals to the movable unit 2. /O. In the control unit 10, several functional modules are realized by the CPU executing a control program. The functional module includes an operation determining section 11 that determines the operation of the movable section 2, and an operation control section 12 that moves the movable section 2 to execute the operation determined by the operation determining section 11.

The operation determining unit 11 includes a machine-learned prediction model 111 that predicts changes in the states of a plurality of workpieces when the movable unit 2 performs an operation on the workpieces, and an evaluation processing unit that evaluates the states of the plurality of workpieces. 112 is included.

The prediction model 111 predicts the states of a plurality of objects (workpieces, etc.) that influence each other after a certain operation is performed on the objects. For the prediction model 111, for example, a neural network that approximates the simulation of a multi-body problem using machine learning can be applied.

The evaluation processing unit 112 evaluates the states of a plurality of objects (workpieces, etc.) predicted by the prediction model 111. The evaluation processing unit 112 is designed to output a higher evaluation value as the states of the plurality of objects are more desirable. The desired state may be, for example, a state that quickly approaches the target state. The evaluation processing unit 112 evaluates the states of a plurality of objects (workpieces, etc.) using an evaluation function. The evaluation function may be configured so that the user can input settings.

The operation determining section 11 determines the next operation of the movable section 2 based on the prediction result of the prediction model 111 and the evaluation of the evaluation processing section 112. For example, the operation determination unit 11 sets a plurality of operation steps ahead as a prediction horizon, and determines the next operation step based on the prediction results and evaluations of the states of the plurality of objects in the prediction horizon. The operation determining unit 11 selects various combinations of operations, executes multiple prediction horizon state predictions and evaluations using the prediction model 111 and the evaluation processing unit 112, and compares the evaluations. Then, the prediction horizon with the highest evaluation is found, and the operation of the first operation step of the prediction horizon is determined as the next operation.

The operation control section 12 moves the movable section 2 to execute the operation determined by the operation determination section 11.

Next, a specific example of the prediction model 111 and the evaluation processing unit 112 will be explained. The prediction model and evaluation processing unit according to the present invention are not specifically limited to the following.

<Prediction model>
In the prediction model 111, the state vector of the i-th object (such as a workpiece) is written as x _i ^k , and the set thereof is written as X ^k ={x _i ^k |i=1,...,N ^k }. Furthermore, an operation applied to an object is written as ^uk . The subscript k represents discrete time. The neural network of the prediction model 111 inputs the state vector set X k-1 of the object at a certain discrete time k-1 and the operation u ^k , as shown in the following equation (1), and calculates the state vector set X ^k-1 of the object at the next discrete time k. can be expressed as a function f whose output is the state vector set X ^k of .

The predictive model 111 includes, for example, Chang, Michael B. et al., "A compositional object-based approach to learning physical dynamic.", arXiv preprint arXiv:1612.00341 (2016). The neural network described in ICLR2017. can be applied. The above-mentioned literature describes a neural network that deals with the simulation of many-body problems. When performing machine learning of the prediction model 111, many learning data sets {X ^k , X ^k-1 , u ^k-1 } are prepared from simulation data obtained by attempting various patterns of operations on multiple objects. . Distinct Element Method etc. can be used for simulation. {X ^k-1 , u ^k-1 } is training input data, and {X ^k } is a target value. A machine-learned prediction model 111 is obtained by giving a learning data set to the neural network and optimizing each parameter by backpropagation (error backpropagation method) or the like.

The state vector x _i ^k of the object includes multiple elements, such as whether it is a workpiece, whether it is a wall surface, whether it is a robot arm or not, a two-dimensional position, a two-dimensional velocity, an orientation θ from a reference point, It may have an angular acceleration ω centered on the reference point, and so on. Since the state vector x _i ^k has an element indicating the type of object, the state vector x _i ^k represents the state of not only the workpiece but also other objects such as the wall of the container and the movable part 2 (robot arm). be able to. The states of other objects can be included in the state vector set ^Xk .

The operation u ^k applied to the object has a plurality of elements, for example, the type of operation a ^k by the movable part 2 and the amount of operation v ^k by the movable part 2. The type of operation ^ak includes an operation of putting a workpiece into a container, an operation of moving a workpiece in a container, and the like. The manipulated variable v _k is information about the direction and length in which the workpiece is moved by the movable part 2 .

<Evaluation processing section>
The evaluation processing unit 112 has an evaluation function L, inputs the state vector set ^Xk , and outputs an evaluation value. The evaluation function L is designed so that a high evaluation value will be obtained if the state vector set X ^k quickly approaches the target state, and a low evaluation value will be obtained if the state vector set X k quickly approaches the target state. In the first embodiment, the evaluation function L is designed so that a state in which a large number of works can be packed into a container results in a high evaluation value. The evaluation function L may have a term for each type of operation if there are multiple types of operations performed on multiple objects.

<Evaluation function L _A of the operation of pushing the workpiece in the container to create a gap>
The evaluation function _LA related to the operation of creating a gap may be designed so that a high evaluation value is obtained when a large gap is obtained. This is because the large gap makes it possible to insert the workpiece. In order to create the evaluation function _LA , first, a vector d indicating the distance between an arbitrary point p and each part is introduced. FIG. 2 is a diagram illustrating the vector d.

Here, p is a position vector of an arbitrary point in the container, D _p is a function representing the distance between two points, D _l is a function representing the distance between one point and a straight line, and y ₁ to y _M are the position vectors of arbitrary points in the container. The position vectors b ₁ to b _L of each workpiece are quantities that can specify the position angle plane length of each wall of the container. Each element of the vector d indicates the distance between an arbitrary point p and each workpiece, and the distance between an arbitrary point p and each wall of the container. As shown in FIG. 2, if there are M=8 workpieces W in the container C1 and the walls of the container C1 are L=4, the elements of the set Y are y ₁ to y ₈ , and the elements of the set B are is b ₁ to b ₄ , and vector d has M+L=12 elements.

If all elements of vector d have large values, it indicates that there is a large gap around point p. On the other hand, if the elements of the vector d include large values and small values, there is a possibility that there are works that are separated from each other by small values within the intervals indicated by large values. In this case, the interval indicated by a large value is not a gap, and the degree of influence on evaluating the gap is low. Therefore, a vector η representing the weight of such influence is introduced.

Here, [ ] _g indicates a vector having g elements. g is 1, . . . , M, M+1, . . . , M+L, and the number of elements of vector η matches the number of elements of vector d. d _h indicates the h-th element of vector d. α and c are constants for adjustment, and are appropriately determined according to the actual workpiece. The expression for the vector η represents a weight such that the evaluation value is discounted for a distant workpiece or a gap to a wall.

Furthermore, using the above vectors d and η, a function γ is introduced as shown in the following equation (6). The function γ is a function that gives an estimated value of the size of the gap at an arbitrary point p, and includes a vector d indicating the distance to the work or the wall, and a vector η indicating the weight of the influence of the gap on the gap. Multiply the same elements to get the sum. The function γ has as arguments an arbitrary point p, a set B of position vectors of all workpieces in the container, and a set Y of information specifying all walls of the container.

任意な点ｐの最適化（γを大きくする点ｐの算出）は、容器内の点をランダムに探索して、γを大きくする点ｐ’を大まかに算出し、この点ｐ’の近傍で、よりγを大きくする点ｐを勾配法により計算することで得てもよい。勾配は、関数γの数値微分により求めることができる。 The evaluation function _LA is defined as η that is the maximum among arbitrary points p using the function γ, as shown in the following equation (7).

Optimization of an arbitrary point p (calculation of a point p that increases γ) is to randomly search points in the container, roughly calculate a point p' that increases γ, and then , the point p at which γ becomes larger may be obtained by calculating using the gradient method. The gradient can be determined by numerical differentiation of the function γ.

<Evaluation function L _B of the operation of putting the work into the container>
The evaluation function _LB related to the operation of inserting the workpieces may be designed so that a higher evaluation value can be obtained as the number of workpieces increases. Therefore, the evaluation function _LB can be defined as the number of works (the number of elements of the set Y) as shown in the following equation (8).

<Comprehensive evaluation function L>
The overall evaluation function L may be designed such that it is possible to evaluate whether to select an operation of pushing the workpiece in the container to create a gap or an operation of throwing the workpiece into the container. The overall evaluation function L can be defined by weighting and combining the evaluation functions L _A and L _B regarding each operation, as shown in the following equation (9). μ is a constant indicating positive weight.

When treating the evaluation value as a cost value (bad value), the sign of the evaluation function L may be reversed.

<Work processing>
FIG. 3 is a flowchart showing the procedure of work processing executed by the control unit. FIG. 4 is an explanatory diagram showing an example of the first operation and evaluation. FIG. 5 is an explanatory diagram showing an example of the second operation and evaluation.

For example, when a start request is received from a user, the control unit 10 starts work processing. When the work process is started, the control unit 10 first acquires images taken by the imaging unit 3 and detects the states of a plurality of objects (step S1). In the first embodiment, the plurality of objects once stops at each operation step, so in step S1, the positions of the plurality of objects are acquired as the state.

Next, in the control unit 10, the operation determining unit 11 initializes a set X ^k- 1 of state vectors x _i ^k- 1 to be used for prediction from the state acquired in step S1, that is, in step S1 The value of the acquired state is set (step S2).

Next, the operation determining unit 11 selects an operation that can be applied to the state vector set X ^k-1 (step S3). For example, as shown in FIGS. 4 and 5, if the state vector set X ^k-1 (a state in which a plurality of workpieces W are placed in the container C1) at a discrete time k-1, a gap larger than a certain level It is possible to select an operation of introducing a new workpiece W and an operation of pushing any workpiece W in the container C1 by how much and in which direction, and one of these operations is selected. Selection may be made randomly, selection may be distributed, or if the range of ideal operations is known in advance, many operations within the ideal range may be selected. . FIG. 4 shows an example in which the operation of pushing one workpiece W1 is selected by the movement of the movable part 2 as indicated by the arrow A1. FIG. 5 shows an example in which the operation of introducing a new workpiece W2 is selected.

Next, the operation determining unit 11 predicts the state vector set X ^k of the next discrete time k from the state vector set X ^k-1 and the selected operation u ^k-1 (step S4). Prediction is performed using prediction model 111.

Next, the operation determining unit 11 determines whether the prediction has reached a predetermined maximum prediction step (prediction horizon) (step S5), and if NO, returns the process to step S3 and performs steps S3 to S5. Repeat the process. If the prediction horizon becomes a large number of steps, the calculation load increases, so the prediction horizon may be set to an appropriate number of steps. For example, it may be performed in about three steps.

By repeating steps S3 to S5, a state vector set X ^k+2 of a prediction horizon (for example, three operation steps ahead) obtained by adding a plurality of operations is estimated from the initialized state of step S2. 4 and 5 show an example in which the prediction horizon is one operation step ahead.

If YES is determined in step S5, the operation determining unit 11 causes the evaluation processing unit 112 to calculate the evaluation value of the predicted state vector set X ^k+2 (step S6). The evaluation processing unit 112 inputs the state vector set X ^k+2 to the evaluation function L and calculates an evaluation value. In FIGS. 4 and 5, the prediction horizon is one operation step ahead, so the state vector set for calculating the evaluation value is X ^k . In the example of FIG. 4, the value of the evaluation function L _A calculated from the predicted state vector set X ^k has improved, and the overall evaluation has also improved, so the selected operation u ^k−1 is determined to be a good operation. There is. In the example of FIG _. 5, the value of the evaluation function _L calculated from the predicted state vector ^set Operation u ^k-1 is determined to be a good operation. Depending on the operation selection, various evaluation values, high and low, are calculated.

Subsequently, the operation determining unit 11 determines whether the evaluation in step S6 has reached a predetermined maximum number of evaluations, and if NO, returns the process to step S2 and repeats the process from step S2. By repeating steps S2 to S7, prediction results and evaluation values based on the prediction results for various operations for the maximum number of evaluations are obtained.

If YES in step S7, the operation determining unit 11 compares the evaluation values for the maximum number of evaluations obtained by repeating steps S2 to S7, and selects the first operation step selected in the prediction horizon with the highest evaluation value. is selected as the next operation to be executed (step S8).

In the control section 10, when the operation determining section 11 determines the next operation, the operation control section 12 controls the movable section 2 to execute the operation (step S9). Then, the control unit 10 determines whether or not the end condition has been reached (step S10). If the end condition has been reached, the work process is ended; if the end condition has not been reached, the process returns to step S1, and the process starts from step S1. Repeat the process. The termination condition may be, for example, a condition based on the state vector set ^Xk measured after the execution of the operation (for example, the number of workpieces in the container has reached the maximum number of fillings), or a case where the maximum number of repetitions has been reached. It may be determined as appropriate from the following.

By repeating the processing of steps S1 to S9, an operation that increases the value of the evaluation function L is selected and executed, and automatic operation that achieves the purpose of the work is realized.

As described above, according to the working device 1 of the first embodiment, the movable part 2 that can operate a plurality of workpieces, the imaging part 3 that acquires the positions of the plurality of workpieces, and the arrangement of the plurality of workpieces after the operation. The apparatus includes an operation determining section 11 that predicts the operation of the movable section 2 for a plurality of workpieces and determines the operation of the movable section 2 for a plurality of workpieces, and an operation control section 12 that causes the movable section 2 to perform the operation determined by the operation determining section 11. Therefore, it is possible to realize automatic operation that accomplishes a target task (such as filling a container with many works) for a plurality of works that interact with each other.

Further, according to the working device 1 of the first embodiment, a machine-learned prediction model 111 that predicts the states of a plurality of workpieces after the operation including changes in arrangement due to interaction of the plurality of workpieces, and The evaluation processing unit 112 evaluates the state of the workpiece, and determines the next operation to be performed based on prediction and evaluation. Therefore, operations can be determined in accordance with the purpose with a small computational load.

The work device 1 of the first embodiment can automate the operation of filling and storing workpieces in containers.

(Embodiment 2)
FIG. 6 is a block diagram showing a working device according to Embodiment 2 of the present invention. The working device 1 of the second embodiment is a device that automatically transports earth and sand, and is intended to efficiently generate a target earth and sand shape. In the second embodiment, earth and sand are used as the object to be operated and the object whose state is predicted.

As shown in FIG. 6, in the working device 1, the movable part 2A is a power shovel (shovel, crawler, swing device, etc.), and the control unit 10 further includes a setting process that allows the user to set setting data for a target state. Section 13 has been added. The setting processing section 13 includes a setting section 131 that stores setting data of the target state. Other components are the same as in the first embodiment.

The prediction model 111 applies the arrangement and density of earth and sand as the states of a plurality of objects, and predicts the state of earth and sand when performing operations such as scooping up earth and sand, transporting scooped earth and sand, and lowering earth and sand. The prediction model 111 can apply a machine learned neural network. Regarding the operation of earth and sand, since there is little interaction with earth and sand in a place far from the operation point, the prediction model 111 is designed to screen out calculations related to objects in areas where there is very little interaction. It may have a function of reducing the amount of calculation. Furthermore, if each grain of earth and sand is used as a unit of object, the amount of calculation will be enormous, so the prediction model 111 may treat a predetermined mass of earth and sand as a unit of object.

The evaluation processing unit 112 calculates an evaluation value for the predicted earth and sand arrangement state based on a predesigned evaluation function as in the first embodiment. The evaluation function includes a function that uses the target state data registered in the setting unit 131, and includes, for example, a function that provides a higher evaluation value as the difference between the predicted sediment shape and the target sediment shape is smaller. .

FIG. 7 is a diagram illustrating automatic operation processing of the working device according to the second embodiment. A line L0 in FIG. 7 indicates the target earth and sand shape registered in the setting unit 131. Also in the working device 1 of the second embodiment, similarly to the first embodiment, the control unit 10 predicts the arrangement of earth and sand using the prediction model 111 and the evaluation value calculated by the evaluation processing unit 112 during work processing. Based on this, the operation determining section 11 determines the operation, and the operation control section 12 causes the movable section 2A to execute the operation. Then, by repeating such operations, the transportation of earth and sand in accordance with the target earth and sand shape is realized through automatic operation.

As described above, the working device 1 of the second embodiment includes the setting processing unit 13 that allows the user to set target state data, and the evaluation processing unit 112 calculates an evaluation value using the target state data. Therefore, by setting the target state data, it is possible to respond to each site where the target state (target earth and sand shape) changes.

Each embodiment of the present invention has been described above. However, the present invention is not limited to the above embodiments. For example, in the above embodiment, since the operation target is an object that remains stationary after being operated, an example is shown in which the position of the object is adopted as the state of the object acquired by the state acquisition unit and the state of the object predicted by the prediction model. Ta. However, the object to be operated may be a moving object, an object in which various physical quantities such as temperature, frictional resistance, weight, current, voltage, etc. change. In this case, the state of the object acquired by the state acquisition unit and the state of the object predicted by the prediction model may include, in addition to position, velocity, each velocity, and various physical quantities. The prediction model may perform prediction by including these physical quantities in the state vector of the object. Moreover, a device that measures these physical quantities may be applied to the state acquisition unit. Other details shown in the embodiments can be changed as appropriate without departing from the spirit of the invention.

1 Working device 2, 2A Movable part 3 Photographing part 10 Control part 11 Operation determining part 12 Operation control part 111 Prediction model 112 Evaluation processing part C1 Container W, W1, W2 Work

Claims (7)

A working device that stores a plurality of objects in a container,
a movable part capable of operating the plurality of objects;
a state acquisition unit that acquires the states of the plurality of objects;
an operation determining unit that predicts a change in the state of the plurality of objects after the operation and determines an operation for the plurality of objects;
an operation control unit that causes the movable unit to perform the operation determined by the operation determination unit;
Equipped with
The operation determining unit uses a machine-learned prediction model that predicts the states of the plurality of objects after the operation by the movable part, including state changes due to interactions of the plurality of objects, and the prediction model. an evaluation processing unit that evaluates a prediction result, and determines an operation based on the prediction using the prediction model and the evaluation by the evaluation processing unit,
The evaluation processing unit evaluates the prediction result based on the gaps between the plurality of objects in the container and the number of objects in the container.
working equipment. A working device that transports earth and sand as multiple objects,
a movable part capable of operating the plurality of objects;
a state acquisition unit that acquires the states of the plurality of objects;
an operation determining unit that predicts a change in the state of the plurality of objects after the operation and determines an operation for the plurality of objects;
an operation control unit that causes the movable unit to perform the operation determined by the operation determination unit;
Equipped with
The operation determining unit uses a machine-learned prediction model that predicts the states of the plurality of objects after the operation by the movable part, including state changes due to interactions of the plurality of objects, and the prediction model. an evaluation processing unit that evaluates a prediction result, and determines an operation based on the prediction using the prediction model and the evaluation by the evaluation processing unit,
The evaluation processing unit evaluates the prediction result based on a comparison between the sediment shape of the prediction result and the target sediment shape.
working equipment. A movable part that can operate on multiple objects,
a state acquisition unit that acquires the states of the plurality of objects;
an operation determining unit that predicts a change in the state of the plurality of objects after the operation and determines an operation for the plurality of objects;
an operation control unit that causes the movable unit to perform the operation determined by the operation determination unit;
a setting processing unit capable of setting target state data of the plurality of objects;
Equipped with
The operation determining unit uses a machine-learned prediction model that predicts the states of the plurality of objects after the operation by the movable part, including state changes due to interactions of the plurality of objects, and the prediction model. an evaluation processing unit that evaluates a prediction result, and determines an operation based on the prediction using the prediction model and the evaluation by the evaluation processing unit,
The evaluation processing unit is a work device that evaluates the prediction result using data of the target state . The movable part is a robot hand,
accommodating a plurality of objects in a container by operating the movable part;
The working device according to any one of claims 1 to 3 . The movable part is a shovel,
the plurality of objects are earth and sand;
transporting the earth and sand by operating the movable part;
The working device according to claim 2 or 3 . The prediction model is a neural network that handles simulation of many-body problems.
The working device according to any one of claims 1 to 5 . The operation determining unit includes:
determining an operation to change the arrangement of some of the plurality of objects, an operation to add an object, or an operation including both;
The working device according to any one of claims 1 to 6 .

Images