JP7263987B2 - Control device, control method, and control program - Google Patents

Description

The present invention relates to a control device, control method, and control program.

Various types of manipulators are used in production lines for producing products. Manipulator mechanisms, end effectors, workpieces, and other components have many variations according to the tasks to be performed. Tasks are difficult to teach. Therefore, conventionally, after determining the types of components such as mechanisms, end effectors, and workpieces, the manipulator is manually moved, and the task to be performed is directly taught while recording the posture in a series of actions to be executed. is adopted.

However, this method teaches the manipulator what task to perform each time a component such as a mechanism, end effector, or workpiece is changed. Therefore, it is too costly to teach the manipulator what task to perform. Therefore, in recent years, research has been conducted to improve the efficiency of methods for making manipulators learn tasks to be performed. For example, Patent Document 1 proposes a control method for determining the moving speed of a hand gripping a flexible object such as a seal based on the relative speed between the hand and the flexible object. According to this control method, it is possible to automate at least a part of the work of creating or teaching the movement of the hand. Therefore, the cost of generating or teaching manipulator movements can be reduced.

JP 2015-174172 A

The inventors of the present invention have found that the conventional manipulator control method described above has the following problems. That is, in the conventional control method, the tip of the manipulator is observed by a sensor, and the coordinates of the tip of the manipulator are estimated from sensing data obtained by the sensor. Based on the result of this estimation, the coordinates of the manipulator's hand are controlled.

As an example of the method of estimating the coordinates of the hand of the manipulator, there is a method using forward kinematics calculation. In this method, an encoder that measures the angle of each joint is an example of a sensor. Estimated values of the coordinates of the manipulator's hand can be analytically calculated from the measured values of the angles of the joints obtained by the encoders.

As another method, there is a method by image analysis. In this method, a camera that captures the environment of the task, including the manipulator's hands, is an example of a sensor. By performing image analysis on the image data obtained by the camera, the coordinates of the hand of the manipulator can be estimated. A known method such as pattern matching may be adopted as the image analysis method.

In either method, the coordinates of the manipulator's hand can be estimated from the sensing data obtained by the sensor. However, sensing data obtained from sensors may contain noise. Also, in the process of analysis, for example, noise such as tolerance of pattern matching may be included. Due to these environment-dependent noises, the error between the estimated values of the manipulator's hand coordinates and the true values becomes large, which degrades the accuracy of controlling the manipulator's hand coordinates. There was a possibility of it getting lost.

SUMMARY OF THE INVENTION In one aspect, the present invention has been made in view of such circumstances, and an object thereof is to provide a technique for improving the accuracy of controlling the coordinates of the tip of a manipulator.

The present invention adopts the following configurations in order to solve the above-described problems.

That is, a control device according to one aspect of the present invention is a control device for controlling the operation of a manipulator, and includes a first data acquisition system that acquires first sensing data from a first sensor system that observes a hand of the manipulator. a first estimating unit that calculates a first estimated value of the current coordinates of the hand in an observation space from the first sensing data obtained using a first estimation model; and a hand of the manipulator. and a second data acquisition unit that acquires second sensing data from a second sensor system that observes the current state of the hand in the observation space from the acquired second sensing data using a second estimation model A second estimating unit that calculates a second estimated value of the coordinates of and calculates a gradient of an error between the first estimated value and the second estimated value, and the error is reduced based on the calculated gradient and an adjusting unit that adjusts the parameter values of at least one of the first estimation model and the second estimation model so that the coordinates of the hand are adjusted based on at least one of the first estimation value and the second estimation value. A command determination unit that determines a control command to be given to the manipulator so as to approach a target value, and a drive unit that drives the manipulator by giving the determined control command to the manipulator.

In the control device according to this configuration, the coordinates of the hand of the manipulator are estimated from the two paths of the first sensor system and the second sensor system. That is, the control device according to this configuration uses the first estimation model to calculate the first estimated value of the coordinates of the hand of the manipulator from the first sensing data obtained by the first sensor system. Further, the control device according to this configuration uses the second estimation model to calculate the second estimated value of the coordinates of the hand of the manipulator from the second sensing data obtained by the second sensor system.

There is only one true value for the coordinates of the manipulator's hand. If there is no noise in the calculation process from each sensor system, the first estimated value and the second estimated value will match. On the other hand, the first estimated value and the second estimated value may differ from each other due to noise generated according to each sensor system. Therefore, the gradient of the error between the obtained first estimated value and the second estimated value is calculated, and based on the calculated gradient, at least one of the first estimated model and the second estimated model is calculated so that the error becomes small. Adjust the value of the parameters in By this adjustment, each estimation result (estimation value) approaches one value, so it can be expected that the accuracy of estimating the coordinates of the hand by each estimation model will be improved.

Whether or not the estimated value by each estimation model approaches the true value after adjustment can be appropriately evaluated. As an example, noise that may be included in sensing data obtained by each sensor system is assumed to be white noise. Therefore, by acquiring sensing data for a predetermined time period and averaging the acquired sensing data, noise contained in the sensing data can be removed or reduced. Whether or not the averaged sensing data includes a component related to the estimated value by each estimation model makes it possible to evaluate whether or not the estimated value by each estimation model is close to the true value. Therefore, according to this configuration, it is possible to improve the accuracy of estimating the coordinates of the hand by each estimation model, thereby improving the accuracy of controlling the coordinates of the hand of the manipulator.

Note that the type of manipulator is not particularly limited, and may be appropriately selected according to the embodiment. Manipulators may include, for example, vertically articulated robots, SCARA robots, parallel link robots, orthogonal robots, collaborative robots, and the like. Each sensor system is provided with one or more sensors, and its configuration is not particularly limited as long as it can observe the hand of the manipulator, and may be appropriately selected according to the embodiment. For each sensor system, for example, a camera, an encoder, a touch sensor, a force sensor, a proximity sensor, a torque sensor, a pressure sensor, etc. may be used. Each estimation model has parameters for calculating hand coordinates from sensing data. The type of each estimation model may not be particularly limited, and may be appropriately selected according to the embodiment. Each estimation model may be represented by, for example, a function formula, a data table, or the like. When expressed by a functional formula, each estimation model may be configured by machine learning models such as neural networks, support vector machines, regression models, and decision trees.

In the control device according to the above aspect, the adjustment unit further acquires a boundary value of the coordinates of the manipulator's hand on the boundary plane of contact with the object when the hand of the manipulator contacts the object, calculating a gradient of a first error between the first estimated value estimated at the time of contact and the acquired boundary value, and reducing the first error based on the calculated gradient of the first error; and calculating the slope of a second error between the second estimated value estimated at the time of contact and the boundary value obtained, and calculating the calculated A parameter value of the second estimation model may be adjusted based on the gradient of the second error so that the second error becomes smaller.

Since there is a physical constraint of contact with the object, the boundary value obtained from the boundary surface at the time of contact with the object is a value with high accuracy as the true value of the coordinates of the manipulator's hand. In this configuration, by adjusting the parameter values of each estimation model based on this highly accurate value, it is possible to increase the accuracy of estimating the coordinates of the hand by each estimation model. Therefore, according to this configuration, it is possible to improve the accuracy of controlling the coordinates of the hand of the manipulator.

Note that the method for obtaining the boundary value is not particularly limited, and may be appropriately selected according to the embodiment. For example, the boundary values may be obtained by operator specification. Further, for example, a boundary value may be selected from points near at least one of the first estimated value and the second estimated value on the boundary surface where the hand of the manipulator contacts the object. As an example, a point closest to at least one of the first estimated value and the second estimated value may be adopted as the point that gives the boundary value. The type of object may not be particularly limited, and may be appropriately selected according to the embodiment. The target may be, for example, a work, a target to which the work is attached (for example, another work), an obstacle, or the like.

In the control device according to the above aspect, the manipulator may have one or more joints. The first sensor system may comprise an encoder that measures the angle of each joint. The second sensor system may comprise a camera. According to this configuration, it is possible to improve the accuracy of controlling the coordinates of the manipulator's hand when the encoder and camera are used to observe the manipulator's hand.

In the control device according to one aspect described above, the manipulator may further include an end effector for holding a workpiece. When the end effector does not hold the workpiece, the point of interest of the end effector may be set to the hand. When the end effector holds the work, the point of interest of the work may be set to the hand. The first sensor system may further include a tactile sensor for estimating the positional relationship of the workpiece with respect to the end effector.

According to this configuration, it is possible to improve the accuracy of controlling the coordinates of the tip of the manipulator when changing the tip of the manipulator depending on whether the end effector is holding a workpiece. In addition, by setting the hand of the manipulator, the movement when the end effector does not hold the work and the movement when the work is held can be regarded as a common task of moving the hand of the manipulator. Therefore, the manipulator control process can be simplified, thereby reducing the cost of generating or teaching manipulator movements.

The types of end effector and work may not be particularly limited, and may be appropriately selected according to the type of task or the like. The type of the task may not be particularly limited as long as at least part of the process involves movement of the manipulator's hand, and may be appropriately selected according to the embodiment. A task to be performed by the manipulator may be, for example, holding a workpiece with an end effector and assembling the held workpiece to another workpiece. In this case, the end effector may be, for example, a gripper, aspirator, driver, or the like. The workpiece may be, for example, a connector, peg, or the like. Other workpieces may be, for example, sockets, holes, and the like. Performance of tasks may be performed in real or virtual space.

As another aspect of the control device according to each of the above embodiments, one aspect of the present invention may be an information processing method or a program for realizing each configuration of the above control device. It may also be a computer-readable storage medium storing such a program. Computer-readable storage media are media that store information such as programs by electrical, magnetic, optical, mechanical, or chemical action.

For example, a control method according to one aspect of the present invention is an information processing method for controlling the operation of a manipulator, in which a computer acquires first sensing data from a first sensor system that observes a hand of the manipulator. calculating a first estimated value of current coordinates of the hand in an observation space from the obtained first sensing data using a first estimation model; and observing the hand of the manipulator. obtaining second sensing data from a second sensor system; and using a second estimation model to obtain a second estimate of the current coordinates of the hand in the observation space from the obtained second sensing data. Calculating a gradient of the error between the first estimated value and the second estimated value; Based on the calculated gradient, the first estimation model and the adjusting the value of at least one parameter of the second estimation model; and adjusting the manipulator so that the coordinates of the hand approach a target value based on at least one of the first estimated value and the second estimated value. and driving the manipulator by giving the determined control command to the manipulator.

Further, for example, a control program according to one aspect of the present invention is a program for controlling the operation of a manipulator, and causes a computer to acquire first sensing data from a first sensor system that observes a hand of the manipulator. calculating a first estimated value of current coordinates of the hand in an observation space from the obtained first sensing data using a first estimation model; and observing the hand of the manipulator. obtaining second sensing data from a second sensor system; and using a second estimation model to obtain a second estimate of the current coordinates of the hand in the observation space from the obtained second sensing data. Calculating a gradient of the error between the first estimated value and the second estimated value; Based on the calculated gradient, the first estimation model and the adjusting the value of at least one parameter of the second estimation model; and adjusting the manipulator so that the coordinates of the hand approach a target value based on at least one of the first estimated value and the second estimated value. and driving the manipulator by giving the determined control command to the manipulator.

ADVANTAGE OF THE INVENTION According to this invention, the improvement of the precision which controls the coordinate of the hand of a manipulator can be aimed at.

FIG. 1 schematically illustrates an example of a scene to which the present invention is applied. FIG. 2A schematically illustrates an example of the positional relationship between two objects according to the embodiment. FIG. 2B schematically illustrates an example of the positional relationship between two objects according to the embodiment. FIG. 3 schematically illustrates an example of the hardware configuration of the first model generation device according to the embodiment. FIG. 4 schematically illustrates an example of the hardware configuration of the second model generation device according to the embodiment. FIG. 5 schematically illustrates an example of the hardware configuration of the control device according to the embodiment. FIG. 6 schematically illustrates an example of a manipulator according to the embodiment. FIG. 7 schematically illustrates an example of the software configuration of the first model generation device according to the embodiment. FIG. 8 schematically illustrates an example of the software configuration of the second model generation device according to the embodiment. FIG. 9 schematically illustrates an example of the software configuration of the control device according to the embodiment. FIG. 10 illustrates an example of a processing procedure of the first model generation device according to the embodiment. FIG. 11 illustrates an example of a processing procedure of the second model generation device according to the embodiment. FIG. 12A schematically illustrates an example of task space according to the embodiment. FIG. 12B schematically illustrates an example of task space according to the embodiment. FIG. 12C schematically illustrates an example of a task space according to the embodiment; FIG. 13 schematically illustrates an example of an inference model configuration and generation method according to the embodiment. FIG. 14 schematically illustrates an example of an inference model configuration and generation method according to the embodiment. FIG. 15A schematically illustrates an example of learning data according to the embodiment; FIG. 15B schematically illustrates an example of the configuration of an inference model according to the embodiment; FIG. 16A illustrates an example of a processing procedure regarding motion control of a manipulator by the control device according to the embodiment. FIG. 16B illustrates an example of a processing procedure regarding motion control of the manipulator by the control device according to the embodiment. FIG. 17 illustrates an example of the calculation process of each element according to the embodiment. FIG. 18 schematically illustrates the positional relationship of each target object according to the embodiment. FIG. 19A schematically shows an example of the relationship between each joint and the hand when the end effector does not hold a work. FIG. 19B schematically shows an example of the relationship between each joint and the hand when the end effector holds a work. FIG. 20 illustrates an example of a processing procedure regarding parameter adjustment of the estimation model of the control device according to the embodiment. FIG. 21 illustrates an example of a processing procedure regarding parameter adjustment of the estimation model of the control device according to the embodiment. FIG. 22 schematically illustrates an example of a scene in which boundary values are obtained on a contact boundary surface. FIG. 23 schematically illustrates an example of the relationship between control timing and adjustment timing. FIG. 24 schematically illustrates an example of a form in which a value indicating whether or not two objects are in contact is held for each coordinate point. FIG. 25 illustrates an example of a processing procedure of a subroutine regarding target determination by the control device according to the modification.

Hereinafter, an embodiment (hereinafter also referred to as "this embodiment") according to one aspect of the present invention will be described based on the drawings. However, this embodiment described below is merely an example of the present invention in every respect. It goes without saying that various modifications and variations can be made without departing from the scope of the invention. That is, in implementing the present invention, a specific configuration according to the embodiment may be appropriately employed. Although the data appearing in this embodiment are explained in terms of natural language, more specifically, they are specified in computer-recognizable pseudo-language, commands, parameters, machine language, and the like.

§1 Application Example First, an example of a scene to which the present invention is applied will be described with reference to FIG. FIG. 1 schematically illustrates an example of an application scene of the present invention. As shown in FIG. 1, a control system 100 according to this embodiment includes a first model generation device 1, a second model generation device 2, and a control device 3. FIG. The first model generation device 1, the second model generation device 2, and the control device 3 may be connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, wireless communication network, mobile communication network, telephone network, dedicated network, and the like.

<First model generation device>
The first model generation device 1 according to this embodiment is a computer configured to generate a judgment model 50 for judging whether or not two objects come into contact with each other in the positional relationship of the objects. Specifically, the first model generation device 1 according to the present embodiment uses training data 122 indicating the positional relationship between two objects and indicating whether or not the two objects come into contact with each other in the positional relationship. A plurality of learning data sets 121 each composed of a combination of correct data 123 are obtained.

In this embodiment, the positional relationship between two objects is represented by relative coordinates. Relative coordinates are coordinates when one object is viewed from the other object. Either of the two objects may be selected as a reference for relative coordinates. "Coordinates" may include at least one of position and orientation. In a three-dimensional space, the position may be represented by three axes of forward/backward, left/right, and up/down, and the posture may be represented by rotation (roll, pitch, yaw) of each axis. In this embodiment, the relative coordinates may be expressed in six dimensions: 3D relative position and 3D relative orientation. Note that the number of dimensions of relative coordinates is not limited to six, and may be reduced as appropriate.

Then, the first model generation device 1 according to the present embodiment performs machine learning of the judgment model 50 using the acquired plurality of learning data sets 121 . Performing machine learning consists of training the decision model 50 to output an output value that matches the corresponding correct data 123 for each training data set 121 given the training data 122 input. . Through this machine learning, it is possible to build a learned decision model 50 that has acquired the ability to decide whether or not two objects touch each other in the positional relationship of the objects.

In this embodiment, the learned judgment model 50 determines whether contact occurs between the work W and the end effector T in a space where the manipulator 4 having the end effector T, the work W, and another work G exist. , and to determine whether contact between the workpiece W and another workpiece G occurs. The end effector T, work W, and another work G are examples of "objects." The types of end effector T, work W, and other work G may not be particularly limited, and may be appropriately selected according to the task. The end effector T may be, for example, a gripper, aspirator, driver, or the like. The work W may be, for example, a connector, a peg, or the like. Other workpieces G may be, for example, sockets, holes, and the like. Another work G is an example of an object to which the work W is attached. Holding the work W with the end effector T may be, for example, gripping the work with a gripper, sucking and holding the work with an aspirator, holding the work at the tip of a driver, or the like.

More specifically, the manipulator 4 according to the present embodiment holds a work W with an end effector T and performs a task of assembling the held work W to another work G, as an example. This task can be divided into two tasks: a first task of holding the work W by the end effector T and a second task of transporting the held work W to another work G. FIG. When the end effector T is moved to perform the first task of holding the work W, the learned determination model 50 determines whether unnecessary contact occurs between the work W and the end effector T. used for Further, in the scene of performing the second task of moving the end effector T after holding the work W and transporting the held work W to another work G, the learned judgment model 50 It is used to determine whether or not unnecessary contact occurs between other works G.

That is, in the present embodiment, at least one of the two objects for which it is determined whether or not contact will occur by the learned determination model 50 is a target that is moved by the operation of the manipulator 4 . Only one of the two objects may be the object to be moved by the operation of the manipulator 4, or both of the two objects may be the objects to be moved by the operation of the manipulator 4. may However, the application target of the first model generation device 1 may not be limited to such an example. The first model generation device 1 may be applied to any situation where contact between two objects is determined.

As described above, when there are a plurality of targets for determining whether or not contact occurs, a plurality of learned determination models 50 for determining whether or not contact occurs between different targets are prepared. good too. Alternatively, the learned judgment model 50 further receives input of information indicating the condition of the object such as the type of the object and the identifier of the object, and between the two objects corresponding to the input condition It may be configured to determine whether contact occurs. Either method may be adopted. In the following, for convenience of explanation, the judgment targets of the learned judgment model 50 are not distinguished.

<Second model generation device>
The second model generation device 2 according to this embodiment is a computer configured to generate an inference model 55 for determining a target task state to be given to the manipulator 4 when controlling the operation of the manipulator 4. . The manipulator 4 according to this embodiment can perform the task of moving the first object with respect to the second object in an environment where the first object and the second object exist. The above first task and second task are examples of "a task of moving a first object with respect to a second object". In the scene of performing the first task, the end effector T is an example of the first object, and the work W is an example of the second object. Also, in the scene of performing the second task, the work W is an example of the first target, and the other work G is an example of the second target. In this embodiment, a task state is defined by the positional relationship between a first object and a second object (ie two objects).

Here, a specific example of the method of defining the task state based on the positional relationship between the first object and the second object will be described with reference to FIGS. 2A and 2B. FIG. 2A schematically illustrates an example of the positional relationship between the end effector T and the work W when the first task is performed. FIG. 2B schematically illustrates an example of the positional relationship between the work W and another work G when the second task is performed. As described above, in this embodiment, the positional relationship between two objects is represented by relative coordinates.

In this embodiment, as shown in FIG. 2A, while the end effector T does not hold the workpiece W, such as when the first task is performed, the point of interest T0 of the end effector T is the tip of the manipulator 4. treated as In the first task, the work W is the target for the end effector T to move. A positional relationship between the end effector T and the work W is represented by relative coordinates RC1 of the work W with respect to the end effector T. As shown in FIG. The relative coordinates RC1 represent the local coordinate system CW having the point of interest W0 of the workpiece W as the origin, viewed from the local coordinate system CT having the point of interest T0 of the end effector T as the origin. In this embodiment, the task state of the manipulator 4 when performing the first task is defined by the relative coordinates RC1.

On the other hand, as shown in FIG. 2B, while the end effector T holds the work W, the target point W0 of the work W is treated as the hand of the manipulator 4, such as when the second task is performed. In the second task, another workpiece G is the target of the end effector T's movement. Another work G is an example of an object to which the work W is attached. The positional relationship between the work W and another work G is expressed by relative coordinates RC2 of the other work G with respect to the work W. As shown in FIG. The relative coordinates RC2 represent a local coordinate system CG whose origin is the point of interest G0 of another work G, viewed from the local coordinate system CW whose origin is the point of interest W0 of the work W. In this embodiment, the task state of the manipulator 4 when performing the second task is defined by this relative coordinate RC2.

That is, in this embodiment, the task state is defined by the positional relationship (relative coordinates in this embodiment) between the hand of the manipulator 4 and the target in both situations where the first task and the second task are performed. be. The hand of the manipulator 4 corresponds to the first object, and the target corresponds to the second object. As a result, both the first task and the second task can be regarded as tasks of moving the hand of the manipulator 4 with respect to the target. Therefore, according to this embodiment, the control processing of the manipulator 4 can be simplified, thereby reducing the cost of generating or teaching the motion of the manipulator 4 .

Note that each attention point (T0, W0, G0) may be set arbitrarily. Also, the method of giving relative coordinates need not be limited to the above example, and may be determined as appropriate according to the embodiment. For example, the relative coordinates RC1 represent the local coordinate system CT having the point of interest T0 of the end effector T as the origin, viewed from the local coordinate system CW having the point of interest W0 of the work W as the origin. The relationship (RC1, RC2) may be inverted. Also, moving the hand need not be limited to bringing the hand closer to the target, and may be determined as appropriate according to the embodiment. Moving the hand may be, for example, moving the hand away from the target, moving the hand to a predetermined position with reference to the target, or the like.

The second model generation device 2 according to the present embodiment provides the learned determination model 50 with information indicating the target task states of the first target and the second target, so that the first target in the target task state Determine whether the object and the second object are in contact with each other. The second model generation device 2 according to the present embodiment utilizes the determination result of the learned determination model 50 to determine the next transition target so that the first object does not come into contact with the second object. Generate an inference model 55 configured to determine task states.

<Control device>
The control device 3 according to this embodiment is a computer configured to control the operation of the manipulator 4 . Specifically, first, the control device 3 according to the present embodiment acquires first sensing data from a first sensor system that observes the tip of the manipulator 4 . Then, the control device 3 according to the present embodiment uses the first estimation model to calculate the first estimated value of the current coordinates of the hand in the observation space from the acquired first sensing data. Also, the control device 3 according to the present embodiment acquires second sensing data from a second sensor system that observes the hand of the manipulator 4 . Then, the control device 3 according to the present embodiment uses the second estimation model to calculate a second estimated value of the current coordinates of the hand in the observation space from the acquired second sensing data. Calculating each estimated value of the current coordinates of the hand corresponds to estimating the current value of the coordinates of the hand (hereinafter also referred to as "hand coordinates").

Each sensor system includes one or more sensors and is appropriately configured to observe the hand of the manipulator 4 . In this embodiment, the first sensor system comprises an encoder S2 for measuring the angle of each joint and a tactile sensor S3 for measuring the force acting on the end effector T. As shown in FIG. Measurement data (angle data, pressure distribution data) obtained by the encoder S2 and the tactile sensor S3 is an example of the first sensing data. Also, the second sensor system is configured by the camera S1. Image data obtained by the camera S1 is an example of the second sensing data. The control device 3 according to the present embodiment uses each estimation model to calculate each estimated value of the current hand coordinates from the sensing data obtained from each sensor system.

There is one true value of the hand coordinates of the manipulator 4 . If there is no noise in the calculation process from each sensor system and the parameters of each estimation model are appropriate, the first estimated value and the second estimated value will match. On the other hand, the first estimated value and the second estimated value may differ from each other due to noise generated according to each sensor system. Therefore, the control device 3 according to the present embodiment calculates the gradient of the error between the first estimated value and the second estimated value, and based on the calculated gradient, the first estimation model and the Adjust the value of at least one parameter of the second estimation model. Thereby, each estimated value calculated can be expected to approach the true value.

Based on at least one of the first estimated value and the second estimated value, the control device 3 according to the present embodiment determines a control command to be given to the manipulator 4 so that the coordinates of the hand end approach the target value. Then, the control device 3 according to the present embodiment drives the manipulator 4 by giving the determined control command to the manipulator 4 . Thereby, the control device 3 according to this embodiment controls the operation of the manipulator 4 .

Note that the method of determining the target value of the hand coordinates is not particularly limited, and may be appropriately selected according to the embodiment. In this embodiment, the inference model 55 described above can be used to determine the hand coordinate target. That is, the control device 3 according to this embodiment acquires the current task state of the manipulator 4 . As described above, the task state is defined by the positional relationship between the hand of the manipulator 4 and the target. The control device 3 according to the present embodiment uses the inference model 55 to determine the target task state to which the obtained current task state should transition next so as to approach the final target task state. . Then, the control device 3 according to the present embodiment calculates the target value of the hand coordinates from the target task state to be transitioned next. As a result, in the present embodiment, it is possible to appropriately determine the target value of the hand coordinates during the process of performing the task.

<Effect>
As described above, the control device 3 according to the present embodiment adjusts the parameters of at least one of the first estimation model and the second estimation model so that the estimation results (estimated values) of each other approach one value. This adjustment is expected to improve the accuracy of estimating the coordinates of the hand using each estimation model. Therefore, according to this configuration, it is possible to improve the accuracy of estimating the coordinates of the hand by each estimation model, thereby improving the accuracy of controlling the coordinates of the hand of the manipulator.

Further, in the conventional manipulator control method, the tasks to be performed are directly associated with time-series control commands given to the manipulator. That is, in the conventional control method, the task to be performed is directly described by a sequence of control commands. Therefore, if at least one of the environment and the object for performing the task changes even slightly, the learning result cannot correspond to the change, and there is a possibility that the task cannot be performed properly.

For example, assume that a manipulator is taught a task of holding a workpiece by an end effector. In this case, if the work is accurately positioned at the target point, the manipulator can hold the work with the end effector based on the learning result. On the other hand, if the posture of the work is different from that during learning, or if the work is arranged at a position different from that during learning, the coordinates at which the end effector holds the work change. This substantially changes the content of the task that the manipulator should perform in this scene. Therefore, the manipulator may not be able to properly hold the workpiece with the end effector in the sequence of control commands obtained from the learning results.

As described above, in the conventional control method, even if at least one of the environment and the object in which the task is performed changes even slightly, the learning result cannot cope with the change. However, there was a problem that there was a possibility that the task could not be performed appropriately. Due to this, in order to operate the manipulator for general purposes, control commands must be learned for each different state even for the same task, and the cost of teaching tasks to the manipulator is still high.

On the other hand, in this embodiment, the state of the task executed by the manipulator 4 is the relative relationship among objects such as the end effector T, the work W, and another work G. is represented by the positional relationship of Thereby, the control command given to the manipulator 4 is not directly related to the task, but is related to the amount of change in the relative positional relationship between the objects. That is, it is possible to generate or teach a time-series control command to be given to the manipulator 4 for changing the relative positional relationship of the objects without depending on the content of the task. For example, in the above example, even if the coordinates of the work change, the change in the coordinates of the work is taken into consideration when grasping the positional relationship between the end effector and the work. Therefore, the manipulator can appropriately hold the workpiece based on the learning result. Therefore, according to this embodiment, it is possible to increase the versatility of the ability to perform the learned task, thereby reducing the cost of teaching the manipulator 4 the task.

Furthermore, the first model generation device 1 according to the present embodiment generates a judgment model 50 for judging whether or not two objects come into contact with each other in the positional relationship of the objects by machine learning. According to the learned judgment model 50 generated by machine learning, even if the positional relationship of the target (relative coordinates in this embodiment) is given as continuous values, the amount of data of the judgment model 50 is greatly increased. It is possible to determine whether or not two objects are in contact with each other based on their positional relationship. Therefore, according to this embodiment, it is possible to greatly reduce the data amount of information representing the boundary where two objects come into contact.

§2 Configuration example [Hardware configuration]
<First model generation device>
Next, an example of the hardware configuration of the first model generating device 1 according to this embodiment will be described using FIG. FIG. 3 schematically illustrates an example of the hardware configuration of the first model generation device 1 according to this embodiment.

As shown in FIG. 3, the first model generation device 1 according to the present embodiment includes a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17. is a computer connected to Incidentally, in FIG. 3, the communication interface and the external interface are described as "communication I/F" and "external I/F".

The control unit 11 includes a CPU (Central Processing Unit), which is a hardware processor, RAM (Random Access Memory), ROM (Read Only Memory), etc., and is configured to execute information processing based on programs and various data. be. The storage unit 12 is an example of memory, and is configured by, for example, a hard disk drive, a solid state drive, or the like. In this embodiment, the storage unit 12 stores various information such as a model generation program 81, CAD (computer-aided design) data 120, a plurality of learning data sets 121, learning result data 125, and the like.

The model generation program 81 is a program for causing the first model generation device 1 to execute information processing ( FIG. 10 ), which will be described later, regarding machine learning of the judgment model 50 . The model generation program 81 includes a series of instructions for the information processing. The CAD data 120 includes configuration information indicating the geometric configuration of models (for example, three-dimensional models) of objects (end effector T, work W, other work G). CAD data 120 may be generated by known software. A plurality of learning data sets 121 are used for machine learning of the judgment model 50 . The learning result data 125 indicates information about the learned judgment model 50 generated by machine learning. Learning result data 125 is obtained as a result of executing the model generation program 81 . Details will be described later.

The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. By using the communication interface 13, the first model generation device 1 can perform data communication with other information processing devices (for example, the second model generation device 2 and the control device 3) via the network.

The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like, and is an interface for connecting with an external device. The type and number of external interfaces 14 may be appropriately selected according to the type and number of external devices to be connected. The first model generation device 1 may be connected to the manipulator 4 and the camera S1 via the external interface 14 in order to determine whether or not the object will come into contact in real space.

The input device 15 is, for example, a device for performing input such as a mouse and a keyboard. Also, the output device 16 is, for example, a device for outputting such as a display and a speaker. An operator can operate the first model generation device 1 by using the input device 15 and the output device 16 .

The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading programs stored in the storage medium 91 . The type of drive 17 may be appropriately selected according to the type of storage medium 91 . At least one of the model generation program 81 , the CAD data 120 , and the plurality of learning data sets 121 may be stored in this storage medium 91 .

The storage medium 91 stores information such as programs by electrical, magnetic, optical, mechanical or chemical action so that computers, other devices, machines, etc. can read information such as programs. It is a storage medium. The first model generation device 1 may acquire at least one of the model generation program 81 , the CAD data 120 , and the plurality of learning data sets 121 from the storage medium 91 .

Here, in FIG. 3, as an example of the storage medium 91, a disk-type storage medium such as a CD or DVD is illustrated. However, the type of storage medium 91 is not limited to the disc type, and may be other than the disc type. As a storage medium other than the disk type, for example, a semiconductor memory such as a flash memory can be cited.

Regarding the specific hardware configuration of the first model generation device 1, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, control unit 11 may include multiple hardware processors. The hardware processor may comprise a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), or the like. The storage unit 12 may be configured by RAM and ROM included in the control unit 11 . At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16 and the drive 17 may be omitted. The first model generation device 1 may be composed of a plurality of computers. In this case, the hardware configuration of each computer may or may not match. The first model generation device 1 may be an information processing device designed exclusively for the service provided, or may be a general-purpose server device, a PC (Personal Computer), or the like.

<Second model generation device>
Next, an example of the hardware configuration of the second model generating device 2 according to this embodiment will be described using FIG. FIG. 4 schematically illustrates an example of the hardware configuration of the second model generation device 2 according to this embodiment.

As shown in FIG. 4, the second model generating device 2 according to the present embodiment includes a control unit 21, a storage unit 22, a communication interface 23, an external interface 24, an input device 25, an output device 26, and a drive 27. is a computer connected to Incidentally, in FIG. 4, the communication interface and the external interface are described as "communication I/F" and "external I/F" as in FIG.

The control section 21 to the drive 27 of the second model generation device 2 may be configured similarly to the control section 11 to the drive 17 of the first model generation device 1, respectively. That is, the control unit 21 includes a hardware processor such as a CPU, a RAM, and a ROM, and is configured to execute various types of information processing based on programs and data. The storage unit 22 is composed of, for example, a hard disk drive, a solid state drive, or the like. The storage unit 22 stores various information such as the model generation program 82, the CAD data 220, the learning result data 125, the learning data 223, the inference model data 225, and the like.

The model generation program 82 is a program for causing the second model generation device 2 to execute information processing (FIG. 11) related to generation of the inference model 55 for inferring the target task state. Model generation program 82 includes a series of instructions for the information processing. Similar to the CAD data 120 described above, the CAD data 220 includes configuration information indicating the geometric configuration of the models of the objects (the end effector T, the work W, and the other work G). The learning result data 125 is used for setting the learned judgment model 50 . Learning data 223 is used to generate inference model 55 . Inference model data 225 indicates information about the generated inference model 55 . Inference model data 225 is obtained as a result of executing model generation program 82 . Details will be described later.

The communication interface 23 is, for example, a wired LAN module, a wireless LAN module, etc., and is an interface for performing wired or wireless communication via a network. By using the communication interface 23, the second model generation device 2 can perform data communication with other information processing devices (for example, the first model generation device 1 and the control device 3) via the network.

The external interface 24 is, for example, a USB port, a dedicated port, etc., and is an interface for connecting with an external device. The type and number of external interfaces 24 may be appropriately selected according to the type and number of external devices to be connected. The second model generation device 2 may be connected to the manipulator 4 and the camera S1 via the external interface 24 in order to reproduce the task state in real space.

The input device 25 is, for example, a device for performing input such as a mouse and a keyboard. Also, the output device 26 is, for example, a device for outputting such as a display and a speaker. The operator can operate the second model generation device 2 by using the input device 25 and the output device 26 .

The drive 27 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading programs stored in the storage medium 92 . As with the storage medium 91, the type of the storage medium 92 may be a disk type, or may be other than the disk type. At least one of the model generation program 82 , the CAD data 220 , the learning result data 125 and the learning data 223 may be stored in the storage medium 92 . Also, the second model generation device 2 may acquire at least one of the model generation program 82 , the CAD data 220 , the learning result data 125 and the learning data 223 from the storage medium 92 .

Regarding the specific hardware configuration of the second model generation device 2, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, the controller 21 may include multiple hardware processors. A hardware processor may comprise a microprocessor, FPGA, DSP, or the like. The storage unit 22 may be configured by RAM and ROM included in the control unit 21 . At least one of the communication interface 23, the external interface 24, the input device 25, the output device 26, and the drive 27 may be omitted. The second model generation device 2 may be composed of a plurality of computers. In this case, the hardware configuration of each computer may or may not match. The second model generation device 2 may be an information processing device designed exclusively for the service provided, or may be a general-purpose server device, a general-purpose PC, or the like.

<Control device>
Next, an example of the hardware configuration of the control device 3 according to this embodiment will be described using FIG. FIG. 5 schematically illustrates an example of the hardware configuration of the control device 3 according to this embodiment.

As shown in FIG. 5, in the control device 3 according to the present embodiment, a control unit 31, a storage unit 32, a communication interface 33, an external interface 34, an input device 35, an output device 36, and a drive 37 are electrically connected. computer. 5, the communication interface and the external interface are described as "communication I/F" and "external I/F" in the same way as in FIGS.

The control units 31 to 37 of the control device 3 may be configured similarly to the control units 11 to 17 of the first model generation device 1, respectively. That is, the control unit 31 includes a hardware processor such as a CPU, a RAM, and a ROM, and is configured to execute various types of information processing based on programs and data. The storage unit 32 is composed of, for example, a hard disk drive, a solid state drive, or the like. The storage unit 32 stores various information such as the control program 83, CAD data 320, robot data 321, inference model data 225, and the like.

The control program 83 is a program for causing the control device 3 to execute information processing (FIGS. 16A, 16B, 20, and 21) regarding the control of the operation of the manipulator 4, which will be described later. The control program 83 includes a series of instructions for the information processing. Similar to the CAD data 120 described above, the CAD data 320 includes configuration information indicating the geometric configuration of the model of each object (end effector T, work W, other work G). The robot data 321 includes configuration information indicating the configuration of the manipulator 4 such as parameters of each joint. The inference model data 225 is used to set the generated inference model 55 . Details will be described later.

The communication interface 33 is, for example, a wired LAN module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via a network. By using the communication interface 33, the control device 3 can perform data communication with other information processing devices (for example, the first model generation device 1 and the second model generation device 2) via the network.

The external interface 34 is, for example, a USB port, a dedicated port, etc., and is an interface for connecting with an external device. The type and number of external interfaces 34 may be appropriately selected according to the type and number of external devices to be connected. The control device 3 may be connected with the camera S1 and the manipulator 4 via the external interface 34 . In this embodiment, the manipulator 4 includes an encoder S2 that measures the angle of each joint, and a tactile sensor S3 that measures the force acting on the end effector T. FIG.

The types of the camera S1, the encoder S2, and the tactile sensor S3 may not be particularly limited, and may be appropriately determined according to the embodiment. The camera S1 is, for example, a general digital camera configured to acquire RGB images, a depth camera configured to acquire depth images, an infrared camera configured to image infrared quantities, or the like. you can The tactile sensor S3 may be, for example, a tactile sensor or the like.

The control device 3 can acquire sensing data from each sensor (camera S1, each encoder S2, tactile sensor S3) via the external interface . Note that the method of connecting the camera S1 and the manipulator 4 need not be limited to such an example. For example, if the camera S1 and the manipulator 4 have communication interfaces, the control device 3 may be connected to the camera S1 and the manipulator 4 via the communication interface 33 .

The input device 35 is, for example, a device for performing input such as a mouse and a keyboard. Also, the output device 36 is, for example, a device for outputting such as a display and a speaker. An operator can operate the control device 3 by using the input device 35 and the output device 36 .

The drive 37 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading programs stored in the storage medium 93 . As with the storage medium 91, the type of the storage medium 93 may be a disk type, or may be other than the disk type. At least one of the control program 83 , the CAD data 320 , the robot data 321 and the inference model data 225 may be stored in the storage medium 93 . Also, the control device 3 may acquire at least one of the control program 83 , the CAD data 320 , the robot data 321 and the inference model data 225 from the storage medium 93 .

Regarding the specific hardware configuration of the control device 3, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, the controller 31 may include multiple hardware processors. A hardware processor may comprise a microprocessor, FPGA, DSP, or the like. The storage unit 32 may be configured by RAM and ROM included in the control unit 31 . At least one of the communication interface 33, the external interface 34, the input device 35, the output device 36, and the drive 37 may be omitted. The control device 3 may be composed of a plurality of computers. In this case, the hardware configuration of each computer may or may not match. The control device 3 may be an information processing device designed exclusively for the service provided, or may be a general-purpose server device, a general-purpose PC, a PLC (programmable logic controller), or the like.

<Manipulator>
Next, an example of the hardware configuration of the manipulator 4 according to this embodiment will be described using FIG. FIG. 6 schematically illustrates an example of the hardware configuration of the manipulator 4 according to this embodiment.

The manipulator 4 according to this embodiment is a 6-axis vertical articulated industrial robot, and includes a pedestal 40 and six joints 41 to 46 . Each of the joints 41 to 46 incorporates a servomotor (not shown) and is configured to be rotatable about each axis. The first joint portion 41 is connected to the pedestal portion 40 and rotates the portion on the distal end side around the axis of the pedestal. The second joint portion 42 is connected to the first joint portion 41 and rotates the distal end side portion in the front-rear direction. The third joint portion 43 is connected to the second joint portion 42 via a link 491, and rotates the portion on the distal end side in the vertical direction. The fourth joint portion 44 is connected to the third joint portion 43 via a link 492 and rotates the tip side portion around the axis of the link 492 . The fifth joint portion 45 is connected to the fourth joint portion 44 via a link 493, and rotates the portion on the distal end side in the vertical direction. The sixth joint portion 46 is connected to the fifth joint portion 45 via a link 494 and rotates the tip side portion around the axis of the link 494 . An end effector T is attached to the distal end side of the sixth joint portion 46 together with the tactile sensor S3.

Each of the joints 41-46 further incorporates an encoder S2. Each encoder S2 is configured to measure the angle (control amount) of each joint 41-46. The measurement data (angle data) of each encoder S2 can be used to control the angles of the joints 41-46. Also, the tactile sensor S3 is configured to detect a force acting on the end effector T. As shown in FIG. The measurement data (pressure distribution data) of the tactile sensor S3 is used to estimate the position and orientation of the workpiece W held by the end effector T, and to detect whether or not an abnormal force is acting on the end effector T. may be used for

Note that the hardware configuration of the manipulator 4 may not be limited to such an example. Regarding the specific hardware configuration of the manipulator 4, it is possible to omit, replace, and add components as appropriate according to the embodiment. For example, the manipulator 4 may have sensors other than the encoder S2 and the tactile sensor S3 to observe control variables or other attributes. For example, manipulator 4 may further comprise a torque sensor. In this case, the manipulator 4 may measure the force acting on the end effector T with a torque sensor, and may be controlled so that excessive force does not act on the end effector T based on the measured value of the torque sensor. Also, the number of axes of the manipulator 4 may not be limited to six. A known industrial robot may be adopted as the manipulator 4 .

[Software configuration]
<First model generation device>
Next, an example of the software configuration of the first model generation device 1 according to this embodiment will be described with reference to FIG. FIG. 7 schematically illustrates an example of the software configuration of the first model generation device 1 according to this embodiment.

The control unit 11 of the first model generation device 1 develops the model generation program 81 stored in the storage unit 12 in RAM. Then, the control unit 11 causes the CPU to interpret and execute instructions included in the model generation program 81 developed in the RAM, and controls each component. Thus, as shown in FIG. 7, the first model generation device 1 according to the present embodiment operates as a computer having a data acquisition unit 111, a machine learning unit 112, and a storage processing unit 113 as software modules. That is, in this embodiment, each software module of the first model generation device 1 is implemented by the control unit 11 (CPU).

The data acquisition unit 111 acquires multiple learning data sets 121 . Each learning data set 121 is composed of a combination of training data 122 indicating the positional relationship between two objects and correct data 123 indicating whether or not the two objects contact each other in the positional relationship. The training data 122 is used as input data for machine learning. The correct data 123 is used as a teacher signal (label) for machine learning. The formats of the training data 122 and the correct answer data 123 are not particularly limited and may be appropriately selected according to the embodiment. For example, the training data 122 may use the relative coordinates between the two objects as they are, or may use values obtained by converting the relative coordinates into feature quantities. The CAD data 120 can determine whether two objects of interest touch each other in their positional relationship. Therefore, each learning data set 121 can be generated by using the CAD data 120 .

The machine learning unit 112 performs machine learning of the judgment model 50 using the acquired plurality of learning data sets 121 . Performing machine learning consists of training the decision model 50 to output an output value that matches the corresponding correct data 123 for each training data set 121 given the training data 122 input. . Through this machine learning, it is possible to build a learned decision model 50 that has acquired the ability to decide whether two objects are in contact with each other. The storage processing unit 113 generates information about the built learned determination model 50 as learning result data 125, and stores the generated learning result data 125 in a predetermined storage area.

(Configuration of decision model)
Next, an example of the configuration of the judgment model 50 will be described. The judgment model 50 according to the present embodiment is composed of a multi-layered neural network used for deep learning. In the example of FIG. 7, the judgment model 50 is composed of a fully-connected neural network with a three-layer structure. The decision model 50 comprises an input layer 501 , an intermediate (hidden) layer 502 and an output layer 503 . However, the structure of the judgment model 50 need not be limited to such an example, and may be appropriately determined according to the embodiment. For example, the number of intermediate layers included in the judgment model 50 may not be limited to one, and may be two or more. Alternatively, intermediate layer 502 may be omitted.

The number of neurons (nodes) included in each layer 501-503 may be determined as appropriate according to the embodiment. For example, the number of neurons in the input layer 501 may be determined according to the number of dimensions of relative coordinates expressing the positional relationship between two objects. Also, the number of neurons in the output layer 503 may be determined according to how to represent whether two objects touch each other. For example, when expressing with one numerical value whether or not two objects are in contact with each other (for example, expressing with a numerical value in the range of [0, 1]), the number of neurons in the output layer 503 is one. you can Further, for example, when expressing whether or not two objects are in contact with each other by using two numerical values, a first numerical value indicating the probability of contact and a second numerical value indicating the probability of non-contact, the output layer 503 The number of neurons may be two.

Neurons in adjacent layers are connected appropriately. In this embodiment, each neuron is connected to all neurons in adjacent layers. However, the connection relationship of each neuron need not be limited to such an example, and may be appropriately set according to the embodiment. A weight (connection weight) is set for each connection. A threshold is set for each neuron, and basically the output of each neuron is determined depending on whether or not the sum of products of each input and each weight exceeds the threshold. The threshold may be expressed by an activation function. In this case, the output of each neuron is determined by inputting the sum of the product of each input and each weight into the activation function and executing the operation of the activation function. The type of activation function need not be particularly limited, and may be appropriately selected according to the embodiment. The weight of the connection between neurons and the threshold value of each neuron included in each layer 501 to 503 are examples of calculation parameters of the judgment model 50 .

In this embodiment, the machine learning unit 112 uses a plurality of learning data sets 121 to carry out machine learning of the judgment model 50 configured by the neural network. Specifically, the machine learning unit 112 adjusts the values of the calculation parameters of the judgment model 50, so that when the training data 122 is input to the input layer 501 for each learning data set 121, the output value matching the correct data 123 The operation parameters of the decision model 50 are trained so that is output from the output layer 503 . Accordingly, the machine learning unit 112 can generate the learned determination model 50 that has acquired the ability to determine whether two objects are in contact with each other.

The storage processing unit 113 generates, as the learning result data 125 , information indicating the structure and calculation parameters of the built learned determination model 50 . Then, the save processing unit 113 saves the generated learning result data 125 in a predetermined storage area. Note that the content of the learning result data 125 need not be limited to such an example as long as the learned determination model 50 can be reproduced. For example, when the structure of the judgment model 50 is shared among the devices, the information indicating the structure of the judgment model 50 may be omitted from the learning result data 125 .

<Second model generation device>
Next, an example of the software configuration of the second model generation device 2 according to this embodiment will be described using FIG. FIG. 8 schematically illustrates an example of the software configuration of the second model generation device 2 according to this embodiment.

The control unit 21 of the second model generation device 2 expands the model generation program 82 stored in the storage unit 22 to RAM. Then, the control unit 21 causes the CPU to interpret and execute instructions included in the model generation program 82 developed in the RAM, and controls each component. As a result, as shown in FIG. 8, the second model generation device 2 according to the present embodiment is a computer having a contact determination unit 211, a data collection unit 212, a model generation unit 213, and a storage processing unit 214 as software modules. Operate. That is, in the present embodiment, each software module of the second model generation device 2 is implemented by the control unit 21 (CPU), similarly to the first model generation device 1 described above.

The contact determination unit 211 has a learned determination model 50 by holding the learning result data 125 . The contact determination unit 211 refers to the learning result data 125 and sets the learned determination model 50 . The learned determination model 50 has acquired the ability to determine whether or not the first object and the second object are in contact with each other through the above machine learning. The contact determination unit 211 provides the learned determination model 50 with information indicating the task states of the first object and the second object, so that the first object and the second object are in the target task state. Determine whether or not they are in contact with each other.

The data collection unit 212 and the model generation unit 213 use the judgment result of the learned judgment model 50 to prevent the first object from coming into contact with the second object, so that the final target task state and the current task Generating an inference model 55 configured to determine the next target task state to transition from state to. That is, the data collection unit 212 collects the learning data 223 used to generate the inference model 55 using the judgment result of the learned judgment model 50 . The CAD data 220 may be further used to collect the learning data 223 . The model generator 213 uses the collected learning data 223 to generate the inference model 55 . Details of the learning data 223 and the inference model 55 will be described later. The storage processing unit 214 generates information about the generated inference model 55 as the inference model data 225, and saves the generated inference model data 225 in a predetermined storage area.

<Control device>
Next, an example of the software configuration of the control device 3 according to this embodiment will be described using FIG. FIG. 9 schematically illustrates an example of the software configuration of the control device 3 according to this embodiment.

The control unit 31 of the control device 3 develops the control program 83 stored in the storage unit 32 in RAM. Then, the control unit 31 causes the CPU to interpret and execute instructions included in the control program 83 developed in the RAM to control each component. Thereby, as shown in FIG. 9, the control device 3 according to the present embodiment includes a target setting unit 310, a first data acquisition unit 311, a second data acquisition unit 312, a first estimation unit 313, a second estimation unit 314 , a state acquisition unit 315, an action determination unit 316, a command determination unit 317, a drive unit 318, and an adjustment unit 319 as software modules. That is, in the present embodiment, each software module of the control device 3 is implemented by the control unit 31 (CPU) as in the first model generation device 1 described above.

The goal setting unit 310 sets the final goal task state according to the task to be performed. In this embodiment, the task state is defined by the positional relationship between the first object and the second object in the task to be performed, more specifically between the hand of the manipulator 4 and the target. In this embodiment, the positional relationship is represented by the relative coordinates. The "final goal" is the end point (goal) and is achieved when the task is completed.

The target may be appropriately set according to the task to be performed. As an example, the target may be the work W when the manipulator 4 (end effector T) does not hold the work W. On the other hand, when the manipulator 4 (end effector T) holds the work W, the target may be an object to which the work W is attached (in this embodiment, another work G).

The first data acquisition unit 311 acquires the first sensing data 323 from the first sensor system that observes the hand of the manipulator 4 . In this embodiment, the first data acquisition unit 311 acquires measurement data from each encoder S2 and tactile sensor S3 as the first sensing data 323 . The second data acquisition unit 312 acquires the second sensing data 324 from the second sensor system that observes the hand of the manipulator 4 . In this embodiment, the second data acquisition unit 312 acquires image data from the camera S1 as the second sensing data 324 .

The first estimation unit 313 uses the first estimation model 61 to calculate a first estimated value of the current coordinates of the hand in the observation space from the acquired first sensing data 323 . The second estimation unit 314 uses the second estimation model 62 to calculate a second estimated value of the current coordinates of the hand in the observation space from the acquired second sensing data 324 . The adjusting unit 319 calculates the gradient of the error between the first estimated value and the second estimated value, and adjusts the first estimated model 61 and the second estimated model 62 based on the calculated gradient so that the error becomes small. Adjust the value of at least one parameter.

The state acquisition unit 315 acquires information indicating the current task state of the manipulator 4 . “Current” is the point in time when the operation of the manipulator 4 is controlled, and is the point in time immediately before determining the control command to be given to the manipulator 4 .

The action determination unit 316 determines the target task state to which the current task state indicated by the acquired information transitions next so as to approach the final target task state. A "goal" includes an end goal and may be set as appropriate to accomplish the task. The number of goals set up to the final goal may be one (in this case, only the final goal is set) or plural. Goals other than the final goal are waypoints through which the task is routed from the start point to the end point. Therefore, the final goal may simply be called a "goal", and the goals other than the final goal may be called a "sub-goal". A sub-goal may be referred to as a "waypoint." "Goal to next transition" is the next target task state from the current task state (provisional task state if the goal is other than the final goal). The closest target.

In this embodiment, the action determination unit 316 has an inference model 55 generated by holding the inference model data 225 . The action determination unit 316 uses the generated inference model 55 to determine the final target task state and the target task state to which the current task state transitions next.

The command determination unit 317 determines a control command to be given to the manipulator 4 based on at least one of the first estimated value and the second estimated value so that the coordinates of the hand end approach the target value. The driving unit 318 drives the manipulator 4 by giving the determined control command to the manipulator 4 .

In this embodiment, the control command is composed of command values for each joint. The command determining unit 317 determines the current values of the hand coordinates of the manipulator 4 based on at least one of the first estimated value and the second estimated value. In addition, the command determination unit 317 calculates the target value of the hand coordinates from the determined next transition target task state. Next, the command determination unit 317 calculates the amount of change in the angle of each joint from the difference between the current hand coordinate value and the target value. Then, the command determination unit 317 determines a command value for each joint based on the calculated amount of change in the angle of each joint. The driving unit 318 drives each joint according to the determined command value. Through these processes, the control device 3 according to this embodiment controls the operation of the manipulator 4 .

<Others>
Each software module of the first model generation device 1, the second model generation device 2, and the control device 3 will be described in detail in operation examples described later. In this embodiment, an example in which each software module of the first model generation device 1, the second model generation device 2, and the control device 3 is realized by a general-purpose CPU is described. However, some or all of the above software modules may be implemented by one or more dedicated processors. Further, regarding the software configurations of the first model generation device 1, the second model generation device 2, and the control device 3, omission, replacement, and addition of software modules may be performed as appropriate according to the embodiment.

§3 Operation example [First model generation device]
Next, an operation example of the first model generation device 1 will be described with reference to FIG. FIG. 10 is a flowchart showing an example of a processing procedure relating to machine learning of the judgment model 50 by the first model generation device 1 according to this embodiment. The processing procedure described below is an example of a model generation method for generating the judgment model 50 . However, each processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, for each processing procedure described below, it is possible to omit, replace, or add steps as appropriate according to the embodiment.

(Step S101)
In step S<b>101 , the control unit 11 operates as the data acquisition unit 111 and acquires a plurality of learning data sets 121 to be used for machine learning of the judgment model 50 . Each learning data set 121 is composed of a combination of training data 122 indicating the positional relationship between two objects and correct data 123 indicating whether or not the two objects contact each other in the positional relationship.

A method for generating each learning data set 121 may not be particularly limited, and may be appropriately selected according to the embodiment. For example, using the CAD data 120, two objects are arranged in various positional relationships in the virtual space. In this embodiment, the positional relationship is represented by relative coordinates. Moreover, in the present embodiment, at least one of the two objects is an object to be moved by the operation of the manipulator 4 . When assuming a scene where the first task is performed, the end effector T and the workpiece W are examples of respective objects. Further, when assuming a scene in which the second task is performed, the work W held by the end effector T and another work G are examples of respective objects. One of the two objects is the hand of the manipulator 4 and the other is the target. The placement of each object may be specified by the operator or determined at random. Alternatively, various positional relationships may be realized by fixing the position of one object and changing the position of the other object according to a rule. A rule for giving the placement of the other object may be set as appropriate. Thereby, relative coordinates in each positional relationship can be acquired as training data 122 of each learning data set 121 . The CAD data 120 also includes a model of each object. As such, the CAD data 120 can determine whether two objects of interest touch each other in the positional relationship of the objects. Using the CAD data 120 , the result of determining whether or not two objects contact each other in each positional relationship is associated with the corresponding training data 122 as correct data 123 . Thereby, each learning data set 121 can be generated. Note that the method of generating each learning data set 121 need not be limited to such an example. Each learning data set 121 may be generated by using the actual object of each object in the real space.

Each learning data set 121 may be automatically generated by computer operations, or may be manually generated by including, at least in part, operator manipulations. Also, the generation of each learning data set 121 may be performed by the first model generation device 1 or may be performed by a computer other than the first model generation device 1 . When the first model generation device 1 generates each learning data set 121, the control unit 11 executes the above series of processes automatically or manually by an operator's operation via the input device 15 to generate a plurality of Acquire the learning data set 121 . On the other hand, when each learning data set 121 is generated by another computer, the control unit 11 acquires a plurality of learning data sets 121 generated by the other computer, for example, via a network, a storage medium 91, or the like. In this case, the CAD data 120 may be omitted from the first model generation device 1 . Some learning data sets 121 may be generated by the first model generation device 1 and other learning data sets 121 may be generated by one or more other computers.

The number of acquired learning data sets 121 may not be particularly limited, and may be appropriately selected according to the embodiment. After acquiring the plurality of learning data sets 121, the control unit 11 proceeds to the next step S102.

(Step S102)
In step S<b>102 , the control unit 11 operates as the machine learning unit 112 and performs machine learning of the judgment model 50 using the acquired plurality of learning data sets 121 . In this embodiment, the control unit 11 uses machine learning to input the training data 122 for each learning data set 121 into the input layer 501, and the output value matching the corresponding correct data 123 is output from the output layer 503. The decision model 50 is trained as follows. As a result, the control unit 11 constructs a learned determination model 50 that has learned the ability to determine whether or not two objects come into contact with each other in the positional relationship of the objects.

A processing procedure of machine learning may be appropriately determined according to the embodiment. As an example, the control unit 11 first prepares the determination model 50 to be processed. The structure of the judgment model 50 to be prepared (for example, the number of layers, the number of neurons included in each layer, the connection relationship between neurons in adjacent layers, etc.), the initial value of the weight of the connection between neurons, and the threshold value of each neuron may be given by a template or may be given by an operator's input. Further, when performing re-learning, the control unit 11 may prepare the determination model 50 based on learning result data obtained by performing past machine learning.

Next, the control unit 11 uses the training data 122 included in each learning data set 121 as input data and the correct data 123 as a teacher signal to execute the learning process of the judgment model 50 (neural network). . Batch gradient descent, stochastic gradient descent, mini-batch gradient descent, or the like may be used for this learning process.

For example, in the first step, the control unit 11 inputs the training data 122 to the judgment model 50 for each learning data set 121 and executes the arithmetic processing of the judgment model 50 . That is, the control unit 11 inputs the training data 122 to the input layer 501, and determines firing of each neuron included in each layer 501 to 503 in order from the input side (that is, performs computation of forward propagation). Through this arithmetic processing, the control unit 11 obtains from the output layer 503 of the determination model 50 an output value corresponding to the result of determining whether or not two objects come into contact with each other in the positional relationship indicated by the training data 122. .

In the second step, the control unit 11 calculates the error (loss) between the output value acquired from the output layer 503 and the correct data 123 based on the loss function. The loss function is a function that evaluates the difference (that is, the degree of difference) between the output of the learning model and the correct answer. The value of the error calculated by is large. The type of loss function used to calculate the error need not be particularly limited, and may be appropriately selected according to the embodiment.

In the third step, the control unit 11 uses the gradient of the error of the output value calculated by the back propagation method to calculate each operation parameter of the decision model 50 (the weight of the connection between each neuron, Calculate the error in the value of each neuron (threshold, etc.). In a fourth step, the controller 11 updates the values of the calculation parameters of the judgment model 50 based on each calculated error. The extent to which the values of the computational parameters are updated may be adjusted by the learning rate. The learning rate may be specified by the operator or may be given as a set value within the program.

By repeating the first to fourth steps, the control unit 11 determines that the sum of the error between the output value output from the output layer 503 and the correct data 123 is small for each learning data set 121. Adjust the values of the calculation parameters of the model 50 . For example, the control unit 11 may repeat the processes of the first to fourth steps until the sum of the errors becomes equal to or less than the threshold. The threshold may be appropriately set according to the embodiment. Based on the result of this machine learning, the control unit 11 trains to output an output value matching the corresponding correct data 123 from the output layer 503 when the training data 122 is input to the input layer 501 for each learning data set 121. A trained decision model 50 can be constructed. This "matching" may include that an acceptable difference, such as by a threshold, occurs between the output value of the output layer 503 and the teacher signal (correct data 123). When the machine learning of the judgment model 50 is completed, the control unit 11 advances the process to the next step S103.

(Step S103)
In step S<b>103 , the control unit 11 operates as the storage processing unit 113 and stores information about the learned judgment model 50 constructed by machine learning as the learning result data 125 in a predetermined storage area. In this embodiment, the control unit 11 generates, as the learning result data 125, information indicating the structure and calculation parameters of the learned judgment model 50 constructed in step S102. Then, the control unit 11 saves the generated learning result data 125 in a predetermined storage area.

The predetermined storage area may be, for example, the RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination thereof. The storage medium may be, for example, a CD, DVD, or the like, and the control section 11 may store the learning result data 125 in the storage medium via the drive 17 . The external storage device may be, for example, a data server such as NAS (Network Attached Storage). In this case, the control unit 11 may use the communication interface 13 to store the learning result data 125 in the data server via the network. Also, the external storage device may be, for example, an external storage device connected to the first model generation device 1 .

Thus, when the storage of the learning result data 125 is completed, the control unit 11 terminates a series of processes regarding generation of the learned determination model 50 .

Note that the generated learning result data 125 may be provided to the second model generation device 2 at any timing. For example, the control unit 11 may transfer the learning result data 125 to the second model generation device 2 as the process of step S103 or separately from the process of step S103. The second model generation device 2 may acquire the learning result data 125 by receiving this transfer. Further, for example, the second model generation device 2 may acquire the learning result data 125 by accessing the first model generation device 1 or the data server via the network using the communication interface 23 . Also, for example, the second model generation device 2 may acquire the learning result data 125 via the storage medium 92 . Also, for example, the learning result data 125 may be incorporated in the second model generation device 2 in advance.

Furthermore, the control unit 11 may update or newly generate the learning result data 125 by repeating the processing of steps S101 to S103 on a regular or irregular basis. During this repetition, at least part of the plurality of learning data sets 121 may be changed, corrected, added, deleted, etc., as appropriate. Then, the control unit 11 provides the updated or newly generated learning result data 125 to the second model generation device 2 each time the learning process is executed, thereby allowing the learning result data 125 held by the second model generation device 2 to may be updated.

[Second model generation device]
Next, an operation example regarding generation of the inference model 55 by the second model generation device 2 will be described with reference to FIG. FIG. 11 is a flowchart showing an example of a processing procedure for generating the inference model 55 by the second model generation device 2 according to this embodiment. Each processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, for each processing procedure described below, it is possible to omit, replace, or add steps as appropriate according to the embodiment.

(Step S201)
In step S<b>201 , the control unit 21 receives designation of the final target task state for the task to be performed by the manipulator 4 . The task state is represented by the positional relationship between the first object and the second object, more specifically, the hand of the manipulator 4 and the target. In this embodiment, the positional relationship is represented by relative coordinates.

The method of specifying the relative coordinates in the final task state is not particularly limited, and may be appropriately selected according to the embodiment. For example, relative coordinates in the final task state may be specified directly by operator input via input device 25 . Also, for example, a task to be performed may be selected by an operator's input, and relative coordinates in the final task state may be designated according to the selected task. Also, for example, by using the CAD data 220 and arranging the models of the respective objects in the virtual space in the positional relationship of the final target, the relative coordinates in the final target may be specified. The placement of the model of each object may be automatically performed by the simulator, or may be manually performed by an operator's input. When the final target task state is specified, the control unit 21 advances the process to the next step S202.

(Steps S202 to S204)
At step S202, the control unit 21 sets an arbitrary task state as a starting point. The task state set at the starting point corresponds to the task state at the time the task starts to be performed. The task state that serves as the starting point may be set randomly or may be designated by operator input. The method of specifying the starting point by the operator may be the same as the method of specifying the final goal. Also, the starting task state may be determined by any algorithm. As an example, the actual object of each object may be arranged in real space, and image data of each object may be obtained by photographing each object with a camera. Then, by performing image processing (for example, matching using the CAD data 220) on the obtained image data, a task state that serves as a starting point may be determined. In addition, the task state that serves as the starting point may be appropriately determined using the CAD data 220 .

In step S203, the control unit 21 operates as the contact determination unit 211 and uses the learned determination model 50 to determine whether or not the two objects are in contact with each other in the task state set as the starting point. do. Specifically, the control unit 21 refers to the learning result data 125 and sets the learned determination model 50 . Subsequently, the control unit 21 inputs the relative coordinates of the task state set in step S202 to the input layer 501 of the learned judgment model 50 . Then, the control unit 21 determines firing of each neuron included in each layer 501 to 503 in order from the input side as the arithmetic processing of the learned determination model 50 . As a result, the control unit 21 acquires from the output layer 503 of the learned determination model 50 the output value corresponding to the result of determining whether or not the two objects are in contact with each other in the task state set as the starting point. do.

In step S204, the control unit 21 determines the branch destination of the process based on the determination result of step S203. If it is determined in step S203 that the two objects come into contact with each other in the task state set as the start point, the control unit 21 returns the process to step S202 and sets the task state as the start point again. On the other hand, when it is determined that the two objects do not contact each other in the task state set as the starting point, the control unit 21 recognizes the set starting point task state as the current task state of the manipulator 4, The process proceeds to the next step S205.

FIG. 12A schematically illustrates an example of a scene in which the task states of the starting point and the final goal are set in the task space SP by the processing of steps S201 to S204. The task space SP represents a set of relative coordinates that define task states. Information indicating the task space SP may or may not be held in the second model generation device 2 . Each node (point) belonging to the task space SP corresponds to a relative coordinate between two objects. In the example of FIG. 12A, the node Ns corresponds to the relative coordinates in the starting task state, and the node Ng corresponds to the relative coordinates in the final goal task state. In this embodiment, a boundary surface (contact boundary surface) indicating whether or not two objects in the task space SP are in contact is derived based on the determination result of the learned determination model 50 .

(Step S205 to Step S207)
In step S205, the control unit 21 determines the next target task state to transition from the current task state so as to approach the final target task state.

A method for determining the target task state may not be particularly limited, and may be appropriately selected according to the embodiment. For example, the relative coordinates in the target task state may be determined by operator input. Similar to the setting of the starting point task state, the relative coordinates in the target task state may be determined by any algorithm, or may be determined as appropriate using the CAD data 220 . Further, for example, the control unit 21 may determine the relative coordinates in the target task state by randomly changing the relative coordinates in the task state of the starting point. Further, for example, the control unit 21 may select a node a predetermined distance away from the node Ns so as to approach the node Ng within the task space SP. The control unit 21 may acquire the task state corresponding to the selected node as the target task state. Further, for example, when the inference model 55 is generated by reinforcement learning, which will be described later, the target task state may be determined using the inference model 55 in the process of reinforcement learning.

Also, for example, a known method such as path planning may be employed to determine the target task state. As an example, the control unit 21 may set nodes that are candidates for the target task state in the task space SP. The node setting may be performed automatically by a method such as random sampling, or may be manually performed by an operator's input. The configuration of some nodes may be done automatically and the configuration of the remaining nodes may be done manually. After the nodes that are candidates for the target task state are set, the control unit 21 may appropriately select a combination of nodes to which transition is possible. For example, a nearest neighbor method or the like may be adopted as a method for selecting combinations of nodes that can be transitioned. Within the task space SP, combinations of transitionable nodes may be represented by edges connecting the nodes. Next, the control unit 21 searches for a route from the starting point node Ns to the final target node Ng. Dijkstra's method or the like may be adopted as a route search method. The control unit 21 may acquire the task state corresponding to the node included in the route obtained by the search as the target task state.

In step S206, the control unit 21 operates as the contact determination unit 211 and uses the learned determination model 50 to determine whether or not the two objects are in contact with each other in the determined target task state. . The control unit 21 may execute the process of step S206 in the same manner as in step S203 above, except that the task state of the starting point is replaced with the target task state. That is, the control unit 21 inputs the relative coordinates of the target task state to the learned judgment model 50 and executes the arithmetic processing of the learned judgment model 50 . Thereby, the control unit 21 acquires from the learned determination model 50 an output value corresponding to the result of determining whether or not the two objects are in contact with each other in the target task state.

In step S207, the control unit 21 determines the branch destination of the process based on the determination result of step S206. If it is determined in step S207 that the two objects are in contact with each other in the target task state, the control unit 21 returns the process to step S205 to determine the target task state again. On the other hand, when it is determined that the two objects do not contact each other in the target task state, the control unit 21 advances the processing to the next step S208.

Note that the branch destination in step S207 does not have to be limited to such an example. For example, if it is determined that the two objects are in contact with each other in the target task state, the control unit 21 may return the process to step S202 and redo the process from setting the starting point. Further, for example, after determining the target task state a plurality of times, if it is determined that the two objects come into contact with each other in the last determined target task state, the control unit 21 returns the process to step S205, The determination of the target task state to which the next transition is to be made may be performed again from the starting point. A sequence of target task states determined up to contact may be collected as failure cases in which the final target task state is unreachable.

FIG. 12B schematically illustrates an example of a scene in which the target task state is determined in the task space SP by the processes of steps S205 to S207. In the example of FIG. 12B, the node N1 corresponds to the relative coordinates in the task state determined as the target task state to transition to after the starting task state (node Ns). In the example of FIG. 12B, it is assumed that the target task state for one transition is determined in step S205. However, the number of target task states determined in step S205 need not be limited to one. In step S205, the control unit 21 may determine a target task state (target task state sequence) for a plurality of transitions toward the final target task state.

(Step S208)
In step S208, the control unit 21 transitions the current task state of the manipulator 4 to the target task state determined in step S205. Then, the control unit 21 determines whether or not the task state of the manipulator 4 has reached the final target task state, that is, whether or not the transition destination task state is the final target task state. A task state transition may be performed in a virtual space by simulation. When determining that the final target task state has been reached, the control unit 21 advances the processing to the next step S209. On the other hand, if it is determined that the final target task state has not been reached, the control unit 21 returns the process to step S205 to determine a further target task state.

FIG. 12C schematically illustrates an example of a scene in which a transition task state sequence from the starting point task state to the final target task state is determined by the processing up to step S208 in the task space SP. Each of the nodes N1 to N4 corresponds to the relative coordinates in the task state determined as the target task state from the starting point node Ns to the final target node Ng. Node N(k+1) indicates the target task state to which node N(k) transitions (k is 1 to 3). As illustrated in FIG. 12C, the processing up to step S208 allows the control unit 21 to obtain a series of target task states transitioning from the starting point to the final target.

(Step S209)
In step S209, the control unit 21 determines whether or not to repeat the processes of steps S202 to S208. A criterion for repeating the process may be appropriately determined according to the embodiment.

For example, a specified number of times to repeat the process may be set. The specified number of times may be given by a set value or may be given by an operator's designation, for example. In this case, the control unit 21 determines whether or not the number of times the processes of steps S202 to S208 have been executed has reached a specified number of times. When determining that the number of times of execution has not reached the specified number of times, the control unit 21 returns the process to step S202 and repeats the processes of steps S202 to S208. On the other hand, when determining that the number of times of execution has reached the specified number of times, the control unit 21 advances the process to the next step S210.

Also, for example, the control unit 21 may inquire of the operator whether to repeat the process. In this case, the control unit 21 determines whether or not to repeat the processing of steps S202 to S208 according to the operator's answer. When the operator replies that the process will be repeated, the control section 21 returns the process to step S202 and repeats the processes of steps S202 to S208. On the other hand, if the operator answers not to repeat the process, the control unit 21 advances the process to the next step S210.

Through the processing up to step S209, one or more series of target task states transitioning from the start point to the final target illustrated in FIG. 12C can be obtained. The control unit 21 operates as the data collection unit 212 and collects one or more series of target task states transitioning from this starting point to the final goal. Then, the control unit 21 generates learning data 223 from the collected series. The control unit 21 may acquire the collected series as the learning data 223 as they are, or may generate the learning data 223 by executing some information processing on the collected series. The configuration of the learning data 223 may be appropriately determined according to the method of generating the inference model 55 . The configuration of the learning data 223 will be described later.

(Step S210 and Step S211)
At step S<b>210 , the control unit 21 operates as the model generation unit 213 . That is, the control unit 21 uses the learning data 223 obtained by using the determination result of the learned determination model 50 to prevent the first object from coming into contact with the second object. An inference model 55 is generated for inferring the next transition target task state from the state and the final target task state. A method for generating the inference model 55 will be described later.

At step S<b>211 , the control unit 21 operates as the saving processing unit 214 . That is, the control unit 21 generates information about the generated inference model 55 as the inference model data 225, and stores the generated inference model data 225 in a predetermined storage area. The predetermined storage area may be, for example, the RAM in the control section 21, the storage section 22, an external storage device, a storage medium, or a combination thereof. The storage medium may be a CD, DVD, or the like, for example, and the control unit 21 may store the inference model data 225 in the storage medium via the drive 27 . The external storage device may be, for example, a data server such as NAS. In this case, the control unit 21 may use the communication interface 23 to store the inference model data 225 in the data server via the network. Also, the external storage device may be, for example, an external storage device connected to the second model generation device 2 .

Thus, when the storage of the inference model data 225 is completed, the control unit 21 terminates a series of processes regarding generation of the inference model 55 .

Note that the generated inference model data 225 may be provided to the control device 3 at any timing. For example, the control unit 21 may transfer the inference model data 225 to the control device 3 as the process of step S211 or separately from the process of step S211. The control device 3 may acquire the inference model data 225 by receiving this transfer. Further, for example, the control device 3 may acquire the inference model data 225 by accessing the second model generation device 2 or the data server via the network using the communication interface 33 . Also, for example, the control device 3 may acquire the inference model data 225 via the storage medium 93 . Also, for example, the inference model data 225 may be pre-installed in the control device 3 .

Furthermore, the control unit 21 may update or newly generate the inference model data 225 by periodically or irregularly repeating the processing of steps S201 to S211. During this repetition, at least part of the learning data 223 may be changed, corrected, added, deleted, etc., as appropriate. Then, the control unit 21 may update the inference model data 225 held by the control device 3 by providing the updated or newly generated inference model data 225 to the control device 3 each time the learning process is executed. .

<Inference model generation method>
Next, a specific example of the method of generating the inference model 55 in step S210 will be described. In this embodiment, the control unit 21 can generate the inference model 55 by at least one of the following two methods.

(1) First Method In the first method, the control unit 21 generates the inference model 55 by performing machine learning. In this case, the inference model 55 is composed of a machine learning model. The type of machine learning model is not particularly limited, and may be appropriately selected according to the embodiment. The inference model 55 may be represented by, for example, a functional formula, a data table, or the like. When expressed as a functional expression, the inference model 55 may be configured by, for example, a neural network, support vector machine, regression model, decision tree, or the like. Also, the machine learning method is not particularly limited, and may be appropriately selected according to the configuration of the inference model 55 . As a machine learning method for the inference model 55, for example, supervised learning, reinforcement learning, or the like may be employed. Two examples of each of the machine learning model and the machine learning method that make up the inference model 55 are described below.

(1-1) First Example FIG. 13 schematically shows a first example of a machine learning model and a machine learning method that constitute the inference model 55. As shown in FIG. In the first example, a neural network is used as the inference model 55, and supervised learning is used as the machine learning method. In the example of FIG. 13, for convenience of explanation, the inference model 551, the learning data 2231, and the inference model data 2251 respectively represent examples of the inference model 55, the learning data 223, and the inference model data 225, respectively.

(1-1-1) Configuration Example of Inference Model In the first example, the inference model 551 is configured by a recurrent neural network with a three-layer structure. Specifically, the inference model 551 includes an input layer N51, an LSTM (Long short-term memory) block N52, and an output layer N53. LSTM block N52 corresponds to the intermediate layer.

The LSTM block N52 is a block that has an input gate and an output gate and is configured to be able to learn the timing of storing and outputting information (S. Hochreiter and J. Schmidhuber, "Long short-term memory" Neural Computation, 9 (8):1735-1780, November 15, 1997). The LSTM block N52 may further include a forget gate that regulates the timing of information forgetting (Felix A. Gers, Jurgen Schmidhuber and Fred Cummins, "Learning to Forget: Continual Prediction with LSTM" Neural Computation, pages 2451-2471). , October 2000). The configuration of the LSTM block N52 may be appropriately set according to the embodiment.

Note that the structure of the inference model 551 does not have to be limited to such an example, and may be determined as appropriate according to the embodiment. The inference model 551 may be composed of recurrent neural networks with different structures. Alternatively, the inference model 551 may be configured by a fully-connected neural network or a convolutional neural network like the judgment model 50 described above instead of the recursive type. Alternatively, the inference model 551 may be configured by combining multiple types of neural networks. Also, the number of intermediate layers included in the inference model 551 is not limited to one, and may be two or more. Alternatively, the intermediate layer may be omitted. In addition, the configuration of the inference model 551 may be the same as that of the judgment model 50 described above.

(1-1-2) Configuration example of learning data The learning data 2231 used for supervised learning of the inference model 551 is a plurality of learning data sets including combinations of training data (input data) and correct data (teacher signal). L30. The training data may consist of relative coordinates in the training current task state L31 and relative coordinates in the training target task state L32. The correct answer data may consist of relative coordinates in task state L33 of the training target. Note that the formats of the training data and correct answer data are not particularly limited, and may be appropriately selected according to the embodiment. For example, as the training data, the relative coordinates may be used as they are, or values obtained by converting the relative coordinates into feature quantities may be used.

The control unit 21 can generate each learning data set L30 from one or more series of target task states obtained by the processing up to step S209. For example, the final target task state indicated by node Ng can be used as the final target task state L32 for training. Further, when the starting task state indicated by the node Ns is set as the current task state for training L31, the control unit 21 sets the target task state for training L33 in the corresponding correct answer data indicated by the node N1. You may set the task state to be Similarly, when the task state indicated by node N(k) is set as the current task state for training L31, the control unit 21 sets node N(k) to the target task state for training L33 in the corresponding correct answer data. k+1) may be set. When the task state indicated by the node N4 is set as the current task state for training L31, the control unit 21 sets the target task state for training L33 in the corresponding correct answer data to the final goal task indicated by the node Ng. You can set the state. Thereby, each learning data set L30 can be generated from one or more sequences of the obtained target task states.

(1-1-3) Step S210 In step S210, the control unit 21 performs machine learning (supervised learning) of the inference model 551 using the acquired plurality of learning data sets L30. In the first example, the control unit 21 uses machine learning to input training data to the input layer N51 for each learning data set L30, and the inference model is configured to output an output value that matches the correct data from the output layer N53. Train 551. As a result, it is possible to generate a trained inference model 551 that has acquired the ability to infer the next transition target task state from the current task state and the final target task state.

The machine learning method for the inference model 551 may be the same as the machine learning method for the judgment model 50 described above. That is, in the first step, the control unit 21 inputs training data to the input layer N51 of the inference model 551 for each learning data set L30, and executes arithmetic processing of the inference model 551. FIG. As a result, the control unit 21 obtains from the output layer L53 of the inference model 551 an output value corresponding to the result of inferring the target task state to which the current task state will transition next. In the second step, the controller 21 calculates the error between the output value of the output layer L53 and the correct data based on the loss function.

Subsequently, in the third step, the control unit 21 calculates the error of each calculation parameter value of the inference model 551 by using the error gradient of the calculated output value by the error backpropagation method. The control unit 21 uses the calculated gradient of the error to calculate the error in the value of each calculation parameter (for example, the weight of the connection between each neuron, the threshold value of each neuron, etc.) of the inference model 551 . In a fourth step, the control unit 21 updates the values of the calculation parameters of the inference model 551 based on each calculated error. The degree of update may be adjusted by the learning rate. The learning rate may be specified by the operator or may be given as a set value within the program.

By repeating the first to fourth steps, the control unit 21 repeats the inference model so that the sum of the error between the output value output from the output layer N53 and the correct data is small for each learning data set L30. 551 calculation parameter values are adjusted. For example, the control unit 21 may repeat the processes of the first to fourth steps until the sum of the errors becomes equal to or less than the threshold. The threshold may be appropriately set according to the embodiment. Alternatively, the control section 21 may repeat the above-described first to fourth steps a predetermined number of times. The number of times the adjustment is repeated may be designated by a set value in the program, or may be designated by an operator's input.

Based on the results of this machine learning (supervised learning), the control unit 21 inputs training data to the input layer N51 for each learning data set L30, and outputs an output value that matches the corresponding correct data from the output layer N53. A learned inference model 551 can be constructed that has been trained as follows. That is, it is possible to construct a trained inference model 551 that has acquired the ability to infer the target task state to which the next transition is to be made from the current task state and the final target task state.

In step S211, the control unit 21 generates, as the inference model data 2251, information indicating the structure and operation parameters of the learned inference model 551 constructed by supervised learning. Then, the control unit 21 saves the generated inference model data 2251 in a predetermined storage area. Note that the contents of the inference model data 2251 need not be limited to such an example as long as the learned inference model 551 can be reproduced. For example, when the structure of the inference model 551 is shared among the devices, information indicating the structure of the inference model 551 may be omitted from the inference model data 2251 .

(1-1-4) Others When supervised learning is adopted as the machine learning method, the configuration of the inference model 551 need not be limited to a neural network. A machine learning model other than a neural network may be employed as the inference model 551 . A support vector machine, a regression model, a decision tree, or the like, for example, may be adopted as the machine learning model that configures the inference model 551 . The supervised learning method is not limited to the above examples, and may be appropriately selected according to the configuration of the machine learning model.

(1-2) Second Example FIG. 14 schematically shows a second example of the machine learning model and the machine learning method that constitute the inference model 55 . A second example employs reinforcement learning as a method of machine learning. In the example of FIG. 14, for convenience of explanation, the inference model 552, the learning data 2232, and the inference model data 2252 represent examples of the inference model 55, the learning data 223, and the inference model data 225, respectively.

(1-2-1) Configuration Example of Inference Model In a second example, the inference model 552 may be value-based, policy-based, or both. When adopting a value base, the inference model 552 may be composed of value functions such as a state value function, an action value function (Q function), and the like. A state-value function is constructed to output the value of a given state. An action-value function is constructed to output the value of each action for a given state. If a policy base is employed, the inference model 552 may be composed of policy functions, for example. The policy function is constructed to output the probability of choosing each action for a given state. If both are employed, the inference model 552 may be composed of, for example, a value function (Critic) and a policy function (Actor). Each function may be represented by, for example, a data table, a function expression, or the like. When expressed by a functional formula, each function may be configured by a neural network, a linear function, a decision tree, or the like. Note that deep reinforcement learning may be performed by configuring each function with a multi-layered neural network having a plurality of intermediate (hidden) layers.

(1-2-2) Configuration Example of Learning Data Reinforcement learning basically assumes an agent that interacts with the learning environment by acting according to a policy. The substance of the agent is, for example, a CPU. The inference model 552 operates as a policy for determining behavior due to the above configuration. The agent observes the state of reinforcing behavior within a given learning environment. In this embodiment, the state to be observed is the task state defined by the relative coordinates, and the action to be performed is the transition from the current task state to the target task state. The policy is configured to determine (infer) the next transition target task state from the current task state and the final target task state.

The agent may provide the observed current task state (input data) to the inference model 552 to infer the target task state to transition to next. The agent may determine the target task state based on the results of this inference. Alternatively, the target task state may be randomly determined. This allows the agent to decide which action to take. When the agent performs an action that transitions to the determined target task state, the observed task state transitions to the next task state. In some cases, agents can obtain immediate rewards from the learning environment.

While repeating this trial-and-error decision and execution of actions, the agent updates the inference model 552 so as to maximize the sum of immediate rewards (ie, value). This reinforces the optimal action, that is, the action that can be expected to acquire high value, and obtains a policy (learned inference model 552) that enables the selection of such action.

Therefore, in reinforcement learning, the learning data 2232 is state transition data obtained by this trial and error. It is composed of state transition data indicating state transitions. One piece of state transition data may be composed of data indicating the trajectory of state transitions for all of one episode, or may be composed of data indicating state transitions for a predetermined number of times (one or more times). In the process of steps S202 to S209, the control unit 21 can acquire the state transition data by performing trial and error using the inference model 552 being trained.

Also, a reward function may be used to calculate immediate rewards in response to state transitions. A reward function may be represented by a data table, a functional expression, or a rule. When expressed by a functional formula, the reward function may be composed of a neural network, a linear function, a decision tree, or the like. The reward function may be manually set by an operator or the like.

Alternatively, the reward function is the result of determining whether the first object and the second object contact each other in the transition target task state by the learned determination model 50, and the target task state and final It may be set to give instant rewards depending on the distance between the target task states. Specifically, the immediate reward is set more as the first object and the second object do not contact each other and the distance between the target task state and the final goal task state is shorter, and the first object and the second object contact each other, or the longer the distance, the smaller the number may be set. Equation 1 below illustrates an example reward function that provides an immediate reward in this way.

ｓ_cは、方策により決定された目標のタスク状態を示す。ｓ_gは、最終目標のタスク状態を示す。Ｆ（ｓ_c）は、タスク状態ｓ_cにおいて第１対象物及び第２対象物が互いに接触するか否かを学習済みの判定モデル５０により判定した結果を示す。互いに接触すると判定された場合、Ｆ（ｓ_c）の値は小さくなり（例えば、０）、互いに接触しないと判定された場合に、Ｆ（ｓ_c）の値は大きくなる（例えば、１）ように設定されてよい。学習済みの判定モデル５０の出力値が当該設定に対応している場合には、学習済みの判定モデル５０の出力値がそのままＦ（ｓ_c）として使用されてもよい。

s _c denotes the target task state determined by the policy. s _g indicates the final target task state. F(s _c ) indicates the result of determination by the learned determination model 50 whether or not the first object and the second object are in contact with each other in the task state s _c . If it is determined that they are in contact with each other, the value of F(s _c ) will be small (eg, 0), and if it is determined that they are not in contact with each other, the value of F(s _c ) will be large (eg, 1). may be set to When the output value of the learned judgment model 50 corresponds to the setting, the output value of the learned judgment model 50 may be used as F(s _c ) as it is.

Alternatively, the reward function may be estimated by inverse reinforcement learning from case data obtained by experts. The example data may consist of data showing (the trajectory of) a demonstration by an expert. In this embodiment, the case data may be composed of, for example, data indicating the actual movement path of the first object so as to reach the final goal task state from the arbitrary starting point task state. A method for generating case data is not particularly limited, and may be appropriately selected according to the embodiment. Case data may be generated, for example, by recording the trajectory of an expert's performance with a sensor or the like.

A method of inverse reinforcement learning may not be particularly limited, and may be appropriately selected according to the embodiment. Inverse reinforcement learning includes, for example, methods based on the maximum entropy principle, methods based on the minimization of relative entropy, and methods using generative adversarial networks (e.g., Justin Fu, et al., "Learning Robust Rewards with Adversarial Inverse Reinforcement Learning" , arXiv:1710.11248, 2018) and the like may be used. When obtaining a reward function by inverse reinforcement learning, the learning data 2232 may further include case data used for inverse reinforcement learning.

(1-2-3) Step S210 At step S210, the control unit 21 updates the values of the calculation parameters of the inference model 552 so as to maximize the value based on the obtained state transition data. A method for adjusting the values of the calculation parameters of the inference model 552 may be appropriately selected according to the configuration of the inference model 552 . For example, when the inference model 552 is composed of a neural network, the values of the calculation parameters of the inference model 552 may be adjusted in the same manner as in the first example, such as by error backpropagation.

The control unit 21 adjusts the values of the calculation parameters of the inference model 552 so that (the expected value of) the obtained value is maximized (for example, until the update amount becomes equal to or less than the threshold). That is, training the inference model 552 repeats correction of the values of the calculation parameters that make up the inference model 552 so that a large reward can be obtained until a predetermined condition (for example, the update amount is equal to or less than a threshold) is satisfied. Including. As a result, the control unit 21 can generate the learned inference model 552 that has acquired the ability to infer the next transition target task state from the current task state and the final target task state.

Note that the control unit 21 may adjust the values of the calculation parameters of the inference model 552 after completing the collection of the learning data 2232 through the processing of steps S202 to S209. Alternatively, the control unit 21 may adjust the values of the calculation parameters of the inference model 552 while repeating the processes of steps S202 to S210.

When the inference model 552 is configured on a value basis, the TD (temporal difference) method, TD(λ) method, Monte Carlo method, dynamic programming, or the like may be used as the reinforcement learning method. Behavioral decisions in trial and error may be on-policy or off-policy. As a specific example, Q-learning, Sarsa, or the like may be used as the reinforcement learning method. During trial and error, random actions may be taken with probability ε (ε-greedy method).

Also, when the inference model 552 is configured on a policy basis, the method of reinforcement learning may be a policy gradient method, TRPO (trust region policy optimization), PPO (proximal policy optimization), or the like. In this case, the control unit 21 calculates the gradient of the calculation parameter of the policy function in the direction in which the value obtained increases, and updates the value of the calculation parameter of the policy function based on the calculated gradient. For example, the REINFORCE algorithm or the like may be used to calculate the gradient of the policy function.

Moreover, when the inference model 55 is composed of both, the Actor Critic method, A2C (Advantage Actor Critic), A3C (Asynchronous Advantage Actor Critic), etc. may be used as the reinforcement learning method.

Furthermore, when performing inverse reinforcement learning, the control unit 21 further acquires case data before executing the reinforcement learning process. The case data may be generated by the second model generation device 2 or may be generated by another computer. When generated by another computer, the control unit 21 may acquire the case data generated by the other computer via a network, the storage medium 92, or the like. Next, the control unit 21 sets a reward function by executing inverse reinforcement learning using the acquired case data. Then, the control unit 21 uses the reward function set by the inverse reinforcement learning to execute the reinforcement learning process. As a result, the control unit 21 utilizes the reward function set by inverse reinforcement learning, and acquires the ability to infer the next transition target task state from the current task state and the final target task state. can generate an inference model 552 of

In step S211, the control unit 21 generates, as the inference model data 2252, information indicating the learned inference model 552 constructed by reinforcement learning. The information indicating the learned inference model 552 may include, for example, information indicating calculation parameters such as the value of each item in the data table and the value of the coefficient of the function expression. Then, the control unit 21 saves the generated inference model data 2252 in a predetermined storage area. According to the second example, the target task state can be determined so that the task state of the manipulator 4 quickly reaches the final target task state while avoiding unnecessary contact between the first object and the second object. An inference model 55 can be generated.

(1-3) Summary In the present embodiment, when configuring the inference model 55 using a machine learning model, at least one of the above two examples may be adopted for the configuration of the inference model 55 . By adopting at least one of the above two machine learning methods, the control unit 21 controls the current task state and the final target task state so that the first object does not come into contact with the second object. A trained inference model 55 can be generated that has acquired the ability to infer transitional target task states. Therefore, according to the first method, it is possible to appropriately generate the inference model 55 that can be used to accomplish the task.

(2) Second Method FIG. 15A schematically illustrates an example of learning data 223 in the second method. FIG. 15B schematically illustrates an example of the configuration of the inference model 55 in the second method. In a second method, the inference model 55 consists of a potential field that defines the potential of each coordinate within the task space SP representing the set of task states. 15A and 15B, examples of the inference model 55, the learning data 223, and the inference model data 225 are denoted as an inference model 553, learning data 2233, and inference model data 2253 for convenience of explanation.

Through the processing of steps S202 to S209, the control unit 21 uses the learned judgment model 50 in the task space SP to perform path planning so that the first object does not come into contact with the second object. implement. Thereby, as illustrated in FIG. 15A, the control unit 21 generates learning data 2233 indicating paths Hb from each of the plurality of task states given as starting points (nodes Ns) to the final target task state. be able to. Each starting point (node Ns) may be given randomly.

In step S<b>210 described above, the control unit 21 generates a potential field by setting the potential of each coordinate according to the frequency of passage of each route Hb indicated by the generated learning data 2233 . A method for deriving the potential field is not particularly limited, and may be appropriately selected according to the embodiment. The control unit 21 may derive the potential field from the learning data 3233 by, for example, kernel density estimation or estimation using a Gaussian Mixture Model (GMM). As a result, the potential field (inference model 553) exemplified in FIG. 15B can be obtained.

The potential of each coordinate in the potential field indicates an evaluation value of the positional relationship of the first object and the second object at each coordinate with respect to reaching the final goal. That is, the higher the potential, the higher the possibility that the positional relationship at that coordinate will reach the final target, and the lower the potential, the lower the possibility that the positional relationship at that coordinate will reach the final target. Therefore, by making a transition to the higher potential gradient, it is possible to appropriately reach the final target task state from an arbitrary task state that is the starting point. Therefore, according to the second method, it is possible to appropriately generate the inference model 55 that can be used to accomplish the task.

In step S<b>211 , the control unit 21 generates information indicating the generated potential field as the inference model data 2253 . A potential field may be represented by a data table, a functional expression, or the like. Then, the control unit 21 saves the generated inference model data 2253 in a predetermined storage area.

(3) Summary In this embodiment, at least one of the above two methods may be adopted as a method of generating the inference model 55 . By adopting at least one of the above two methods, the control unit 21 transitions from the current task state and the final target task state to the next so that the first object does not come into contact with the second object. An inference model 55 may be generated that is configured to infer a target task state to be executed.

It should be noted that inferring the target task state such that the first object does not touch the second object is to avoid unintended contact between the first object and the second object to avoid unintended contact between the first object and the second object. Determining a state, and may include determining, as a target task state, a task state in which the first object properly contacts the second object, such as the end effector T holding the work W. . That is, the "contact" state to be avoided is, for example, when excessive force acts between the first object and the second object, and one of the first object and the second object is in the correct posture with respect to the other. It is a state of improper contact such as contacting in a state other than assembled with. Therefore, "the first object does not contact the second object" may be replaced with "avoid the first object from improperly contacting the second object".

[Control device]
(A) Operation Control Next, an operation example regarding operation control of the manipulator 4 of the control device 3 according to the present embodiment will be described with reference to FIGS. 16A, 16B, and 17. FIG. FIGS. 16A and 16B are flowcharts showing an example of a processing procedure regarding motion control of the manipulator 4 by the control device 3 according to this embodiment. FIG. 17 shows an example of the calculation processing flow of each element in the process of motion control. However, each processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, for each processing procedure described below, it is possible to omit, replace, or add steps as appropriate according to the embodiment. Note that the control of the operation of the manipulator 4, which will be described below, may be performed in real space or in virtual space.

(Step S301 and Step S302)
In step S301, control unit 31 accepts designation of a task to be performed. A method of accepting task designation is not particularly limited, and may be appropriately selected according to the embodiment. For example, the control unit 31 may receive designation of the task to be performed by inputting the name of the task via the input device 35 . Further, for example, the control unit 31 may receive designation of a task to be performed by outputting a list indicating candidates of tasks to be performed to the output device 36 and having the operator select a task to be performed from the list.

In this embodiment, the control unit 31 receives execution of a task of moving the first object with respect to the second object in an environment where the first object and the second object exist. Specifically, a series of operations of driving the manipulator 4, holding the work W by the end effector T, and assembling the held work W to another work G is an example of the designated task. In this embodiment, in the process of the first task to hold the work W, the point of interest T0 of the end effector T is treated as the tip of the manipulator 4, and the work W is the target of movement of the tip. On the other hand, after holding the work W, in the process of the second task of assembling the work W to another work G, the attention point W0 of the work W held by the end effector T is treated as the hand of the manipulator 4, Another workpiece W to be assembled is the target for movement of the hand. In each task, the hand of the manipulator 4 corresponds to the first object, and the target corresponds to the second object.

In step S302, the control unit 31 operates as the target setting unit 310, and sets the final target task state s _g according to the specified task. As described above, in this embodiment, the task state is defined by the positional relationship between the hand of the manipulator 4 and the target. Also, the positional relationship is represented by relative coordinates. The relative coordinates in the final target task state s _g may be given by a simulator such as CAD, or may be given by an operator's designation. The relative coordinates in the final target task state s _g may be set by the same method as in step S201. After setting the final target task state s _g , the control unit 31 proceeds to the next step S303.

(Step S303)
In step S303, the control unit 31 operates as the first data acquisition unit 311 and acquires the first sensing data 323 from the first sensor system. Also, the control unit 31 operates as a second data acquisition unit 312 and acquires second sensing data 324 from the second sensor system.

In this embodiment, the first sensor system comprises an encoder S2 for measuring the angle of each joint (joints 41 to 46) and a tactile sensor S3 for measuring the force acting on the end effector T. FIG. As the first sensing data 323, the control unit 31 can acquire the current value q _(j) of the angle of each joint in the manipulator 4 (that is, the current measured value) from each encoder S2. Furthermore, the control unit 31 can acquire measurement data of force acting on the end effector T from the tactile sensor S3 as the first sensing data 323 . Moreover, in this embodiment, the second sensor system is configured by the camera S1. As the second sensing data 324, the control unit 31 can acquire image data showing the environment in which the task is performed from the camera S1. In the following, for the convenience of explanation, when the timing such as "now" is particularly distinguished, a symbol representing the timing such as (j) will be attached, and if not, the symbol will be omitted.

The control unit 31 may directly acquire each sensing data (323, 324) from each sensor (camera S1, encoder S2, tactile sensor S3), or, for example, via another computer. The sensing data (323, 324) may be obtained indirectly by using the The camera S1 and the tactile sensor S3 are examples of observation sensors that observe the state of the work W with respect to the end effector T, respectively. After acquiring each sensing data (323, 324), the control unit 31 advances the process to the next step S304.

(Step S304)
At step S304, the control unit 31 determines whether or not the end effector T holds the workpiece W based on the sensing data of the observation sensor obtained at step S303. The determination method is not particularly limited, and may be determined as appropriate according to sensing data.

For example, in the present embodiment, as the second sensing data 324, image data representing the environment of the task can be acquired from the camera S1. Therefore, the control unit 31 may use the CAD data 320 to match models of the end effector T and the work W with the acquired image data. Then, the control unit 31 may determine whether or not the end effector T holds the work W based on the positional relationship between the end effector T and the work W specified as a result of the matching. A known image processing method may be used as the matching method.

Further, for example, in the present embodiment, measurement data of force acting on the end effector T can be acquired as the first sensing data 323 . Therefore, the control section 31 may determine whether or not the end effector T holds the workpiece W based on the distribution of force appearing in the measurement data. If it is estimated from the measurement data that the end effector T is holding the workpiece W with force acting on the end effector T, the control unit 31 determines that the end effector T is holding the workpiece W. You can judge. On the other hand, otherwise, the control section 31 may determine that the end effector T does not hold the work W. FIG.

After completing the determination of whether or not the end effector T holds the workpiece W based on the sensing data, the control unit 31 proceeds to the next step S305.

(Step S305)
At step S305, the control unit 31 sets the operation mode of the manipulator 4 based on the determination result at step S304. Specifically, when it is determined that the end effector T does not hold the work W, the control unit 31 sets the attention point T0 of the end effector T to the tip of the manipulator 4, and holds the work W by the end effector T. The operation mode is set to the mode in which the first task to be executed is performed. On the other hand, when it is determined that the end effector T holds the work W, the control unit 31 sets the attention point W0 of the work W to the tip of the manipulator 4, and moves the work W held by the end effector T to another position. The operation mode is set to the mode in which the second task to assemble the workpiece G is performed. When the setting of the operation mode is completed, the control unit 31 advances the process to the next step S306.

(Step S306)
In step S<b>306 , the control unit 31 operates as the state acquisition unit 315 and acquires the current task state s _(j) of the manipulator 4 .

As described above, in this embodiment, when the end effector T does not hold the work W, the task state s is defined by the relative coordinates of the work W with respect to the end effector T. On the other hand, when the end effector T holds the work W, the task state s is defined by the relative coordinates of another work G with respect to the work W. In this embodiment, the control unit 31 uses the CAD data 320 to match each object with the image data obtained by the camera S1. The control unit 31 can obtain the current task state s _(j) from the result of this matching.

An example of a method for acquiring the current task state s _(j) will now be described with further reference to FIG. FIG. 18 schematically illustrates an example of the positional relationship of each target. In the example of FIG. 18 , the origin of the observation space is set on the pedestal 40 of the manipulator 4 . However, the position of the origin need not be limited to such an example, and may be determined according to the embodiment. The homogeneous coordinates (T _C ) of the camera S1 with respect to the origin can be expressed by Equation 2 below.

Ｒ_rcは、原点の座標系からカメラＳ１の座標系を見た回転成分を示し、ｔ_rcは、平行移動成分を示す。以下では、説明の便宜上、原点の同次座標（Ｔ_R）以下の式３を満たすようにカメラＳ１がキャリブレーションされていると想定する。

_Rrc indicates a rotation component when the coordinate system of the camera S1 is viewed from the coordinate system of the origin, and _trc indicates a translation component. In the following, for convenience of explanation, it is assumed that the camera S1 is calibrated so that the homogeneous coordinates (T _R ) of the origin and the following Equation 3 are satisfied.

Ｉは、単位行列を示す。図１８の例では、原点に対するエンドエフェクタＴの注目点Ｔ０の相対座標がエンドエフェクタＴの座標（Ｔ_t）である。原点に対するワークＷの注目点Ｗ０の相対座標がワークＷの座標（Ｔ_w）である。原点に対する他のワークＧの注目点Ｇ０の相対座標が他のワークＧの座標（Ｔ_g）である。制御部３１は、ＣＡＤデータ３２０を利用して、画像データに対して各対象物のモデルをマッチングすることで、各座標（Ｔ_t、Ｔ_w、Ｔ_g）の推定値を得ることができる。制御部３１は、処理タイミングにおいて得られた各座標の推定値を各座標の現在値として利用することができる。

I indicates an identity matrix. In the example of FIG. 18, the coordinates of the target point T0 of the end effector T relative to the origin are the coordinates of the end effector T (T _t ). The relative coordinates of the target point W0 of the work W with respect to the origin are the coordinates of the work W (T _w ). The relative coordinates of the target point G0 of another work G with respect to the origin are the coordinates of the other work G (T _g ). Using the CAD data 320, the control unit 31 can obtain an estimated value of each coordinate (T _t , T _w , T _g ) by matching the model of each object to the image data. The control unit 31 can use the estimated value of each coordinate obtained at the processing timing as the current value of each coordinate.

エンドエフェクタＴがワークＷを保持していない場合には、タスク状態ｓとエンドエフェクタＴ及びワークＷの各座標（Ｔ_t、Ｔ_w）との関係は、上記式４により表現することができる。そのため、制御部３１は、マッチングの結果により推定されたエンドエフェクタＴ及びワークＷの各座標の現在値（Ｔ_t(j)、Ｔ_w(j)）を上記式４に代入し、上記式４の演算処理を実行することで、現在のタスク状態ｓ_(j)の推定値を算出することができる。この現在のタスク状態ｓ_(j)の推定値を算出することが、現在のタスク状態ｓ_(j)を取得することに相当する。

When the end effector T does not hold the work W, the relationship between the task state s and the coordinates (T _t , T _w ) of the end effector T and the work W can be expressed by Equation 4 above. Therefore, the control unit 31 substitutes the current values (T _t(j) , T _w(j) ) of the coordinates of the end effector T and the work W estimated from the matching result into the above equation 4, and By executing the arithmetic processing of , an estimated value of the current task state s _(j) can be calculated. Calculating the estimated value of the current task state s _(j) corresponds to obtaining the current task state s _(j) .

一方、エンドエフェクタＴがワークＷを保持している場合には、タスク状態ｓとワークＷ及び他のワークＧの各座標（Ｔ_w、Ｔ_g）との関係は、上記式５により表現することができる。そのため、制御部３１は、マッチングの結果により推定されたワークＷ及び他のワークＧの各座標の現在値（Ｔ_w(j)、Ｔ_g(j)）を上記式５に代入し、上記式５の演算処理を実行することで、現在のタスク状態ｓ_(j)の推定値を算出することができる。なお、各座標（Ｔ_t、Ｔ_w、Ｔ_g）の表現は、適宜選択されてよい。各座標（Ｔ_t、Ｔ_w、Ｔ_g）の表現には、例えば、同次座標系が用いられてよい。以下についても同様である。

On the other hand, when the end effector T holds the work W, the relationship between the task state s and the coordinates (T _w , T _g ) of the work W and another work G can be expressed by the above equation 5. can be done. Therefore, the control unit 31 substitutes the current values (T _w(j) , T _g(j) ) of the coordinates of the work W and the other work G estimated by the matching result into the above formula 5, and the above formula 5, an estimated value of the current task state s _(j) can be calculated. Note that the representation of each coordinate (T _t , T _w , T _g ) may be selected as appropriate. For example, a homogeneous coordinate system may be used to express each coordinate (T _t , T _w , T _g ). The same applies to the following.

If the camera S1 is not calibrated, the control unit 31 may further calculate an estimate of the origin coordinates (T _R ) within the image data obtained by the camera S1. A mark such as a marker may be used to detect the origin. That is, by matching landmarks within the image data, an estimate of the coordinates of the origin (T _R ) may be calculated. The control unit 31 can calculate the estimated value of the current task state s _(j) by applying the calculated estimated value of the coordinates (T _R ) of the origin to each of the above calculations. In subsequent steps, the case of performing matching with CAD data 320 may be similarly handled.

After obtaining the current task state s _(j) , the control unit 31 advances the process to the next step S307. Note that the timing of executing the process of step S306 need not be limited to such an example. The processing of step S306 may be performed at any timing before step S308, which will be described later, is performed. For example, when matching is performed using the CAD data 320 also in step S304, the process of step S306 may be performed together with the process of step S304.

(Step S307)
In step S<b>307 , the control unit 31 operates as the first estimation unit 313 and uses the first estimation model 61 to determine the current coordinates of the hand in the observation space from the acquired first sensing data 323 . Calculate an estimate. In addition, the control unit 31 operates as a second estimation unit 314 and uses the second estimation model 62 to obtain a second estimated value of the current coordinates of the hand in the observation space from the acquired second sensing data 324. Calculate

(1) First Estimated Value Calculation Process First, an example of the first estimated value calculation process will be described. As shown in FIG. 17, the control unit 31 uses the forward kinematics calculation to calculate the current values q _(j) of the angles of the joints of the manipulator 4 in the joint space obtained by the encoders S2 (first sensing data 323). A first estimate of the hand coordinates of the manipulator 4 in the observation space is calculated (in other words, the current value x _(j) is estimated). Hereinafter, the case where the end effector T does not hold the work W and the case where the end effector T holds the work W will be described separately.

(1-1) Scene where the work W is not held When the end effector T does not hold the work W, the attention point T0 of the end effector T is set at the hand. In this case, the control unit 31 calculates the current value _q A first estimated value of the set hand coordinates is calculated from _j) .

具体的には、順運動学により、エンドエフェクタＴの注目点Ｔ０の座標（ｘ_t）と各関節の角度（ｑ）との関係は、上記式６により表現することができる。角度（ｑ）は、関節数に応じた次元数を有する変数である。また、各関節の第１同次変換行列（_m-1Ｔ^m）と第１変換行列群（φ）との関係は、上記式７により与えられる（ｍは、０～ｎ。ｎは、関節数）。第１同次変換行列は、対象の関節よりも手元側の座標系から見た対象の関節の座標系の相対座標を表し、手元側の座標系から対象の関節の座標系に座標を変換するのに利用される。

Specifically, the relationship between the coordinates (x _t ) of the point of interest T0 of the end effector T and the angle (q) of each joint can be expressed by the above equation 6 according to forward kinematics. Angle (q) is a variable with a number of dimensions corresponding to the number of joints. The relationship between the first homogeneous transformation matrix ( _m-1 ^Tm ) of each joint and the first transformation matrix group (φ) is given by Equation 7 above (m is 0 to n; n is the joint number). The first homogeneous transformation matrix represents the relative coordinates of the coordinate system of the target joint viewed from the coordinate system closer to the target joint than the target joint, and transforms the coordinates from the coordinate system of the target joint to the coordinate system of the target joint. used for

The parameter values of the first homogeneous transformation matrix of each joint are known except for the angle of each joint, and are included in the robot data 321 in this embodiment. The parameters may be set by a known method such as DH (Denavit-Hartenberg) notation, modified DH notation, or the like. The control unit 31 refers to the robot data 321 to derive the first transformation matrix group (φ) shown in Equation 7 above. Then, the control unit 31 substitutes the current value q _(j) of the angle of each joint into the derived first transformation matrix group (φ) as shown in Equation 6 above, and calculates the first transformation matrix group (φ). Execute the process. Based on the result of this forward kinematics calculation, the control unit 31 calculates an estimated value of the current coordinates of (the target point T0 of) the end effector T (in other words, estimates the current coordinate value x _t(j) ). be able to. The control unit 31 acquires the calculated estimated value as the first estimated value of the current hand coordinates.

(1-2) Scene of Holding Work W On the other hand, when the end effector T holds the work W, the target point W0 of the work W is set at the end of the hand. In this case, first, the control unit 31 acquires a second homogeneous transformation matrix ( _t T ^w ) for transforming the coordinates from the coordinate system of the point of interest T0 of the end effector T to the coordinate system of the point of interest W0 of the work W. do.

A method for obtaining the second homogeneous transformation matrix ( _t T ^w ) is not particularly limited, and may be appropriately selected according to the embodiment. For example, when the work W is held by the end effector T, there are cases where the position and orientation of the work W with respect to the end effector T are constant. Therefore, the second homogeneous transformation matrix ( _tTw ) may be given by ^a constant.

Alternatively, the control unit 31 may estimate the second homogeneous transformation matrix ( _tTw ⁾ from the sensing data acquired in step S303. As an example of the estimation method, the control unit 31 may use the CAD data 320 to match models of the end effector T and the workpiece W with image data obtained by the camera S1. The control unit 31 can obtain estimated values of the coordinates (T _t ) of the end effector T and the coordinates (T _w ) of the work W from the result of this matching. Assuming that the camera S1 has been calibrated in the same manner as described above, the control unit 31 calculates the estimated values of the coordinates (T _t ) of the end effector T and the coordinates (T _w ) of the workpiece W using the following equation 8: A second homogeneous transformation matrix ( _t T ^w ) can be estimated from

ＣＡＤデータ３２０によるマッチングは、上記順運動学計算により算出されるエンドエフェクタＴの注目点Ｔ０の座標（ｘ_t）付近で実施されてもよい。また、制御部３１は、順運動学計算により算出される座標（ｘ_t）の推定値を座標（Ｔ_t）の推定値として利用してもよい。これにより、制御部３１は、第２同次変換行列（_tＴ^w）を推定することができる。なお、上記ステップＳ３０６の上記式４の演算処理でも同様に、マッチングにより推定されたエンドエフェクタＴの座標の現在値（Ｔ_t(j)）の代わりに、順運動学計算により推定された座標の現在値（ｘ_t(j)）が用いられてよい。

Matching by the CAD data 320 may be performed near the coordinate (x _t ) of the target point T0 of the end effector T calculated by the above forward kinematics calculation. Also, the control unit 31 may use the estimated value of the coordinate (x _t ) calculated by the forward kinematics calculation as the estimated value of the coordinate (T _t ). Thereby, the control unit 31 can estimate the second homogeneous transformation matrix ( _t T ^w ). It should be noted that in the arithmetic processing of the above equation 4 in step S306, similarly, instead of the current value (T _t(j) ) of the coordinates of the end effector T estimated by matching, the coordinates estimated by forward kinematics calculation are A current value (x _t(j) ) may be used.

As another example of the estimation method, the distribution of force acting on the end effector T measured by the tactile sensor S3 can depend on the position and posture of the work W with respect to the end effector T. Therefore, the control unit 31 may estimate the relative coordinates (relative position and relative orientation) of the work W with respect to the end effector T based on measurement data (first sensing data 323) obtained by the tactile sensor S3. ^The control unit 31 can estimate the second homogeneous transformation matrix ( _tTw ) from the result of this estimation.

Note that the method of estimating the second homogeneous transformation matrix ( _tTw ) from the sensing data ³²² need not be limited to the analytical method described above. For estimating the second homogeneous transformation matrix ( _t T ^w ), for example, similar to the judgment model 50, the inference model 551, etc., the second homogeneous transformation matrix ( _t T ^w ) is obtained from the sensing data 322 by machine learning. A trained machine learning model that has acquired the ability to estimate may be utilized. In this case, the control unit 31 gives the acquired sensing data 322 to the learned machine learning model, and executes the arithmetic processing of the learned machine learning model. Thereby, the control unit 31 can acquire the output value corresponding to the result of estimating the second homogeneous transformation matrix ( _t T ^w ) from the learned machine learning model.

Next, the control unit 31 multiplies the obtained second homogeneous transformation matrix ( _t T ^w ) by the first transformation matrix group (φ) to obtain the second transformation matrix group (φ(q)· _t T ^w ) is calculated. The second transformation matrix group can be expressed by Equation 9 below. The first homogeneous transformation matrix is an example of a first transformation formula, and the second homogeneous transformation matrix is an example of a second transformation formula. The first transformation matrix group (φ) is an example of a first transformation equation group, and the second transformation matrix group (φ(q)· _t T ^w ) is an example of a second transformation equation group. The format of each conversion formula does not have to be particularly limited as long as it can be used for calculation of hand coordinates. For example, each transform formula may be represented by a transform matrix in a format other than a homogeneous coordinate system, or may be represented by a formula in a format other than a matrix.

制御部３１は、算出された第２変換行列群を変換関数として用いた順運動学計算により、各関節の角度の現在値ｑ_(j)から設定された手先座標の第１推定値を算出する。すなわち、制御部３１は、第２変換行列群（φ（ｑ）・_tＴ^w）に各関節の角度の現在値ｑ_(j)を代入し、第２変換行列群（φ（ｑ）・_tＴ^w）の演算処理を実行する。この順運動学計算の結果により、制御部３１は、ワークＷ（の注目点Ｗ０）の現在の座標の推定値を算出する（換言すると、座標の現在値を推定する）ことができる。制御部３１は、算出されたワークＷの現在の座標の推定値を現在の手先座標の第１推定値として取得する。

The control unit 31 calculates a first estimated value of the set hand coordinate from the current value q _(j) of the angle of each joint by forward kinematics calculation using the calculated second transformation matrix group as a transformation function. . That is, the control unit 31 substitutes the current value q _(j) of the angle of each joint into the second transformation matrix group (φ(q)· _t T ^w ), and the second transformation matrix group (φ(q)· _t T ^w ). Based on the results of this forward kinematics calculation, the control unit 31 can calculate estimated values of the current coordinates of (the point of interest W0 of) the work W (in other words, estimate the current values of the coordinates). The control unit 31 acquires the calculated estimated value of the current coordinates of the work W as the first estimated value of the current hand coordinates.

In the above, by estimating the second homogeneous transformation matrix ( _t T ^w ) from the sensing data, even if the holding state of the workpiece W in the end effector T fluctuates, the second homogeneous transformation matrix reflecting the fluctuation can be obtained. A transformation matrix ( _t T ^w ) can be obtained. As a result, even if the holding state of the work W on the end effector T may fluctuate, the current values of the coordinates of the work W, that is, the current values of the hand coordinates of the manipulator 4 can be appropriately estimated.

(1-3) Summary As described above, in each of the above situations, the control unit 31 performs forward kinematics calculation using the derived conversion function to convert the current value q _(j) of the angle of each joint to the manipulator 4 A first estimate of the current hand coordinates of . A conversion function used for this forward kinematics calculation is an example of the first estimation model 61 . That is, when the end effector T does not hold the workpiece W, the first transformation matrix group (φ) corresponds to an example of the first estimation model 61 . In addition, when the end effector T holds the workpiece W, the second transformation matrix group (φ(q)· _t T ^w ) corresponds to an example of the first estimation model 61 . Each parameter of each conversion function corresponds to an example of the parameters of the first estimation model 61 .

(2) Second Estimated Value Calculation Process Next, an example of the second estimated value calculation process will be described. The control unit 31 uses the CAD data 320 to match the model of each object with the image data (second sensing data 324) obtained by the camera S1. Thereby, the control unit 31 can calculate the second estimated value of the current hand coordinates of the manipulator 4 (in other words, estimate the current value x _(j) ). In this case, the control unit 31 may calculate a second estimated value of the current hand coordinates of the manipulator 4 from the current task state s _(j) estimated in the task space in step S306.

図１８に示される各対象物の位置関係に基づいて、エンドエフェクタＴがワークＷを保持していない場合におけるタスク状態ｓ及び手先の座標ｘの間の関係は、上記式１０により表現することができる。この場合には、タスク空間から観測空間への変換関数（ψ）は、ワークＷの座標（Ｔ_w）により与えられる。制御部３１は、上記ステップＳ３０６により取得された現在のタスク状態ｓ_(j)及びマッチングにより推定されたワークＷの座標の現在値（Ｔ_w(j)）を式１０に代入し、上記式１０の演算処理を実行することで、マニピュレータ４の現在の手先座標の第２推定値を算出することができる。

Based on the positional relationship of each object shown in FIG. 18, the relationship between the task state s and the coordinate x of the hand when the end effector T does not hold the work W can be expressed by Equation 10 above. can. In this case, the transformation function (φ) from task space to observation space is given by the coordinates of the work W (T _w ). The control unit 31 substitutes the current task state s _(j) obtained in step S306 and the current value (T _w(j) ) of the coordinates of the work W estimated by matching into equation 10 to obtain the equation 10 , the second estimated value of the current hand coordinates of the manipulator 4 can be calculated.

同様に、エンドエフェクタＴがワークＷを保持している場合におけるタスク状態ｓ及び手先の座標ｘの間の関係は、上記式１１により表現することができる。この場合には、タスク空間から観測空間への変換関数（ψ）は、他のワークＧの座標（Ｔ_g）により与えられる。制御部３１は、上記ステップＳ３０６により取得された現在のタスク状態ｓ_(j)及びマッチングにより推定された他のワークＧの座標の現在値（Ｔ_g(j)）を式１１に代入し、上記式１１の演算処理を実行することで、マニピュレータ４の現在の手先座標の第２推定値を算出することができる。

Similarly, the relationship between the task state s and the hand coordinate x when the end effector T holds the work W can be expressed by Equation 11 above. In this case, the transformation function (φ) from task space to observation space is given by the coordinates (T _g ) of another work G. The control unit 31 substitutes the current task state s _(j) obtained in step S306 and the current value (T _g(j) ) of the coordinates of the other work G estimated by matching into Equation 11, and A second estimated value of the current hand coordinates of the manipulator 4 can be calculated by executing the arithmetic processing of Expression 11.

The conversion function (ψ) of Equations 10 and 11 above is an example of the second estimation model 62 . Each parameter of each conversion function (ψ) corresponds to a parameter of the second estimation model 62 . Note that the method of estimating the current value x _(j) of the hand coordinates of the manipulator 4 by matching using the CAD data 320 need not be limited to this example. When the end effector T does not hold the workpiece W, the control unit 31 estimates the current coordinate value (T _t(j) ) of the end effector T by the above matching, and calculates the estimated current value (T _{t( j)} ) may be taken as a second estimate of the current hand coordinates. Similarly, when the end effector T holds the work W, the control unit 31 estimates the current values (T _w(j) ) of the coordinates of the work W by the above matching, and the estimated current values (T _w(j) ) may be taken as a second estimate of the current hand coordinates. That is, the control unit 31 may directly derive the second estimated value of the current hand coordinates of the manipulator 4 by the above matching. In this case, each coordinate (T _t , T _w ) is an example of the second estimation model 62 . Also, each term of each coordinate (T _t , T _w ) is an example of a parameter of the second estimation model 62 .

(3) Summary As described above, the control unit 31 can calculate the first estimated value and the second estimated value of the current hand coordinates of the manipulator 4 . The control unit 31 recognizes the current value x _(j) of the coordinates of the hand of the manipulator 4 based on at least one of the first estimating unit and the second estimating unit. This accreditation may be done as appropriate. For example, the control unit 31 may directly adopt either the first estimated value or the second estimated value as the current hand coordinate value x _(j) . Further, for example, the control unit 31 may calculate an average value of the first estimated value and the second estimated value, and obtain the calculated average value as the current hand coordinate value x _(j) . In this case, the average value may be calculated by weighted averaging. Each estimated value may be weighted so that an estimated value assumed to have high estimation accuracy is given priority. As an example, it is assumed that the estimation accuracy of the hand coordinates by forward kinematics calculation is higher than the estimation accuracy of the hand coordinates by matching to the image data of the camera S1. In this case, the estimates may be weighted to favor the first estimate over the second estimate. After acquiring the current hand coordinate value x _(j) , the control unit 31 advances the process to the next step S308.

Note that when the adjustment process described later is not executed, the process of calculating either the first estimated value or the second estimated value may be omitted. Also, the timing of executing the process of step S307 need not be limited to such an example. The process of step S307 may be executed at any timing before executing the process of step S310, which will be described later. For example, the process of step S307 may be performed before step S306. Further, for example, in the case of performing matching using the CAD data 320, the process of step S307 may be executed together with the process of step S306 or step S304.

(Step S308)
In step S308, the control unit 31 operates as the action determination unit 316, and determines the next target task to transition to the acquired current task state s _(j) so as to approach the final target task state _sg . Determine the state s _s(j) . In this embodiment, the control unit 31 refers to the inference model data 225 and uses the inference model 55 generated by the process of step S210 to make the next transition to the current task state s _(j). Determine the target task state s _s(j) .

Arithmetic processing of the inference model 55 for inferring the target task state s _s(j) to be transitioned to next may be appropriately executed according to the configuration of the inference model 55 . If the inference model 55 is generated by the first method, and the inference model 55 is composed of a functional expression, the control unit 31 converts the current task state s _(j) and the final target task state s _g into a function Substitute into the expression and execute the arithmetic processing of the function expression. When the inference model 55 is composed of a neural network, the control unit 31 inputs the current task state s _(j) and the final target task state s _g to the input layer, and each neuron included in each layer in order from the input side fire determination. When the inference model 55 is composed of a data table, the control unit 31 checks the current task state s _(j) and the final target task state s _g against the data table. As a result, the control unit 31 acquires, as an output of the inference model 55, the result of inferring the target task state s _s(j) to which the task transitions next. Based on this inference result, the control unit 31 can determine the target task state s _s(j) to which the task transitions next.

Further, when the inference model 55 is generated by the second method, that is, when the inference model 55 is composed of a potential field, the control unit 31 calculates the current task state s _(j) in the generated potential field. Refer to the potential value set at the coordinates corresponding to . Then, the control unit 31 determines the target task state s s _(j) to which the next transition is to be made, according to the gradient of the potential set at the coordinates corresponding to the current task state _{s (j)} . Specifically, the control unit 31 determines the target task state s _s(j) such that the transition to the higher gradient of the potential (for example, transition to the highest gradient by a predetermined distance). .

The number of target task states to be determined may not be limited to one. In step S308, the control unit 31 may use the determined target task state as the current task state to further determine the next target task state to transition to. The control unit 31 may determine the target task state multiple times by repeating this process. After determining the next target task state s _s(j) to transition to, the control unit 31 advances the process to the next step S309.

(Step S309)
In step S309, the control unit 31 operates as the command determination unit 317 and calculates the target value x _s(j) of the hand coordinates from the determined target task state s _s(j) . As shown in FIG. 17, the control unit 31 converts the target task state s _s(j) in the task space to the target coordinate x _s(j ) of the hand in the observation space by using the transformation function (ψ). ₎ can be converted to

That is, the conversion function (ψ) from the task space to the observation space when the end effector T does not hold the work W is given by the above equation (10). The control unit 31 substitutes the determined target task state s _s(j) into the above equation 10, and executes the arithmetic processing of the above equation 10 to calculate the target value x _s(j) of the hand coordinates. can do. On the other hand, the conversion function (ψ) from the task space to the observation space when the end effector T holds the work W is given by the above equation (11). The control unit 31 substitutes the determined target task state s _s(j) into the above equation 11 and executes the arithmetic processing of the above equation 11 to calculate the target value x _s(j) of the hand coordinates. can do. After calculating the target value x _s(j) of the hand coordinates, the control unit 31 proceeds to the next step S310.

(Step S310)
In step S310, the control unit 31 operates as the command determination unit 317, and determines the change amount (Δx(j)) of the hand coordinate from the current hand coordinate value x _(j) and the target hand coordinate value _{xs( j).} decide. Specifically, as shown in FIG. 17, the control unit 31 controls the _change amount ₍ Δx _(j) ). For example, the relationship between the deviation (x _s −x) of the current value of the hand coordinate and the target value and the amount of change (Δx) may be given by Equation 12 below. Note that the change amount (Δx) of the hand coordinates is an example of the difference between the current value and the target value of the hand coordinates.

αは任意の係数である。例えば、αの値は、１以下でかつ０を超える範囲内で適宜決定されてよい。αは省略されてよい。制御部３１は、ステップＳ３０７及びステップＳ３０９により得られた手先座標の現在値ｘ_(j)及び手先座標の目標値ｘ_s(j)を上記式１２に代入し、上記式１２の演算処理を実行することで、手先座標の変化量（Δｘ_(j)）を決定することができる。手先座標の変化量（Δｘ_(j)）を決定すると、制御部３１は、次のステップＳ３１１に処理を進める。

α is an arbitrary coefficient. For example, the value of α may be appropriately determined within a range of 1 or less and greater than 0. α may be omitted. The control unit 31 substitutes the current value x _(j) of the hand coordinates and the target value x _s(j) of the hand coordinates obtained in steps S307 and S309 into the above equation 12, and executes the arithmetic processing of the above equation 12. By doing so, the change amount (Δx _(j) ) of the hand coordinates can be determined. After determining the change amount (Δx _(j) ) of the hand coordinates, the control unit 31 advances the process to the next step S311.

(Step S311)
In step S311, the control unit 31 operates as the command determination unit 317, and performs the inverse kinematics calculation using the inverse function of the transform function in the forward kinematics calculation _. ), the amount of change in the angle of each joint (Δq _(j) ) is calculated. Specifically, the amount of change in hand coordinates (Δx) and the amount of change in angle of each joint (Δq) can be expressed by Equation 13 below.

Ｊは、上記順運動学計算における変換関数から導出されるヤコビ行列である。ｊ_iは、ｉ番目の関節の行列成分を示し、Δｑ_iは、ｉ番目の関節の変化量を示す。

J is the Jacobian matrix derived from the transformation function in the above forward kinematics calculation. j _i indicates the matrix element of the i-th joint, and Δq _i indicates the amount of change of the i-th joint.

Here, an example of a method for calculating the Jacobian matrix will be described with further reference to FIGS. 19A and 19B. FIG. 19A schematically illustrates an example of the relationship between each joint and the hand when the end effector T does not hold the work W. FIG. FIG. 19B schematically illustrates an example of the relationship between each joint and the hand while the end effector T holds the work W. FIG.

As shown in FIG. 19A, when the end effector T does not hold the work W, the component of each joint of the Jacobian matrix is calculated based on the positional relationship between each joint and the end effector T. For example, the control unit 31 can calculate the component of each joint using Equation 14 below. On the other hand, as shown in FIG. 19B, when the end effector T holds the work W, the component of each joint in the Jacobian matrix is calculated based on the positional relationship between each joint and the work W. For example, the control unit 31 can calculate the component of each joint using Equation 15 below.

ｚ_iは、ｉ番目の関節の同次座標における回転軸の成分を示し、ａ_iは、ｉ番目の関節の同次座標における平行移動成分を示す。ｚ_i及びａ_iは、ｉ番目の関節の第１同次変換行列から抽出される。ａ_tは、エンドエフェクタＴの同次座標における平行移動成分を示す。ａ_wは、ワークＷの同次座標における平行移動成分を示す。ａ_tは、エンドエフェクタＴの座標（Ｔ_t）から抽出される。ａ_wは、ワークＷの座標（Ｔ_w）から抽出される。ヤコビ行列の各成分ｊ_iは、各関節の第１同次変換行列の微分成分を示す。

Z _i indicates the rotation axis component in the homogeneous coordinates of the i-th joint, and a _i indicates the translation component in the homogeneous coordinates of the i-th joint. The z _i and a _i are extracted from the first homogeneous transformation matrix of the i th joint. a _t indicates the translational component of the end effector T in homogeneous coordinates. a _w represents a translation component in the homogeneous coordinates of the workpiece W; a _t is extracted from the coordinates of the end effector T (T _t ). a _w is extracted from the coordinates of the workpiece W (T _w ). Each component j _i of the Jacobian matrix represents a differential component of the first homogeneous transformation matrix of each joint.

The control unit 31 calculates the Jacobian matrix according to the operation mode according to Equations 14 and 15 above. In the present embodiment, the components of the end effector T ( a _t ) and the components of the workpiece W (a _w ) are simply interchanged. Therefore, the control unit 31 can calculate the Jacobian matrix in each case by simple calculation processing.

Next, the control unit 31 calculates an inverse matrix (J ⁻¹ ) of the calculated Jacobian matrix. The control unit 31 performs inverse kinematics calculation using the calculated inverse matrix (J ⁻¹ ). Specifically, the relationship between each amount of change (Δx, Δq) and the inverse matrix (J ⁻¹ ) is derived from Equation 13 above as Equation 16 below.

制御部３１は、算出された逆行列（Ｊ^-1）及び手先座標の変化量（Δｘ_(j)）を式１６に代入し、上記式１６の演算処理を実行することで、各関節の角度の変化量（Δｑ_(j)）を算出することができる。各関節の角度の変化量（Δｑ_(j)）を算出すると、制御部３１は、次のステップＳ３１２に処理を進める。

The control unit 31 substitutes the calculated inverse matrix (J ⁻¹ ) and the amount of change in the hand coordinates (Δx _(j) ) into Equation 16, and executes the arithmetic processing of Equation 16 to obtain the angle of each joint. can be calculated (Δq _(j) ). After calculating the amount of change in the angle of each joint (Δq _(j) ), the control unit 31 proceeds to the next step S312.

(Step S312)
In step S312, the control unit 31 operates as the command determination unit 317, and determines a command value for each joint based on the calculated amount of change in the angle of each joint. A known method such as PID (Proportional-Integral-Differential) control, PI control, or the like may be employed as a method for determining the command value. A command value for each joint is an example of a control command given to the manipulator 4 . In this embodiment, the control unit 31 causes the hand coordinates to approach the target value (furthermore, changes the task state of the manipulator 4 from the current task state s _(j) to the target task It is possible to determine the control commands to give to the manipulator 4, such as changing to state s _s(j) . After determining the control command, the control unit 31 advances the process to the next step S313.

(Step S313)
In step S<b>313 , the control unit 31 operates as the driving unit 318 and drives the manipulator 4 by giving the determined control command to the manipulator 4 . In this embodiment, the controller 31 drives each joint of the manipulator 4 according to each determined command value. Note that the driving method is not particularly limited, and may be appropriately selected according to the embodiment. For example, the controller 31 may directly drive each joint of the manipulator 4 . Alternatively, manipulator 4 may comprise a controller (not shown). In this case, the control unit 31 may indirectly drive each joint of the manipulator 4 by giving a command value for each joint to the controller. After driving the manipulator 4 according to the determined control command, the control unit 31 proceeds to the next step S314.

(Steps S314 to S316)
The processing of steps S314 to S316 is the same as the processing of steps S303, S306 and S307 except that the cycle proceeds from (j) to (j+1). That is, in step S314, the control unit 31 acquires each sensing data (323, 324) from each sensor system. In step S<b>315 , the control unit 31 operates as the state acquisition unit 315 and acquires the current task state s _(j+1) of the manipulator 4 . In step S316, the control unit 31 operates as each estimation unit (313, 314) and calculates each estimated value of the current hand coordinates of the manipulator 4 from each acquired sensing data (323, 324). The control unit 31 recognizes the current hand coordinate value x _(j+1) based on at least one of the calculated first estimated value and second estimated value. Thus, when the current value x _(j+1) of the hand coordinates is acquired, the control unit 31 advances the process to the next step S317.

(Step S317)
In step S317, the control unit 31 determines whether or not the task state of the manipulator 4 has transitioned to the target task state s _s(j) as a result of the driving in step S313.

The determination method is not particularly limited, and may be appropriately selected according to the embodiment. For example, as shown in FIG. 17, the relationship between the angle of each joint after driving (q _(j+1) ) and the angle of each joint before driving (q _{(j) )} is expressed by Equation 17 below. be able to.

そこで、制御部３１は、ステップＳ３１４において各エンコーダＳ２により得られた各関節の角度の値が、駆動前に各エンコーダＳ２により得られた各関節の角度の値（ｑ_(j)）及びステップＳ３１１において算出された変化量（Δｑ_(j)）の和と一致するか否かを判定してもよい。駆動後の各関節の角度が駆動前の各関節の角度及び算出された変化量の和（ｑ_(j)＋Δｑ_(j)）と一致する場合、制御部３１は、マニピュレータ４のタスク状態が目標のタスク状態ｓ_s(j)に遷移したと判定してもよい。一方、そうではない場合、制御部３１は、マニピュレータ４のタスク状態が目標のタスク状態ｓ_s(j)に遷移していないと判定してもよい。

Therefore, the control unit 31 converts the joint angle value obtained by each encoder S2 in step S314 into the joint angle value (q _(j) obtained by each encoder S2 before driving and step S311). It may be determined whether or not it matches the sum of the amount of change (Δq _(j) ) calculated in . When the angle of each joint after driving matches the angle of each joint before driving and the sum of the calculated amount of change (q _(j) +Δq _(j) ), the control unit 31 determines that the task state of the manipulator 4 is the target may be determined to have transitioned to the task state s _s(j) of . On the other hand, otherwise, the control unit 31 may determine that the task state of the manipulator 4 has not transitioned to the target task state s _s(j) .

Also, for example, the Jacobian matrix J _ψ may be derived for the transformation function (ψ) as well as the transformation function in the forward kinematics calculation. The Jacobian matrix J _ψ indicates the differential component of the transformation function (ψ). An inverse matrix (J _ψ ^-1 ) may be calculated from the derived Jacobian matrix J _ψ . The relationship between the amount of change (Δx) in the hand coordinates and the amount of change (Δs) in the task state and the inverse matrix (J _ψ ⁻¹ ) can be expressed by Equation 18 below.

制御部３１は、算出された逆行列（Ｊ_ψ ^-1）及び手先座標の変化量（Δｘ_(j)）を式１８に代入し、上記式１８の演算処理を実行することで、タスク状態の変化量（Δｓ_(j)）を算出することができる。駆動後のタスク状態ｓ_(j+1)と駆動前のタスク状態ｓ_(j)との関係との関係は、上記式１７と同様に、以下の式１９により表現することができる。

The control unit 31 substitutes the calculated inverse matrix (J _ψ ^-1 ) and the amount of change in the hand coordinates (Δx _(j) ) into Equation 18, and executes the arithmetic processing of Equation 18, thereby determining the task state. The amount of change (Δs _(j) ) can be calculated. The relationship between the task state s _(j+1) after driving and the task state s _(j) before driving can be expressed by the following equation 19, like equation 17 above.

そこで、制御部３１は、ステップＳ３１５により駆動後に得られた現在のタスク状態が、ステップＳ３０６により駆動前に得られた現在のタスク状態ｓ_(j)及び上記により算出された変化量（Δｓ_(j)）の和と一致するか否かを判定してもよい。駆動後に得られた現在のタスク状態が、駆動前に得られた現在のタスク状態及び算出された変化量の和（ｓ_(j)＋Δｓ_(j)）と一致する場合、制御部３１は、マニピュレータ４のタスク状態が目標のタスク状態ｓ_s(j)に遷移したと判定してもよい。一方、そうではない場合、制御部３１は、マニピュレータ４のタスク状態が目標のタスク状態ｓ_s(j)に遷移していないと判定してもよい。なお、本実施形態では、タスク空間は２つの対象物間の相対座標で規定されるため、タスク空間及び観測空間は互いに共通の次元で表現可能である。そのため、場合によっては、式１８の逆行列（Ｊ_ψ ^-1）は、単位行列に置き換えられ、手先座標の変化量（Δｘ）がそのままタスク状態の変化量（Δｓ）として取り扱われてもよい。一例として、他のワークＧから見たワークＷの相対座標によりタスク状態を規定した場合、式１８の逆行列（Ｊ_ψ ^-1）は、単位行列に置き換えられてよい。

Therefore, the control unit 31 determines that the current task state obtained after driving in step S315 is the current task state s _(j) obtained before driving in step S306 and the amount of change (Δs _{(j )} and ). If the current task state obtained after driving matches the sum of the current task state obtained before driving and the calculated amount of change (s _(j) +Δs _(j) ), the control unit 31 controls the manipulator 4 has transitioned to the target task state s _{s (j)} . On the other hand, otherwise, the control unit 31 may determine that the task state of the manipulator 4 has not transitioned to the target task state s _s(j) . In this embodiment, since the task space is defined by relative coordinates between two objects, the task space and the observation space can be expressed in a common dimension. Therefore, in some cases, the inverse matrix (J _ψ ⁻¹ ) of Equation 18 may be replaced with a unit matrix, and the amount of change (Δx) in hand coordinates may be treated as it is as the amount of change (Δs) in task state. As an example, when the task state is defined by the relative coordinates of a work W viewed from another work G, the inverse matrix (J _ψ ⁻¹ ) of Equation 18 may be replaced with a unit matrix.

Alternatively, the control unit 31 may determine whether the current task state obtained in step S315 matches the target task state s _s(j) determined in step S308. If the obtained current task state matches the target task state s _s(j) , the control unit 31 determines that the task state of the manipulator 4 has transitioned to the target task state s _s(j). good. On the other hand, otherwise, the control unit 31 may determine that the task state of the manipulator 4 has not transitioned to the target task state s _s(j) .

Further, for example, the relationship between the current value of the hand coordinates after driving (x _(j+1) ) and the current value of the hand coordinates before driving (x _(j) ) can be expressed by the following equation, similar to Equation 17 above. 20.

そこで、制御部３１は、ステップＳ３１６により取得された駆動後の手先座標の現在値が、ステップＳ３０７により取得された駆動前の手先座標の現在値（ｘ_(j)）とステップＳ３１０により決定された変化量（Δｘ_(j)）の和と一致するか否かを判定してもよい。駆動後の手先座標の現在値が駆動前の手先座標の現在値及び算出された変化量の和（ｘ_(j)＋Δｘ_(j)）と一致する場合、制御部３１は、マニピュレータ４のタスク状態が目標のタスク状態ｓ_s(j)に遷移したと判定してもよい。一方、そうではない場合、制御部３１は、マニピュレータ４のタスク状態が目標のタスク状態ｓ_s(j)に遷移していないと判定してもよい。

Therefore, the control unit 31 determines the current value of the hand coordinates after driving obtained in step S316 from the current value of the hand coordinates before driving (x _(j) ) obtained in step S307 and the current value of the hand coordinates before driving obtained in step S310. It may be determined whether or not it matches the sum of the amount of change (Δx _{(j) )} . If the current value of the hand coordinates after driving matches the sum of the current value of the hand coordinates before driving and the calculated amount of change (x _(j) +Δx _(j) ), the control unit 31 changes the task state of the manipulator 4 to has transitioned to the target task state s _s(j) . On the other hand, otherwise, the control unit 31 may determine that the task state of the manipulator 4 has not transitioned to the target task state s _s(j) .

Alternatively, the control unit 31 may determine whether or not the current value of the hand coordinates obtained in step S316 matches the target value of the hand coordinates (x _s(j) ) calculated in step S309. . If the current value of the hand coordinates after driving matches the target value (x _s(j) ) of the hand coordinates calculated before driving, the control unit 31 determines that the task state of the manipulator 4 is the target task state s _s( It may be determined that the transition to _j) has occurred. On the other hand, otherwise, the control unit 31 may determine that the task state of the manipulator 4 has not transitioned to the target task state s _s(j) .

By any of the above methods, the control unit 31 can determine whether or not the task state of the manipulator 4 has transitioned to the target task state s _s(j) . In addition, in each of the above determinations, "matching" may include not only that the values of both completely match, but also that the difference between the values of both is equal to or less than a threshold (permissible error). When determining that the task state of the manipulator 4 has transitioned to the target task state s _{s (j)} , the control unit 31 advances the process to the next step S318. On the other hand, otherwise, the controller 31 returns to step S310 and drives the manipulator 4 again. At this time, the control unit 31 may use the current value of the hand coordinates calculated in step S316 as the current value x _(j) to execute the processes after step S310.

(Step S318)
In step S318, the control unit 31 determines whether or not the final target task state s _g has been achieved.

The determination method is not particularly limited, and may be appropriately selected according to the embodiment. For example, the control unit 31 may determine whether or not the current task state s _(j+1) obtained in step S315 matches the final target task state s _g . When the current task state s _(j+1) matches the final target task state s _g , the control unit 31 determines that the final target task state s _g has been achieved. On the other hand, otherwise, the control unit 31 determines that the final target task state s _g cannot be achieved. Similar to the above, "matching" in the determination may include not only the values of both being completely matching, but also the difference between the values being equal to or less than a threshold (permissible error).

When determining that the final target task state s _g has been achieved, the control unit 31 terminates a series of processes related to motion control of the manipulator 4 . On the other hand, when determining that the final target task state s _g cannot be achieved, the control unit 31 returns the process to step S308. Then, the control unit 31 uses the results of steps S315 and S316 to execute the processes of steps S308 to S313 again. The control unit 31 realizes the final target task state s _g by repeating the series of processes described above. Thereby, the control device 3 according to this embodiment can control the operation of the manipulator 4 so as to perform the designated task.

Note that the branch destination when it is determined that the final target task state s _g cannot be realized does not have to be limited to step S308. For example, when the manipulator 4 is caused to perform a series of tasks composed of a plurality of tasks, the final goal task state s _g may be set to the final goal task state of the last task to be performed. In this embodiment, when a task of holding a work W by the end effector T and assembling the held work W to another work G is carried out, the final target task state s _g may be adopted. In this case, the performance of the series of tasks may start from the starting point of the first task. Accordingly, when it is determined that the final target task state s _g cannot be realized, the branch destination may be step S303 instead of step S308. As a result, the control unit 31 can drive the manipulator 4 while confirming the operation mode through the processing of steps S304 and S305. As a result, a series of tasks can be accomplished while switching tasks smoothly. In this embodiment, when the work W is held by the end effector T, the operation mode can be smoothly switched to the task of transporting the work W to another work G. FIG.

(B) Adjustment Processing Next, an operation example regarding parameter adjustment of the estimation models (61, 62) of the control device 3 according to the present embodiment will be described with reference to FIG. FIG. 20 is a flowchart showing an example of a processing procedure regarding parameter adjustment of each estimation model (61, 62) by the control device 3 according to this embodiment. In this embodiment, the first estimation model 61 is a transformation function (first transformation matrix group or second transformation matrix group) used for the forward kinematics calculation. Also, the second estimation model 62 is a transformation function (ψ) or each coordinate (T _t , T _w ) that transforms values in the task space into values in the observation space. The information processing related to parameter adjustment may be executed together with the information processing related to motion control of the manipulator 4, or may be executed separately. The processing procedures described below, including the processing procedures relating to the above operation control, are examples of the "control method" of the present invention. However, each processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, for each processing procedure described below, it is possible to omit, replace, or add steps as appropriate according to the embodiment.

(Step S401 and Step S402)
In step S401, the control unit 31 operates as each data acquisition unit (311, 312) and acquires each sensing data (323, 324) from each sensor system. The processing of step S401 is the same as the processing of steps S303 and S314. In step S402, the control unit 31 operates as each estimation unit (313, 314) and utilizes each estimation model (61, 62) to determine the current hand coordinates from each acquired sensing data (323, 324). Calculate each estimated value of The processing of step S402 is the same as the processing of steps S307 and S316. After calculating each estimated value, the control unit 31 advances the process to the next step S403.

Note that when the information processing regarding this parameter adjustment is executed together with the information processing regarding the operation control, the execution of step S401 corresponds to the execution of step S303 or step S314. Execution of step S402 corresponds to execution of step S307 or step S316. In this case, after executing step S307 or step S316, the control unit 31 may next execute the process of step S403 at any timing.

(Steps S403 and S404)
In step S403, the control unit 31 operates as the adjustment unit 319 and calculates the gradient of the error between the calculated first estimated value and second estimated value. A functional expression such as an error function may be used to calculate the error. For example, the control unit 31 calculates the difference between the first estimated value and the second estimated value, calculates the power (for example, squares) of the calculated difference, and acquires the obtained value as an error. can be done. Then, the control unit 31 can calculate the gradient of the error regarding each parameter of each estimation model (61, 62) by calculating a partial derivative of the calculated error.

In step S404, the control unit 31 operates as the adjustment unit 319, and adjusts the first estimation model 61 and the second estimation model 61 based on the calculated gradient so that the error between the first estimated value and the second estimated value becomes small. 2 Adjust the value of at least one parameter of the estimation model 62 . As an example, the control unit 31 updates the value of each parameter by subtracting the gradient calculated for each parameter from the value of each parameter. Thereby, the controller 31 can adjust the value of each parameter based on the calculated slope.

The parameter values of both estimation models (61, 62) may be adjusted. Alternatively, the parameter values of only one of the estimation models (61, 62) may be adjusted. It is preferable to adjust the parameters of both estimation models (61, 62) if noise may occur in both, or if both estimation models (61, 62) may have incorrect parameters. When the parameter adjustment is completed, the control unit 31 advances the process to the next step S405.

(Step S405)
In step S405, the control unit 31 determines whether or not to end the process of adjusting the parameters of each estimation model (61, 62). A criterion for terminating the parameter adjustment process may be appropriately determined according to the embodiment.

For example, a specified number of repetitions of parameter adjustment may be set until completion. The specified number of times may be given by a set value or may be given by an operator's designation, for example. In this case, the control unit 31 determines whether or not the number of times the processes of steps S401 to S404 have been executed has reached a specified number of times. When determining that the number of times of execution has not reached the specified number of times, the control unit 31 returns the process to step S401 and repeats the processes of steps S401 to S404. On the other hand, when determining that the number of times of execution has reached the specified number of times, the control unit 31 terminates a series of processes related to parameter adjustment based on the gradient of the error between the first estimated value and the second estimated value.

Also, for example, the control unit 31 may inquire of the operator whether to repeat the process. In this case, the control unit 31 determines whether or not to repeat the processing related to parameter adjustment according to the operator's answer. When the operator replies that the process will be repeated, the control unit 31 returns the process to step S401 and repeats the processes of steps S401 to S404. On the other hand, if the operator answers not to repeat the process, the control unit 31 terminates a series of processes related to parameter adjustment based on the gradient of the error between the first estimated value and the second estimated value.

<Adjustment processing when contact occurs>
Next, with reference to FIG. 21, an operation example regarding parameter adjustment of each estimation model (61, 62) by another method of the control device 3 according to the present embodiment will be described. FIG. 21 is a flowchart showing an example of a processing procedure regarding parameter adjustment of each estimation model (61, 62) by another method.

The control device 3 according to the present embodiment executes the parameter adjustment process illustrated in FIG. 21 when the hand of the manipulator 4 touches some object, in addition to the parameter adjustment of the method illustrated in FIG. . In this embodiment, when the end effector T does not hold the work W, the end effector T is the tip of the manipulator 4, and the work W is an example of the object with which the tip of the manipulator 4 contacts. On the other hand, when the end effector T holds the work W, the work G is the tip of the manipulator 4, and the other work G is an example of the object with which the tip of the manipulator 4 comes into contact.

The information processing related to parameter adjustment of the method illustrated in FIG. 21 may be executed together with the information processing related to motion control of the manipulator 4, or may be executed separately. The processing procedures described below, including the processing procedures relating to the above operation control, are examples of the "control method" of the present invention. However, each processing procedure described below is merely an example, and each step may be changed as much as possible. Furthermore, for each processing procedure described below, it is possible to omit, replace, or add steps as appropriate according to the embodiment.

(Steps S411 and S412)
In step S411, the control unit 31 operates as each data acquisition unit (311, 312) and acquires each sensing data (323, 324) from each sensor system. The processing of step S411 is the same as the processing of step S401. In step S412, the control unit 31 operates as each estimation unit (313, 314), uses each estimation model (61, 62), and calculates the current hand coordinates from each acquired sensing data (323, 324). Calculate each estimated value of The processing of step S412 is the same as the processing of step S402. After calculating each estimated value, the control unit 31 advances the process to the next step S413.

(Step S413)
In step S413, the control unit 31 operates as the adjustment unit 319 and acquires the boundary value of the hand coordinates on the contact boundary surface with the object. A method for acquiring the boundary value of the hand coordinates may not be particularly limited, and may be appropriately selected according to the embodiment. For example, the boundary value of the hand coordinates may be obtained by the operator's designation via the input device 35 . Also, in this embodiment, the task state is defined by the relative coordinates between the hand of the manipulator 4 and the target. The boundary value of the hand coordinates may be obtained by using the task space SP representing the set of task states (relative coordinates).

Here, an example of a method of acquiring the boundary value of the hand coordinates using the task space SP will be described with reference to FIG. FIG. 22 schematically illustrates an example of a scene in which a boundary value is obtained on a contact boundary surface in the task space SP. First, the control unit 31 inputs each estimated value of the hand coordinates calculated in step S412 to the inverse function of the transformation function (ψ), and performs arithmetic processing of the inverse function, thereby corresponding to each estimated value. Calculate the coordinates in the task space SP. In the example of FIG. 22, the node Ne1 indicates the coordinates corresponding to the first estimated value, and the node Ne2 indicates the coordinates corresponding to the second estimated value.

Next, the control unit 31 derives a contact boundary surface in the task space SP. A learned judgment model 50 may be used to derive the contact boundary surface. In this case, the control device 3 (storage unit 32 ) may include the learned determination model 50 by holding the learning result data 125 . Alternatively, the control unit 31 may acquire the interface derivation result from the second model generation device 2 by inquiring of the second model generation device 2 via the network.

Subsequently, the control unit 31 controls a node near at least one of the coordinates corresponding to the first estimated value (node Ne1) and the coordinates corresponding to the second estimated value (node Ne2) on the derived contact boundary surface. Select Nb. For example, the control unit 31 may select a node closest to both nodes (Ne1, Ne2) as the node Nb. The control unit 31 inputs the coordinates of the selected node Nb to the transformation function (ψ) and executes the arithmetic processing of the transformation function (ψ), thereby calculating the boundary value of the hand coordinates.

By any of the above methods, the control unit 31 can acquire the boundary value of the hand coordinates. After acquiring the boundary value of the hand coordinates, the control unit 31 advances the process to the next step S414. Note that the method of acquiring the boundary value of the hand coordinates need not be limited to these examples. For example, the control unit 31 may directly derive the interface of contact in the observation space. The control unit 31 may select a point near at least one of the coordinates of the first estimated value and the coordinates of the second estimated value on the derived contact boundary surface. Then, the control unit 31 may acquire the coordinates of the selected point as the boundary value.

(Step S414 and Step S415)
In step S414, the control unit 31 operates as the adjustment unit 319 and calculates the gradient of the first error between the first estimated value estimated at the time of contact and the acquired boundary value. The control unit 31 also calculates the gradient of the second error between the second estimated value estimated at the time of contact and the acquired boundary value. The method for calculating each gradient may be the same as in step S403 above.

In step S415, the control unit 31 operates as the adjustment unit 319, and adjusts the parameter values of the first estimation model 61 based on the calculated slope of the first error so that the first error becomes smaller. Also, based on the calculated gradient of the second error, the control unit 31 adjusts the parameter values of the second estimation model 62 so that the second error becomes smaller. The method for adjusting the parameter values may be the same as in step S404 above. When the parameter adjustment is completed, the control unit 31 terminates a series of processes related to parameter adjustment using the boundary value.

<Processing timing>
Next, an example of the relationship between the timing of the operation control and the timing of parameter adjustment will be described with reference to FIG. FIG. 23 schematically illustrates an example of the relationship between the timing of operation control and the timing of parameter adjustment.

The processing cycle of each of the first sensor system and the second sensor system, in other words, the cycle of acquiring each sensing data (323, 324) is not necessarily the same. If the respective processing cycles are different, the information processing regarding the motion control of the manipulator 4 may be executed at the timing when sensing data can be obtained from at least one of the first sensor system and the second sensor system. On the other hand, the control unit 31 may perform the above parameter adjustment at a timing when both sensing data (323, 324) can be acquired.

In the example of FIG. 23, it is assumed that the processing cycle of the first sensor system composed of the encoders S2 and the tactile sensors S3 is shorter than the processing cycle of the second sensor system composed of the camera S1. As an example, assume that the processing cycle of the first sensor system is 10 ms (milliseconds) and the processing cycle of the second sensor system is 30 ms. In this case, the first sensing data 323 can be acquired three times by the first sensor system while the second sensing data 324 is acquired once by the second sensor system.

In the example of FIG. 23, at the timing when only the first sensing data 323 can be acquired from the first sensor system, the control unit 31 uses the first estimated value of the current hand coordinates calculated from the first sensing data 323. may be used to control the operation of the manipulator 4 . On the other hand, at the timing when both sensing data (323, 324) can be acquired from the first sensor system and the second sensor system, the control unit 31 calculates the first estimated value and the second estimated value of the current hand coordinates, respectively. At least one of the values may be used to control the operation of manipulator 4 . Also, at this timing, the control unit 31 may execute the parameter adjustment information processing shown in FIG.

Furthermore, when the hand of the manipulator 4 touches some object, the control unit 31 stops the operation control of the manipulator 4, and both sensing data (323, 324) from the first sensor system and the second sensor system are collected. can be acquired. Then, at the timing when both sensing data (323, 324) can be acquired, the control unit 31 may execute the parameter adjustment information processing shown in FIG. In addition, the control unit 31 may also execute the information processing for parameter adjustment shown in FIG. When both parameter adjustments shown in FIGS. 20 and 21 are performed, step S401 may be performed as common processing with step S411, and step S402 may be performed as common processing with step S412. As a result, the operation of the manipulator 4 can be controlled, and the parameter values of each estimation model (61, 62) can be adjusted at appropriate timing.

After adjusting the parameter values of each estimation model (61, 62) by each method shown in FIG. 20 and FIG. Whether or not may be evaluated as appropriate. As an example, noise that can be included in each sensing data (323, 324) obtained by each sensor system is assumed to be white noise. Therefore, by acquiring sensing data for at least one predetermined period of time and averaging the acquired sensing data, noise contained in the sensing data can be removed or reduced. Each estimated value by each estimation model (61, 62) approaches the true value depending on whether or not the averaged sensing data includes a component related to each estimated value by each estimation model (61, 62). It is possible to evaluate whether or not

For example, assume that a depth camera is used as the camera S<b>1 and a depth map (image data including depth information) is acquired as the second sensing data 324 . In this case, the control unit 31 plots each estimated value on the depth map and compares the coordinates corresponding to each estimated value with the coordinates of the hand of the manipulator 4 on the depth map. As a result of this comparison, when the coordinates match (or approximate), the control unit 31 evaluates that each estimated value by each estimation model (61, 62) is close to the true value. be able to. On the other hand, when the coordinates are deviated from each other, there is a possibility that each estimated value by each estimation model (61, 62) is not close to the true value. In this case, the control unit 31 may repeat the parameter adjustment by at least one of the methods of FIGS. 20 and 21 until it can be evaluated that each estimated value is close to the true value.

[feature]
As described above, in the present embodiment, in step S404, the values of the parameters of at least one of the first estimation model 61 and the second estimation model 62 are adjusted so that the estimation results (estimated values) of each other approach one value. adjust. Further, when the hand of the manipulator 4 touches the object, in step S415, the values of the parameters of each estimation model (61, 62) are adjusted so that each estimated value approaches the contact boundary value. These adjustments are expected to improve the estimation accuracy of the hand coordinates by each estimation model (61, 62). In particular, according to step S415, since there is a physical constraint of contact with the object, by adjusting the parameter values of each estimation model (61, 62) based on highly accurate information (boundary values), , the estimation accuracy of the hand coordinates by each estimation model (61, 62) can be improved. Therefore, according to this embodiment, it is possible to improve the accuracy of controlling the hand coordinates of the manipulator 4 .

Further, in the present embodiment, the first estimated value of the current hand coordinates of the manipulator 4 can be calculated by forward kinematics calculation in steps S307 and S316. When the end effector T does not hold the workpiece W, the end effector T is set at the end of the hand, and the forward kinematics calculation is performed using the first homogeneous transformation matrix of each joint (joints 41 to 46). One transformation matrix group (φ) is used as the transformation function. On the other hand, when the end effector T holds the work W, the work W is set at the end, and the conversion function used for the forward kinematics calculation is extended. Specifically, in the forward kinematics calculation, the second homogeneous transformation matrix ( _tTw ) for transforming the coordinates from the coordinate system of the end effector T to the coordinate system ^of the work W is combined with the first transformation matrix group (φ ₎ is used as a transformation ^function . That is, in this embodiment, when the work W is held by the end effector T, the kinematic reference point is changed from the end effector T to the work W. FIG.

As a result, the forward kinematics calculation in steps S307 and S316 and the inverse kinematics calculation in step S311 are substantially the same whether the end effector T does not hold the work W or holds the work W. can be processed. In other words, the first task of holding the work W by the end effector T and the second task of assembling the work W held by the end effector T to another work G are common to "moving the hand of the manipulator 4 with respect to the target". can be treated as a task of Therefore, according to the present embodiment, the control process is defined in a general and uniform manner without distinguishing between the case where the end effector T does not hold the work W and the case where the end effector T holds the work W. can do. Therefore, the control process can be simplified, thereby reducing the cost of generating or teaching the movement of the manipulator 4 . In the above embodiment, the work W is held by the end effector T, and the cost of creating or teaching a series of operations for assembling the held work W to another work G can be reduced.

Also, in this embodiment, the state of the task executed by the manipulator 4 is represented by the relative positional relationship among objects such as the end effector T (end effector), the work W, another work G, and the like. Thereby, the control command is not directly related to the task, but is related to the amount of change in the relative positional relationship between the objects. That is, it is possible to generate or teach a time-series control command to be given to the manipulator 4 for changing the relative positional relationship of the object without depending on the content of the task. For example, even if the coordinates of the work W change, when grasping the positional relationship (task state) between the end effector T and the work W in steps S306 and S315, the change in the coordinates of the work W is considered. Therefore, the manipulator 4 can appropriately hold the workpiece W with the end effector T based on the learning result. Therefore, according to this embodiment, it is possible to increase the versatility of the ability to perform the learned task, thereby reducing the cost of teaching the manipulator 4 the task.

Further, in this embodiment, the positional relationship between objects is represented by relative coordinates. This makes it possible to express the positional relationship between the two objects appropriately and concisely. Therefore, it is possible to easily grasp the positional relationship (task state in the case of control) between the two objects.

In addition, the first model generation device 1 according to the present embodiment performs machine learning through the processes of steps S101 and S102 to determine whether or not two objects are in contact with each other in terms of the positional relationship of the objects. A judgment model 50 is generated for According to the learned judgment model 50 generated by machine learning, even if the positional relationship of the objects is given as a continuous value, the positional relationship between the two objects can be determined without a large increase in the amount of data of the judgment model 50. It can be determined whether objects touch each other. Therefore, according to this embodiment, it is possible to greatly reduce the data amount of information representing the boundary where two objects come into contact.

Here, a specific example of this action and effect will be described with reference to FIG. FIG. 24 schematically illustrates an example of a form in which a value indicating whether or not two objects are in contact with each other is held for each coordinate point. A white circle indicates that the two objects do not contact each other in the positional relationship corresponding to the coordinates, and a black circle indicates that the two objects contact each other in the positional relationship corresponding to the coordinates. In FIG. 24, each coordinate point is expressed in two dimensions, but in the six-dimensional relative coordinate space, each coordinate point is expressed in six dimensions. In this case, if the spatial resolution is increased, the amount of data will increase in the order of the sixth power. For example, if a coordinate point is set with a resolution that can be used for operation in real space, the data amount of the information can easily reach gigabytes.

On the other hand, in the present embodiment, the learned determination model 50 holds information indicating whether or not two objects come into contact with each other in the positional relationship of the objects. Although the number of calculation parameters of this learned judgment model 50 can depend on the number of dimensions of the relative coordinates, continuous values can be handled without increasing the number of calculation parameters. Therefore, as will be described later, for example, when the judgment model 50 is configured by a neural network with a three-layer structure, the data amount of the learned judgment model 50 can be suppressed to about several megabytes. Therefore, according to this embodiment, it is possible to greatly reduce the data amount of information representing the boundary where two objects come into contact.

Further, in the present embodiment, the hand of the manipulator 4 and the target are two objects for which it is determined whether or not contact will occur by the learned determination model 50 . Therefore, in a scene that defines the operation of the manipulator 4, the data amount of information expressing the boundary where two objects come into contact can be greatly reduced. In the second model generation device 2, even if the capacities of the RAM, ROM, and storage unit 22 are relatively small, it is possible to use the learned judgment model 50, thereby preventing the hand from contacting the target unnecessarily. An inference model 55 can be generated to determine the target task state so that it does not.

In addition, the second model generating device 2 according to the present embodiment uses the learned determination model 50 through the processing of steps S201 to S210 to prevent the first object from coming into contact with the second object. generates an inference model 55 for determining the task state of The control device 3 according to the present embodiment uses the generated inference model 55 to determine the target task state in step S308. As a result, the control device 3 according to the present embodiment prevents the first object from coming into contact with the second object, that is, the hand of the manipulator 4 does not come into contact with the target object, even without the arithmetic processing of the learned determination model 50 . It is possible to determine the target task state so as not to touch the Therefore, it is possible to reduce the calculation cost of the motion control of the manipulator 4 .

§4 Modifications Although the embodiments of the present invention have been described in detail, the above description is merely an example of the present invention in every respect. It goes without saying that various modifications or variations can be made without departing from the scope of the invention. For example, the following changes are possible. In addition, below, the same code|symbol is used about the component similar to the said embodiment, and description is abbreviate|omitted suitably about the point similar to the said embodiment. The following modified examples can be combined as appropriate.

<4.1>
In the above embodiments, the end effector T, the work W, and the other work G are examples of objects. In particular, another work G is an example of an object to which the work W is attached. When the end effector T does not hold the work W, the work W is an example of the object with which the hand of the manipulator 4 contacts. It is an example of an object with which the hand of the manipulator 4 contacts. However, the target object does not have to be limited to such an example. Objects may include any kind of object that can be manipulated in real or virtual space. In addition to the end effector T, workpiece W, and other workpiece G, the target may be an object such as an obstacle that can be related to the operation of the manipulator.

Note that one target object may be composed of one object, or may be composed of a plurality of objects. If there are three or more objects, the decision model 50 is configured to treat the objects as one object and to determine whether contact occurs between the objects and other objects. may Alternatively, the determination model 50 may be configured to regard individual objects as one object and determine whether or not contact occurs between the respective objects.

Also, in the above embodiments, at least one of the two objects is an object that is moved by the operation of the manipulator. The object moved by the operation of the manipulator may be, for example, a component of the manipulator such as an end effector, the manipulator itself, or a configuration of the manipulator such as a workpiece held by the end effector. Objects other than elements may be used. However, the type of target does not have to be limited to such an example. Both of the two objects may be objects other than objects that are moved by manipulator operations.

Moreover, in the above embodiment, the manipulator 4 is a vertical articulated robot. However, when estimating the hand coordinates of the manipulator 4 from the current values of the angles of the joints obtained by the encoder S2, the type of the manipulator 4 is not particularly limited as long as the manipulator 4 has one or more joints. may be selected as appropriate according to the embodiment. In other cases, the manipulator 4 may comprise components other than joints. The manipulator 4 may include a SCARA robot, a parallel link robot, an orthogonal robot, a cooperative robot, etc., in addition to the vertically articulated robot. Further, in the above embodiment, the control command is composed of the command value for the angle of each joint. However, the configuration of the control command need not be limited to such an example, and may be appropriately determined according to the type of manipulator 4 .

Further, in the above embodiment, the work of holding the work W by the end effector T and the work of assembling the held work W to another work G are examples of tasks performed by the manipulator. The type of task is not particularly limited as long as at least part of the process involves movement of the manipulator's hand, and may be appropriately selected according to the embodiment. In addition to holding the workpiece W and transporting the workpiece W, the task may be, for example, fitting parts, screwing, and the like. A task may be, for example, a simple task such as hold work, release work. The task may be, for example, holding the target work and changing the coordinates of the target work, such as placing the target work at specified coordinates (position and orientation). A task may be, for example, using a sprayer as an end effector and spraying a workpiece with the sprayer from specified relative coordinates. A task may also be, for example, to position a camera attached to an end effector at specified coordinates. A task may be given in advance or may be given by an operator's designation.

Further, in the above embodiment, the first sensor system is composed of the encoders S2 and the tactile sensors S3. Also, the second sensor system is composed of the camera S1. However, as long as each sensor system can observe the hand of the manipulator 4, the type of sensor used for each sensor system need not be limited to such an example, and can be appropriately selected according to the embodiment. you can At least some sensors may be used in common between the first sensor system and the second sensor system. The sensor may be a camera, an encoder, a tactile sensor, a proximity sensor, a force sensor, a torque sensor, a pressure sensor, or the like. The proximity sensor may be arranged in an observable range around the end effector T and used to observe the presence or absence of an object approaching the end effector T. Further, the force sensor, torque sensor, and pressure sensor are arranged within a range where the force acting on the end effector T can be measured in the same manner as the tactile sensor S3, and are used to observe the force acting on the end effector T. may be At least one of a proximity sensor, a force sensor, a torque sensor, and a pressure sensor may be used as a sensor for observing the state of the work W with respect to the end effector T. Note that the camera S1 may be configured to be arbitrarily movable by the manipulator 4 or another robot device. In this case, the coordinates of camera S1 may be calibrated accordingly. Thereby, the range observed by the camera S1 can be arbitrarily controlled.

In the above embodiment, the tactile sensor S3 may be omitted from the first sensor system. If the tactile sensor S3 ^is omitted, a sensor other than the tactile sensor S3 (for example, the camera S1) may be used to estimate the second homogeneous transformation matrix ( _tTw ). The first sensor system may include other sensors for observing the state of the workpiece W with respect to the end effector T. Alternatively, the second homogeneous transformation matrix ( _tTw ) may be given as ^a constant. Further, in the above embodiment, the setting of the hand of the manipulator 4 according to whether the workpiece W is being held may be omitted. In this case, the hand of the manipulator 4 may be set appropriately. For example, the end effector T may be set at the tip of the manipulator 4 regardless of whether it holds the workpiece W or not.

<4.2>
In the above embodiment, the learned judgment model 50 is used when the inference model 55 is generated in the second model generation device 2 . However, the usage form of the learned judgment model 50 need not be limited to such an example. The control device 3 according to the above embodiment may use the learned determination model 50 when controlling the operation of the manipulator 4 . In this case, the learning result data 125 may be provided to the control device 3 at any timing as described above. Further, the control device 3 is configured to further include a contact determination section as a software module.

FIG. 25 illustrates an example of a subroutine processing procedure regarding determination of a target task state according to this modification. The process of determining the target task state in step S308 may be replaced with a subroutine process illustrated in FIG.

In step S501, the control unit 31 operates as the action determination unit 316, and determines a target task state to transition to next from the acquired current task state so as to approach the final target task state. Step S501 may be processed in the same manner as step S308 above.

In step S502, the control unit 31 operates as a contact determination unit and uses the learned determination model 50 to determine whether or not the two objects contact each other in the determined target task state. . Step S502 may be processed in the same manner as steps S203 and S206 described above.

In step S503, the control unit 31 determines the branch destination of the process based on the determination result of step S502. If it is determined in step S502 that the two objects come into contact with each other in the target task state, the control unit 31 returns the processing to step S501 to determine the target task state again. On the other hand, when it is determined that the two objects do not come into contact with each other in the target task state, the control unit 31 executes the processing of the next step S309. Thereby, when controlling the operation of the manipulator 4, the control device 3 uses the learned judgment model 50 to determine the operation of the manipulator 4 so that the tip of the manipulator 4 does not contact the target unnecessarily. can do.

<4.3>
In the above embodiment, the control device 3 uses the inference model 55 to determine the target task state in step S308. However, the method of determining the target task state need not be limited to such an example. The inference model 55 need not be used to determine the target task state. For example, in step S308 above, a target task state may be determined in the same manner as in step S205 above. As an example, the control unit 31 may determine the target task state by a known method such as path planning. Also, for example, the sequence of target task states may be given in advance. In this case, in step S<b>308 , the control unit 31 may determine the next target task state to transition to by referring to the data indicating the series. The same applies to step S501.

Also, in the above embodiment, the generation of the inference model 55 (steps S201 to S211) may be omitted. Alternatively, in the above embodiment, the control device 3 may include each component of the second model generation device 2 . Thereby, the control device 3 may be configured to further execute a series of processes (steps S201 to S211) for generating the inference model 55 described above. In these cases, the second model generation device 2 may be omitted from the control system 100 .

Further, in the above-described embodiment, the judgment result of the learned judgment model 50 is used to collect the learning data 223 . However, collection of learning data 223 need not be limited to such an example. For example, the learning data 223 may be collected without using the learned determination model 50 by using the actual object of each object. Thereby, the inference model 55 may be generated without using the learned judgment model 50 .

Further, in the above embodiment, the processing related to the collection of learning data 223 from steps S201 to S209 may be performed by another computer. In this case, the second model generation device 2 according to the above embodiment acquires the learning data 223 generated by another computer, and uses the acquired learning data 223 to execute steps S210 and S211. good too.

<4.4>
In the above embodiments, the positional relationship between two objects is represented by relative coordinates. However, the method of representing the positional relationship need not be limited to such an example. For example, the positional relationship may be represented by the absolute coordinates of each of the two objects. In this case, each absolute coordinate may be converted into a relative coordinate, and each of the above information processing may be executed.

<4.5>
In the above embodiment, the control device 3 calculates the target value of the hand coordinates from the target task state in step S309. However, the method of acquiring the target value of the hand coordinates need not be limited to such an example. The target value of the hand coordinates may be appropriately determined so as to approach the final target task state.

For example, the target value of the hand coordinates may be determined directly from the current value of the hand coordinates and the value of the hand coordinates in the final target task state. As an example, reference data such as a data table may be used to determine the target values of the hand coordinates. In this case, the control unit 31 can obtain the target values of the hand coordinates from the reference data by comparing the current values of the hand coordinates and the values of the hand coordinates in the final target task state with the reference data. As another example, for example, the control unit 31 uses the nearest neighbor method or the like to determine the target values of the hand coordinates so that the current values of the hand coordinates reach the value of the hand coordinates in the final target task state in the shortest distance. may As another example, for example, the ability to determine the target value of the hand coordinates from the current value of the hand coordinates and the value of the hand coordinates in the final target task state by machine learning, similar to the judgment model 50, the inference model 551, etc. A trained machine learning model that has learned the may be used. In this case, the control unit 31 gives the current values of the hand coordinates and the values of the hand coordinates in the final target task state to the learned machine learning model, and executes arithmetic processing of the learned machine learning model. Thereby, the control unit 31 can acquire the output value corresponding to the result of determining the target value of the hand coordinate from the learned machine learning model.

In this case, steps S306, S308, S309, and S315 may be omitted from the processing procedure of the control device 3. Also, the state acquisition unit 315 and the action determination unit 316 may be omitted from the software configuration of the control device 3 .

Further, in the above embodiment, for example, when the final target task state is set in advance, or when the final target task state is set by another method, steps S301 and The process of step S302 may be omitted. In this case, the target setting unit 310 may be omitted from the software configuration of the control device 3 .

Also, in the above embodiment, the first model generation device 1 may be omitted from the control system 100 . In this case, a method other than the learned determination model 50 may be used to hold information indicating whether or not two objects are in contact with each other. For example, a mode may be adopted in which a value indicating whether or not two objects are in contact with each other is held for each coordinate point.

<4.6>
In the above embodiment, the judgment model 50 is configured by a fully-connected neural network. However, the types of neural networks forming the judgment model 50 need not be limited to such examples. The judgment model 50 may be configured by, for example, a convolutional neural network, a recursive neural network, or the like, in addition to a fully connected neural network. Also, the judgment model 50 may be configured by combining a plurality of types of neural networks.

Moreover, the type of machine learning model that constitutes the determination model 50 is not limited to a neural network, and may be appropriately selected according to the embodiment. For the judgment model 50, machine learning models such as support vector machines, regression models, and decision trees may be employed in addition to neural networks. Whether or not two objects are in contact with each other may be determined with respect to real space or virtual space.

In the above embodiment, the inference model 55 may be prepared for each type of task that the manipulator 4 performs. That is, multiple inference models 55 may be provided, each trained to infer a target task state for a different task. In this case, the control unit 31 of the control device 3 may select the inference model 55 to be used for inference from among the plurality of prepared inference models 55 according to the operation mode set in step S305. Thereby, the control unit 31 may switch the inference model 55 according to the operation mode. Alternatively, the inference model 55 further receives input of information indicating task conditions such as the type of object, the identifier of the object, the identifier of the task, the type of task, etc., and determines the goal of the task corresponding to the input conditions. may be configured to infer the task state of In this case, the control unit 31 further inputs information indicating the operation mode set in step S305 to the inference model 55 when determining the target task state to be transitioned to next, and performs the arithmetic processing in step S308. may be executed.

Also, in the above embodiment, the input and output formats for the judgment model 50 and the inference model 55 are not particularly limited, and may be determined as appropriate according to the embodiment. For example, the judgment model 50 may be configured to receive input of information other than information indicating the task state. Similarly, the inference model 55 may be configured to accept further input of information other than the current task state and the final target task state. If the final goal task state is constant, the information indicating the final goal task state may be omitted from the input of the inference model 55 . The output format of the judgment model 50 and the inference model 55 may be either discrimination or regression.

Further, in the above embodiment, the conversion function (first conversion matrix group or second conversion matrix group) used for the forward kinematics calculation is an example of the first estimation model 61 . Also, the transformation function (ψ) that transforms the values in the task space to the values in the observation space or each coordinate (T _t , T _w ) is an example of the second estimation model 62 . The configuration of each estimation model (61, 62) need not be limited to such an example. Each estimation model (61, 62) may be appropriately configured to be able to calculate the hand coordinates of the manipulator 4 from each sensing data (323, 324).

Each estimation model (61, 62) has parameters for calculating hand coordinates from each sensing data (323, 324). The type of each estimation model (61, 62) may not be particularly limited, and may be appropriately selected according to the embodiment. Each estimation model (61, 62) may be represented by, for example, a functional formula, a data table, or the like. When expressed by functional formulas, each estimation model (61, 62) may be configured by machine learning models such as neural networks, support vector machines, regression models, and decision trees.

<4.7>
In the above embodiment, the control device 3 is configured to be able to execute information processing for parameter adjustment in both FIGS. 20 and 21 . However, the configuration of the control device 3 need not be limited to such an example. In the above embodiments, the controller 3 may be configured to perform only one of FIGS. 20 and 21. FIG. For example, the control device 3 omits the execution of the information processing related to parameter adjustment shown in FIG. 20, and executes the information processing related to parameter adjustment shown in FIG. may be configured to

1 ... first model generation device,
11... control unit, 12... storage unit, 13... communication interface,
14 ... external interface,
15... input device, 16... output device, 17... drive,
91... Storage medium, 81... Model generation program,
111... data acquisition unit, 112... machine learning unit,
113 ... storage processing unit,
120... CAD data,
121 ... learning data set,
122...Training data, 123...Correct data,
125... Learning result data,
2 ... the first model generation device,
21... control unit, 22... storage unit, 23... communication interface,
24 ... external interface,
25... input device, 26... output device, 27... drive,
92... Storage medium, 82... Model generation program,
211 ... contact determination unit, 212 ... data collection unit,
213 ... model generation unit, 214 ... storage processing unit,
220... CAD data, 223... learning data,
225 inference model data,
3 ... control device,
31... control unit, 32... storage unit, 33... communication interface,
34 ... external interface,
35... input device, 36... output device, 37... drive,
93... Storage medium, 83... Control program,
310... Goal setting unit, 311... First data acquisition unit,
312... second data acquiring unit, 313... first estimating unit,
314 ... second estimation unit, 315 ... state acquisition unit,
316... action determination unit, 317... command determination unit,
318... drive section, 319... adjustment section,
320... CAD data, 321... robot data,
323... First sensing data,
324 ... second sensing data,
4... Manipulator,
40... Pedestal part,
41 to 46... Joints, 491 to 494... Links,
T... end effector,
T0: point of interest, CT: local coordinate system,
W... work,
W0: point of interest, CW: local coordinate system,
G... other work, CG... local coordinate system,
RC1 and RC2 ... relative coordinates,
S1... camera, S2... encoder, S3... tactile sensor,
50 ... Judgment model,
501... Input layer, 502... Intermediate (hidden) layer,
503 ... output layer,
55 Inference model,
61... First estimation model, 62... Second estimation model

Claims (6)

A control device for controlling the operation of a manipulator,
a first data acquisition unit that acquires first sensing data from a first sensor system that observes the tip of the manipulator;
a first estimating unit that calculates a first estimated value of the current coordinates of the hand in an observation space from the first sensing data obtained using a first estimation model;
a second data acquisition unit that acquires second sensing data from a second sensor system that observes the tip of the manipulator;
a second estimation unit that calculates a second estimated value of the current coordinates of the hand in the observation space from the acquired second sensing data using a second estimation model;
calculating a gradient of an error between the first estimated value and the second estimated value, and based on the calculated gradient, at least one of the first estimated model and the second estimated model such that the error is reduced; an adjustment unit that adjusts the value of the parameter of
a command determination unit that determines a control command to be given to the manipulator based on at least one of the first estimated value and the second estimated value so that the coordinates of the hand end approach a target value;
a driving unit that drives the manipulator by giving the determined control command to the manipulator;
comprising
Control device. The adjustment unit further
when the hand of the manipulator touches an object, obtaining a boundary value of the coordinates of the hand on the boundary surface of contact with the object;
calculating a gradient of a first error between the first estimated value estimated at the time of contact and the acquired boundary value, and reducing the first error based on the calculated gradient of the first error; and calculating a gradient of a second error between the second estimated value estimated at the time of contact and the obtained boundary value, and calculating the calculated Adjusting the parameter values of the second estimation model so that the second error is smaller based on the gradient of the second error;
A control device according to claim 1 . the manipulator comprises one or more joints;
The first sensor system includes an encoder that measures the angle of each joint,
the second sensor system comprises a camera;
3. A control device according to claim 1 or 2. The manipulator further comprises an end effector for holding a workpiece,
When the end effector does not hold the workpiece, the point of interest of the end effector is set to the hand,
When the end effector holds the work, the point of interest of the work is set to the hand,
The first sensor system further includes a tactile sensor for estimating the positional relationship of the workpiece with respect to the end effector.
4. A control device according to claim 3. A control method for controlling the operation of a manipulator, comprising:
the computer
obtaining first sensing data from a first sensor system that observes the hand of the manipulator;
calculating a first estimated value of the current coordinates of the hand in the observation space from the obtained first sensing data using a first estimation model;
obtaining second sensing data from a second sensor system that observes the hand of the manipulator;
calculating a second estimated value of the current coordinates of the hand in the observation space from the acquired second sensing data using a second estimation model;
calculating the slope of the error between the first estimate and the second estimate;
adjusting a parameter value of at least one of the first estimation model and the second estimation model so as to reduce the error based on the calculated gradient;
determining a control command to be given to the manipulator based on at least one of the first estimated value and the second estimated value so that the coordinates of the hand approach a target value;
driving the manipulator by giving the determined control command to the manipulator;
run the
control method. A control program for controlling the operation of a manipulator,
to the computer,
obtaining first sensing data from a first sensor system that observes the hand of the manipulator;
calculating a first estimated value of the current coordinates of the hand in the observation space from the obtained first sensing data using a first estimation model;
obtaining second sensing data from a second sensor system that observes the hand of the manipulator;
calculating a second estimated value of the current coordinates of the hand in the observation space from the acquired second sensing data using a second estimation model;
calculating the slope of the error between the first estimate and the second estimate;
adjusting a parameter value of at least one of the first estimation model and the second estimation model so as to reduce the error based on the calculated gradient;
determining a control command to be given to the manipulator based on at least one of the first estimated value and the second estimated value so that the coordinates of the hand approach a target value;
driving the manipulator by giving the determined control command to the manipulator;
to run the
control program.

Images