EP3993959A1

EP3993959A1 - Carrying out an application using at least one robot

Info

Publication number: EP3993959A1
Application number: EP20735370.7A
Authority: EP
Inventors: Manuel KASPAR; Pierre VENET; Jonas SCHWINN
Original assignee: KUKA Deutschland GmbH
Current assignee: KUKA Deutschland GmbH
Priority date: 2019-07-01
Filing date: 2020-06-29
Publication date: 2022-05-11
Also published as: WO2021001312A1; US20220339787A1; CN114051444B; CN114051444A

Abstract

The invention relates to a method for carrying out an application using at least one robot (1), comprising the steps, repeated multiple times: ascertaining (S100) a stochastic value of at least one robot parameter and/or at least one environmental model parameter, in particular on the basis of a specified stochastic parameter model and/or using at least one random generator (3a); and carrying out (S200) a simulation, in particular a multi-stage simulation, of the application on the basis of the ascertained stochastic value; the steps: training (S200, S500) at least one control agent and/or at least one classification agent using the simulations by means of machine learning, in particular training a first control agent and/or first classification agent by means of first stages of the simulations, and at least one additional control agent and/or additional classification agent by means of additional stages of the simulations; and carrying out (S600) the application using the robot; and at least one of the steps: configuring (S400) a controller of the robot, by means of which the application is carried out wholly or in part, on the basis of the trained control agent, in particular on the basis of the trained control agents; and/or classifying (S700) the application using the trained classification agent, in particular using the trained classification agents.

Description

description

Carrying out an application using at least one robot

The present invention relates in particular to a method for performing an application using at least one robot, a method for configuring a controller of a robot for performing an application or a predetermined task, a method for training at least one classification agent for classifying a robot application, a method for Carrying out a predetermined task with the aid of at least one robot with a correspondingly configured controller and a system and a computer program product for carrying out at least one of these methods.

In order to carry out applications or specified tasks, controls of robots must be configured accordingly, conventionally by

manual creation of robot programs or the like.

One object of an embodiment of the present invention is to improve the implementation of an application or a given task with the aid of at least one robot. One object of an embodiment of the present invention is to improve the configuration of a controller of the robot for carrying out the application or the specified task. One task of performing the

The present invention is to improve a classification of a robot application. One object of an embodiment of the present invention is to improve control of a robot with which an application is carried out.

One or more of these objects, the subjects of the independent claims and / or one or more of the methods or means described here, in particular a method with the features of claims 1, 9, 10 or 11, are achieved. Claims 14, 15 provide a system or computer program product for performing a method described here under protection. The subclaims relate to advantageous developments.

According to one embodiment of the present invention, a method for - Configuring a controller of a robot for carrying out an application with the aid of the robot (robot application);

- Train at least one classification (KI) agent to classify a robot application; and or

- Carrying out one or the (robot) application with the aid of at least one or the robot

the steps repeated several times, cyclically in one execution:

- Determination of a one-dimensional or multi-dimensional stochastic value of at least one one-dimensional or multi-dimensional robot parameter and / or at least one one-dimensional or multi-dimensional environmental model parameter, in one embodiment on the basis of a predetermined stochastic parameter model and / or with the aid of at least one random generator; and

- Carrying out a multi-stage simulation of the application on the basis of the determined stochastic value.

The stochastic value for a simulation can be determined before the simulation is carried out and then used in it. Likewise, several stochastic values of the robot and / or the

Environmental model parameters are determined in advance and then one of these stochastic values is used for or in one of the simulations.

According to one embodiment of the present invention, the method comprises the step:

- Work out

- At least one control (AI) agent and / or

- at least one classification (AI) agent

using the simulations using machine learning, in one implementation

- Training a first control (KI) agent and / or a first classification (KI) agent using first stages of the simulations, and

- Training at least one further control (KI) agent and / or at least one further classification (KI) agent with the aid of further stages of the simulations.

According to one embodiment of the present invention, the method comprises the step: single or multiple execution of the (real) application using the robot.

Several control (AI) agents can also use several

S (simulation) levels and only one classification (AI) agent can be trained using the simulations or several classification (AI) agents using several S (simulation) levels and only one control (AI) agent can be trained using the simulations .

In one embodiment, the robot or environmental model parameter (value) is randomized or the simulations with randomized robot or

Environmental model parameters (values) n carried out and the agent or agents trained or machine-learned with the aid of these simulations.

As a result, machine learning can be improved in one implementation and more robust and / or faster in one implementation. In addition or as an alternative, this means that in one embodiment such or on the basis of randomized

Robot or environmental model parameters (values) trained agent improve the implementation of the (real) application with the help of the robot, in particular the control of the robot and / or classification of the application, in particular robust (more) and / or flexible (e) l ( he) act.

When an agent is referred to in the present case, this includes in particular an artificial intelligence agent, in particular one

Control (AI) agents or a classification (AI) agent.

In one embodiment, determining a stochastic value comprises generating the value, in particular numerically and / or physically, and can in particular be generating the value.

The stochastic values on the basis of which the simulations are carried out are, in particular are, in one embodiment with the aid of at least one

Random generator, in particular pseudo-random number generator, determined, in particular generated, and / or are stochastically or randomly distributed values, in one embodiment random numbers, in particular pseudo-random numbers, which in one embodiment are determined by the specified stochastic parameter model or satisfy it. In one embodiment, the stochastic parameter model has one or more stochastic parameters, in particular minimum, maximum, expected and / or mean value (s), variance (s), standard deviation (s), measure (s) of dispersion or the like , and / or a probability distribution, for example a

Gaussian or normal distribution, a uniform distribution, or the like.

For example, a user and / or a user input support or a software assistant in an embodiment for a robot or

Environmental model parameters specify a minimum and maximum value as well as a uniform distribution and thus a stochastic parameter model, whereby corresponding stochastically (distributed values are then generated with the help of a (pseudo) random number generator and determined in this way on the basis of this predetermined stochastic parameter model and with the aid of this (pseudo) random number generator Likewise, the user and / or the

User input support, for example, specify a specific Gaussian distribution and thus a different stochastic parameter model, with the help of a (pseudo) random number generator then again generated corresponding stochastically (distributed values and in this way on the basis of this other specified stochastic parameter model and with the aid of this

(Pseudo) random number generator can be determined.

Thus, in one embodiment, the determined stochastic values are (also) determined by the specified stochastic parameter model, for example limited by minimum and / or maximum value (s), scattered by variance (s) around an expected or mean value or like that.

A simulation is understood to mean, in particular, a simulation run or a numerical simulation of the application or its course over time.

A multi-stage simulation comprises two or more in one implementation

successive, in one embodiment, consecutive time and / or functional sections or stages of the application, for example the robot-assisted joining of a first gear (first stage) and the subsequent robot-assisted joining of a further gear (further stage) or the like. In one implementation, a first control agent is established using first stages or

Sections of the simulations and at least one further control agent with the aid of further stages or sections of the simulations and / or a first

Classification agent trained using the same or different first stages or sections of the simulations and at least one further classification agent using further stages or sections of the simulations, in the above example a first (control or classification) agent for joining the first Gear using the first simulation stages or simulations of joining the first gear and another (control or classification) agent for joining the further gear using the further simulation stages or simulations of joining the further gear.

In one embodiment, an initial state for a later simulation stage is determined or specified on the basis of a final state or result of a previous simulation stage, in one embodiment this initial state also varies, in particular randomized, in particular on the basis of a user input or specification, will or can be. In the above example, for example, a pose of the first gearwheel after its simulated joining can be used as a starting value in the further simulation stage and, if necessary, can be changed and / or randomized by a user.

In this way, a multi-stage application can be carried out particularly advantageously in one embodiment.

- Configuring a controller of the robot to carry out the application, in particular a controller of the robot with which the application is carried out, on the basis of the or one or more of the trained control agents.

Thus, according to one aspect of the present invention, the simulations with stochastic or randomized values are used to learn machine control of the robot for performing the (real) application with the aid of the robot or to train one or more agents for this purpose. As a result, in one embodiment, the implementation of the (real) application can be improved with the aid of the robot, in particular the application can be implemented in a robust (more) and / or flexible (more) manner.

In one embodiment, a control of the robot, with which only a part of the application is or is to be carried out, is configured on the basis of the, in particular the, trained control agent.

In particular, the application can have one or more sections that are (should) be carried out with a (different) controller of the robot that is or is not configured on the basis of the trained control agent (s), as well as one or more sections, which are (should) be carried out with a controller of the robot, which are based on the trained

Control agent, is or is configured. A controller within the meaning of the present invention can in particular have, in particular be, a control device and / or a computer program, in particular (computer) program module or part.

In particular, it can be useful for transfer sections in which the robot moves a load freely, a (different) control by means of geometric or

To configure dynamic path planning, teaching or the like, and for contact, in particular gripping and / or joining sections, in which there is or is an environmental contact of the robot, in particular it grips or adds a load, a control based on the or of the trained agents.

Additionally or alternatively to this aspect, the method according to a

Implementing the present invention includes the step of:

Classification of the application with the aid of the or one or more of the trained classification agents.

Thus, according to one aspect of the present invention, the simulations with stochastic or randomized values are used in order to machine a classification of the (real) application or to learn one or more

To train classification agents to do this. In this way, in one embodiment, the implementation of the (real) application can be improved with the aid of the robot, in particular the application is (more) robust and / or

be monitored flexibly.

In one embodiment, the one or more of the

Classification agent (s each) a machine-learned anomaly detection. Additionally or alternatively, in one embodiment the or one or more of the classification agent (s) has / have a machine-learned one

Error detection on.

In one embodiment, an anomaly detection comprises a classification of the application (s) carried out into normal (s) and abnormal applications). In one embodiment, the anomaly detection is or is machine learned, in particular only, on the basis of simulated applications labeled as normal and / or the anomaly detection classifies an application as abnormal if it deviates (too greatly) from the simulated applications labeled as normal.

If, for example, an obstacle not taken into account in the simulations blocks the implementation of the real application with the aid of the robot, force and / or pose data of the robot in particular deviate from the courses as normal

labeled simulated applications and the agent classifies this real application as abnormal.

In one embodiment, error detection includes a classification of the

executed application (s) in error-free (e) and defective applications), in one execution in different error classes. It is or will be learned automatically in an implementation based on simulated applications labeled as error-free and simulated applications labeled as defective or corresponding to a corresponding error class and / or classifies an application into a (corresponding) error class if it matches the correspondingly labeled simulated applications sufficiently, in particular most strongly, resembles.

For example, in the above example, joining the first gear with the aid of the robot, in particular on the basis of force and / or posture data of the robot, can be considered to be free of errors, as being attached, but not sufficiently deep and / or jammed, or as are classified as not joined if the force or pose data are sufficiently similar to the courses of appropriately labeled simulated applications and the agent classifies this real application into the corresponding error class.

The invention can be used with particular advantage for such classifications of

Robot applications are used, as these are (only) difficult to learn by machine using (real) applications carried out with the robot.

In one embodiment, the one or more of the

Control agent (s) and / or one or more of the

Classification agent (s) at least one artificial neural network, in one embodiment the control of the robot is configured based on the structure and / or weightings of the trained network.

As a result, a particularly advantageous control can be implemented in one embodiment and / or this can be configured particularly advantageously.

In one embodiment, the or one or more of the control agent (s) and / or the or one or more of the classification agent (s) is trained with the aid of reinforcement learning, in particular deep reinforcement learning.

This is particularly suitable for configuring a controller for the robot and for classifying, in particular anomaly and / or error detection, of the application.

In one embodiment, the or one or more of the control agent (s) and / or the or one or more of the classification agent (s) is trained, in particular additionally, with the aid of the robot, in one embodiment based on one or more of the several (real) applications carried out with the help of the robot.

As a result, in one embodiment, the corresponding agent can be used particularly advantageously when performing the real application with the aid of the robot and / or machine learning can be (further) improved. In one embodiment, the one or more of the

Control agent (s) and / or one or more of the

Classification agent (s), trained on the basis of at least one state variable that is not measured when the application is carried out and cannot be measured in an embodiment.

This is based in particular on the knowledge or idea that the

Simulations are also calculable state variables, in particular are calculated that are not measured when the application is carried out,

possibly - with the existing environment or configuration, in particular measurement equipment, - are not measurable, and that such state variables, which arise or are calculable anyway in simulations for (the purpose of) configuring (s) the control, in particular are calculated, can be used particularly advantageously (also) for training or machine learning.

In the above example, it can be the case that the distance between the (first or additional) gear and a stop cannot be measured, for example because there is no corresponding sensor or the space between the gear and the stop is not accessible. In a simulation of the joining, however, this distance can be calculated and then used as a state variable for training, in particular in a quality criterion.

In one embodiment, when training the or one or more of the

Control agents and / or classification agents used quality criterion, in particular a quality or cost function, based on at least one at

Execution of the application not measured, in one version with the

existing configuration or environment, which cannot be measured, is determined or depends on this.

As a result, machine learning can be improved in one implementation and more robust and / or faster in one implementation.

In addition or as an alternative to the idea of using state variables for training the agent or one or more of the agents that are not measured when the application is carried out but are calculated during the simulations, there is one Execution of the present invention is based on the knowledge or idea,

Simulations that are (anyway) carried out or used to train at least one control agent, on the basis of which the control of the robot is configured, with which the (real) application is or should be carried out, also used or used for this purpose. use one or more

To train classification agents, with whose help the (real)

Application that is carried out with the aid of the robot is or should be classified.

Accordingly, according to one embodiment of the present invention, the method comprises

Invention both the step:

- Configuring a controller of the robot to carry out the application, in particular a controller of the robot with which the application is carried out in whole or in part, on the basis of the or one or more of the trained control agents;

as well as the step:

- Training the or one or more of the classification agents,

in particular, classification of the application with the aid of one or more of the trained classification agents,

wherein, in one embodiment, control and classification agent (s) are or have been trained using the same simulations, wherein in one

Further development of the or one or more of the classification agent (s) with the aid of simulations that have already taken place, with the aid of which the or one or more of the control agent (s) have been trained beforehand and / or are trained synchronously with the aid of current simulations, with the help of which the or one or more of the control agent (s) are currently being trained.

In other words, in one embodiment the invention uses simulations on the basis of which, in particular by means of reinforcement learning, the controller is configured or with the aid of which the or one or more of the control agent (s) is or are trained, in one embodiment has been or are also used for training at least one machine-learned classification or the or one or more of the classification agents. In one embodiment, for this purpose, data, in particular state variables, in one embodiment (temporal) state variable curves, in particular trajectories, of the application, in one embodiment of the robot, which are or have been calculated in simulations, in one embodiment, simulations with the aid of which the or one or more of the control agent (s) are or have been trained, stored and the or one or more of the classification agent (s) are trained with the aid of this stored data, in an execution following this

Simulations and / or during these simulations.

In one embodiment, these data include poses of one or more robot-fixed references, in particular an end effector, TCPs, robot-guided tool or piece or the like, joint or axis divisions of the robot, internal and / or external forces on the robot, in particular joint and / or Driving forces, frictional forces, contact forces or the like, current variables, in particular

Tensions and / or currents in the drives of the robot, tracking errors of the robot and / or time derivatives of such poses, positions, forces, current variables or tracking errors, in particular speeds and / or accelerations of one or more robot-fixed references, axes or joints, drives, temporal changes in such forces, current variables or following errors or

like that. Following errors can in particular include force, position and / or speed errors.

In one execution, the, in one execution,

Simulations, in one embodiment simulations with the aid of which the or one or more of the control agent (s) are or have been trained, in particular using the stored data, those simulations or data are selected in which a quality criterion is met, and used to train the anomaly detection, or those simulations or data sorted into different defect classes on the basis of a quality criterion and used to train the defect detection.

If, for example, trajj denotes the data of a simulation i, traj = {traj_i} the amount of all data stored during the simulation, are converted into a

Execution for machine learning of the anomaly detection the data

{traj_success} of those simulations, in which a successful course of the Application was simulated or resulted, or the data {traj_failure_k1}, {traj_failure_ k2}, ... of those simulations in which an error k1, k2, ... was or resulted, selected and then the Anomaly detection using {traj_success} or error detection using {{traj_success}, {traj_failure_k1}, {traj_failure_ k2}, ...} learned by machine.

As a result, the machine-learned classification can in each case be improved in one embodiment, in particular learned more quickly and / or classified more precisely, more robustly and / or more reliably.

In one implementation, the one or more of the agent (s) classifies

in particular the anomaly detection and / or the error detection, the application on the basis of at least one, migrating in one execution, in particular

wandering, time segment. In one embodiment, a continuous and / or serial evaluation is carried out in addition to or as an alternative to an evaluation of the complete application and the agent classifies the application on the basis of this continuous or serial evaluation. Recurrent networks, Markov models or autoregressive networks are particularly suitable for this.

As a result, machine learning can be improved in one implementation and more efficient and / or faster in one implementation.

In one implementation, the one or more of the agent (s) classifies

in particular the anomaly detection and / or the error detection, the application while the application is being carried out.

In one embodiment, this enables a reaction to the result of the classification. Correspondingly, in one execution the one that is just or currently being carried out

Application may be changed based on the classification, in one embodiment a corresponding signal is output in the event of a recognized anomaly and / or a recognized error and / or a movement of the robot is modified and / or a workpiece handled, in particular transported and / or processed during the application sorted out or reworked. In one embodiment, the or one or more of the agent (s), in particular the anomaly detection and / or the error detection, classifies the application after the application has been carried out.

This allows the application to be classified more precisely in one embodiment.

In one embodiment, the robot parameter has a one-dimensional or multi-dimensional start pose, one or more one or more one-dimensional or multi-dimensional intermediate poses and / or a one-dimensional or multi-dimensional target pose of the application, in particular of the

Robot. Accordingly, in one version, the simulations of the application are based on stochastic (distributed or generated start,

Intermediate and / or target poses carried out. As a result, inaccuracies as a result of previous processes, deviations when running or the like can be taken into account in one implementation, thereby improving machine learning or the trained agent (s), in particular robust (more) and / or flexible (e) l (he), will.

In one embodiment, especially before performing the simulation, it is checked whether (the stochastic value for) the starting pose, intermediate pose (s) and / or target pose can be achieved with the robot, in particular on the basis of a kinematic model of the robot. If the pose or the corresponding stochastic value of the

Robot parameter is not achievable, the value is determined again in one embodiment or until (it is determined that) the pose or the value can be reached with the robot, and then this value as the value determined when the simulation of the application is carried out used. As a result, machine learning can be improved in one implementation and more efficient and / or faster in one implementation.

A pose within the meaning of the present invention can in particular have, in particular be, a one, two or three-dimensional position and / or one, two or three-dimensional orientation.

Additionally or alternatively, in one embodiment the robot parameter has a one- or multi-dimensional force parameter of a robot-internal force, in particular at least one axis and / or at least one end effector rigidity and / or damping. This can lead to wear and tear in one version

Tolerances between robots of the same model or the like are taken into account, thereby improving machine learning or the trained agent (s), in particular robust (more) and / or flexible (more).

Additionally or alternatively, in one embodiment the robot parameter has a one-dimensional or multi-dimensional force parameter of an external force that acts on the robot at least, in one embodiment only temporarily, in particular an external force

(Stochastic) disturbance or disturbance force, in particular an external force as a result of environmental contact or the like. As a result, real process conditions or random disruptions can be taken into account in an execution and thereby improve machine learning or the trained agent (s),

in particular robust (more) and / or flexible (e) l (more).

A force within the meaning of the present invention can in particular have, in particular be, an antiparallel force pair or torque. A force parameter can in particular include a force, but also a rigidity, a damping and / or a friction coefficient or the like.

Additionally or alternatively, the robot parameter has in one embodiment a one- or multi-dimensional kinematic, in one embodiment dynamic, robot structure parameter, in particular a one-dimensional or multi-dimensional dimension and / or a weight and / or one or multi-dimensional

Moment of inertia of the robot or individual structural members or

Structural member groups, or the like. As a result, tolerances between robots of the same model or the like can be taken into account in one embodiment, thereby improving machine learning or the trained agent (s), in particular robust (more) and / or flexible (more), will.

Additionally or alternatively, the environmental model parameter in one embodiment has a one- or multi-dimensional kinematic, in one embodiment dynamic, environmental, in one embodiment load structure parameter, in particular a one or multi-dimensional pose and / or dimension and / or weight and / or moment of inertia an environmental, in particular load structure,

in particular a tool and / or workpiece used in the application or similar. As a result, tolerances between tools or workpieces of the same model, inaccuracies as a result of

previous processes or the like are taken into account, thereby improving machine learning or the trained agent (s),

in particular robust (more) and / or flexible (e) l (more).

Additionally or alternatively, in one embodiment, the robot parameters and / or the environmental model parameters are determined with the aid of a robot-supported parameter identification, for example minimum, maximum and / or mean value (s) of or for the stochastic parameter model. This allows the

Correspondence with the real application is improved and, as a result, machine learning or the trained agent (s) are improved, in particular robust (more) and / or flexible (more).

In one embodiment, the predetermined stochastic one is, in particular becomes

Parameter model based on a user input and / or application-specific predetermined, selected in one embodiment from a plurality of different parameter models made available.

In one embodiment, a user can initially choose one of several

Probability distributions, for example a Gaussian distribution, a

uniform distribution or another probability distribution, and specify minimum and maximum values or the like for this or here.

Similarly, for certain joining applications, for example, a

Probability distribution, for example a uniform distribution, for certain gripping applications a different probability distribution, for example a Gaussian distribution, or the like is selected and application-specific minimum and maximum values or the like are specified for this or here. Mixed forms are also possible, in particular one

application-specific preselection or default value assignment and a

Input option for the user to change this.

Additionally or alternatively, in one embodiment of the

Robot and / or the environmental model parameters based on user input and / or specified application-specifically, selected in one embodiment from a plurality of different parameters made available.

For example, for inserting a robot-guided workpiece into a recess in a flat surface, a two-dimensional position within the surface and a one-dimensional orientation or angular position around a

Surface normals can be specified or selected as the target pose, whereas for drilling with a robot-guided drill in a flat surface, a one-dimensional distance to the surface along the drill axis is used as the target or

Intermediate pose.

In one embodiment, the stochastic parameter model and / or the

Robot and / or the environmental model parameters visualized in a, in particular virtual, image of the application through a marked area, in one embodiment through corresponding geometric spaces, in particular bodies such as preferably cuboids, spheres, cones, cylinders or the like, or, in particular, planar or environment-adapted , Surfaces.

In the above example, the area within the area in which the target position can be (stochastically) can be visualized in an image of the application, for example by a corresponding circular area, the possible orientations or angular positions around the surface normal, for example, by two appropriately twisted cuboids or workpiece avatars in the respective maximum possible deflections.

In one embodiment, a probability distribution of the stochastic parameter model is visualized by a different color, in one embodiment different (color) brightness, of the marked area, the respective color or brightness (level) depending on the probability that the robot or environmental model parameter has the corresponding value.

In the above example, the area within the area in which the target position can (stochastically) be visualized in an image of the application, for example, by a corresponding circular area, areas of the circular area in which the target position is more likely, for example are or become darker or a first area of the circular area in which the Target position lies with a first probability, for example is or is colored with a first color and / or brightness, and at least one other area of the circular area in which the target position lies with a different probability is colored with a different color and / or brightness is or will.

As a result, in one embodiment, in particular in combination, a particularly suitable parameter model or particularly suitable parameters can be selected, in particular the speed and / or error tolerance of the input can be improved. Both with the user input for specifying, in particular selection, the stochastic parameter model and the user input for specifying, in particular selection, the robot and / or the

Environmental model parameter is one described elsewhere

User input support by a software assistant is particularly advantageous.

In one embodiment, the configured control of the robot and / or the machine-learned anomaly detection and / or error detection is tested with the aid of at least one further simulation, in particular based on an automated specification or user specification of a value of at least one robot parameter and / or at least one environmental model parameter.

For example, the user can pose a workpiece for the

Change the test simulation and then use the test simulation to check whether and how well the configured control or anomaly or error detection (for this) works or performs. Similarly, a test script can automate further simulation with the trained control agent or agents

Carry out anomaly and / or error detection and thereby the values of at least one robot parameter and / or at least one

Environmental model parameters vary.

Additionally or alternatively, the configured control of the robot and / or the machine-learned anomaly detection and / or

Error detection, in particular by means of machine learning, in particular

Reinforcement learning, further trained with the help of the robot, in particular on the basis of applications carried out with the help of the robot. As a result, a control which is particularly advantageous in practice can be implemented in one embodiment, in particular in combination.

In one embodiment, the stochastic parameter model is, in particular, specified with the aid of machine learning. In particular, a

Parameter model (KI) agent based on earlier applications carried out with the aid of the robot, which have been classified with the aid of a classification agent trained according to a method described here and / or in which the

Control of the robot, with which these applications were carried out, have been configured on the basis of a control agent trained according to a method described here, on the basis of the results of the earlier carried out

Applications and when training this classification or

Control agents used stochastic parameter model, which predefined the stochastic parameter model, which is then used in a method described here for performing simulations to train the at least one classification agent, with the help of which a new application is then classified, and / or the at least one control agent is used, with the help of which a controller is then configured to carry out a new application.

In this way, a particularly advantageous, in particular realistic, stochastic parameter model can be used, preselected in one embodiment, in particular by the user input support or the software assistant. In addition or as an alternative to the earlier applications carried out with the aid of the robot, simulated applications can also be used as earlier applications for machine learning for specifying the stochastic parameter model.

In one implementation, one or more of the steps include one of the following

described method, in particular the specification, in particular selection, of the stochastic parameter model and / or the robot and / or the

Environmental model parameters, user input support by a software assistant, in particular a user interface guide, in particular a so-called wizard. In one embodiment, the robot and / or the environmental model parameter and / or the stochastic parameter model is preselected from a plurality of different parameters or parameter models made available, in particular application-specific and / or by the

User input support or the software assistant.

As a result, in one embodiment, in particular in combination, a particularly suitable parameter model or particularly suitable parameters can be selected, in particular the speed and / or error tolerance of the input can be improved.

Additionally or alternatively, one or more of the steps of one of the methods described here are carried out in a cloud.

As a result, this method can advantageously be carried out in parallel and / or quickly and / or in a distributed manner.

According to one embodiment of the present invention, a method for

To configure a controller of a robot to perform a given task the steps include:

- Detecting at least one one-dimensional or multi-dimensional robot parameter and at least one one-dimensional or multi-dimensional environmental model parameter;

- Training an (AI) agent with the help of one or more simulations on the basis of this recorded robot parameter and recorded it

Environmental model parameters using machine learning based on a

given cost function; and

- Configure the control of the robot based on the trained agent.

By training an agent by means of machine learning with the aid of one or more simulations, a controller of a robot for performing a predetermined task can be configured particularly advantageously in one embodiment.

In one embodiment, the robot has a stationary or mobile, in particular mobile, base and / or a robot arm with at least three, in particular at least six, in one embodiment at least seven joints or joints. (Movement) axes, in one version swivel joints or axes. The present invention is particularly suitable for such robots because of its kinematics, variability and / or complexity.

In one embodiment, the specified task has at least one movement of the robot, in particular at least one scheduled environmental contact of the robot, that is to say in particular can have robot-assisted gripping and / or joining. For such purposes, the present invention is due to its

Complexity particularly suitable.

In one embodiment, the robot has parameters

- a one or more dimensional kinematic, in particular dynamic, robot model parameter, in particular one or more center distances,

Masses, centers of mass, inertia and / or stiffness; and or

a one or more dimensional kinematic, in particular dynamic, load model parameter, in particular one or more dimensions, masses, centers of gravity and / or inertia; and or

- a current robot pose, in particular one or more current axis or

Joint positions; and or

- a current robot operating time.

Additionally or alternatively, in one embodiment, the environmental model parameter has a one- or multi-dimensional CAD model parameter and / or a, in particular current, robot positioning in the environmental model and / or is determined with the aid of at least one optical sensor, in particular a camera.

In one development, this optical sensor is guided, in particular held or carried by a person, in another development by one

Robot, which for its part travels a programmed or automatically determined path, in particular by means of collision avoidance, or is or is guided manually or by forces exerted manually on the robot.

In one embodiment, the agent has an artificial neural network. In a further development, the control of the robot is then configured on the basis of the structure and / or weightings of the trained network, this structure and / or Transferring weights in one execution to the controller of the robot. Additionally or alternatively, the agent is implemented using

Reinforcement learning, preferably deep reinforcement learning, trained.

In one embodiment, after it has been configured in the manner described here, the control of the robot is trained further by means of machine learning, in particular reinforcement learning, preferably deep reinforcement learning, with the aid of the real robot.

In one embodiment is, in particular, the robot parameter and / or

Environmental model parameters are stored at least temporarily in an administration shell and / or in a data cloud.

According to one embodiment of the present invention, in a method for performing a predetermined task with the aid of at least one robot, a controller of the robot is configured according to a method described here. Accordingly, in one embodiment, an inventive

Method A method described here for configuring a controller of a robot for performing a predetermined task and the step of performing the predetermined task using the robot with the

have control configured according to the invention.

According to one embodiment of the present invention, a system, in particular in terms of hardware and / or software, in particular in terms of programming, is set up to carry out one or more methods described here. It shows in a

Execution means for acquiring at least one robot parameter and at least one environmental model parameter; Means for training an agent with the aid of at least one simulation based on the detected robot parameters and environmental model parameters by means of machine learning based on a predetermined cost function; and means for configuring the control of the robot based on the trained agent.

In one implementation, the system comprises:

Means for repeatedly determining a stochastic value of at least one robot parameter and / or at least one environmental model parameter, in particular on the basis of a predetermined stochastic parameter model and / or with the aid of at least one random generator; and carrying out an, in particular multi-stage, simulation of the application on the basis of the determined stochastic value; and

Means for training at least one control agent and / or at least one classification agent with the aid of the simulations by means of machine learning, in particular training a first control agent and / or first

Classification agents using first stages of the simulations, and at least one further control agent and / or further classification agents using further stages of the simulations.

Additionally or alternatively, in one embodiment the system has means for

Configuring a controller of the robot on the basis of, in particular the, trained control agent for performing the application.

Additionally or alternatively, the system has means for in one embodiment

Classification of the application using the, in particular the trained,

Classification agents.

Additionally or alternatively, in one embodiment the system has means for

Execution of the application using the robot, with a controller of the

Robot, with which the application is carried out in whole or in part, is configured on the basis of the, in particular the, trained control agent and / or the

Application is classified with the aid of, in particular, the trained classification agent.

In one embodiment, the system or its means has:

a machine-learned anomaly detection and / or machine-learned

Error detection and / or at least one artificial neural network; and / or means for training at least one control agent and / or at least one classification agent with the aid of reinforcement learning and / or with the aid of the robot; and or

Means for classifying the application on the basis of at least one, in particular migrating, time segment and / or during or after the application is carried out with the aid of the at least one classification agent; and or Means for training the at least one control agent and / or the at least one classification agent on the basis of at least one at

Execution of the application not measured state variable; and or

Means for determining the robot parameter and / or the environmental model parameter with the aid of a robot-assisted parameter identification; and or

Means for checking whether the start pose, intermediate pose and / or target pose with the

Robot is reachable; and or

Means for specifying the stochastic parameter model on the basis of the application and / or a user input, in particular for selecting from a plurality of different parameter models made available; and or

Means for visualizing the stochastic parameter model in an image of the application through a marked area; and or

Means for specifying the robot and / or the environmental model parameter on the basis of the application and / or a user input, in particular for selecting from a plurality of different parameters made available; and / or means for visualizing the robot and / or the environmental model parameter in an image of the application through a marked area; and or

Means for testing the configured control of the robot and / or the machine-learned anomaly detection and / or error detection with the aid of at least one further simulation, in particular on the basis of an automated or

User specification of a value of at least one robot parameter and / or at least one environmental model parameter; and or

Means for further training the configured control of the robot and / or the machine-learned anomaly detection and / or error detection with the aid of the robot; and or

Means for specifying the stochastic parameter model using machine learning; and or

Means for user input support of at least one of the method steps by a software assistant, in particular a user interface guide; and or

Means for performing at least one of the method steps in a cloud.

A means within the meaning of the present invention can be designed in terms of hardware and / or software, in particular one that is data or signal-connected, in particular digital, processing, preferably with a memory and / or bus system. in particular have a microprocessor unit (CPU), graphics card (GPU) or the like, and / or one or more programs or program modules. The

Processing unit can be designed to receive commands as one in one

Program stored in the storage system are implemented,

Acquire input signals from a data bus and / or deliver output signals to a data bus. A storage system can have one or more,

in particular have various storage media, in particular optical, magnetic, solid-state and / or other non-volatile media. The program can be designed in such a way that it embodies or is able to carry out one or more of the methods described here in whole or in part, so that the

Processing unit can carry out steps of such methods and thus

in particular can configure the controller or classify or carry out the application or operate or control the robot. A

In one embodiment, a computer program product can have, in particular a non-volatile, storage medium for storing a program or with a program stored on it Process or one or more of its steps.

In one embodiment, one or more, in particular all, steps of the

The method is carried out completely or partially automatically, in particular by the system or its means.

In one embodiment, the system has the robot.

In one embodiment, a framework is created that enables motion or task learning to be implemented more efficiently using reinforcement learning methods. In one embodiment, parameters of the robot are queried simply and efficiently and / or the environmental model is recorded. In particular, in order to learn efficiently (more) and quickly (more) and / or not to block the real system, this is not carried out in one embodiment on the real system, but in a cloud simulation environment. This can advantageously enable a parallelization of the learning process and thereby an advantageous increase in speed and, in particular, a more robust model (through randomization of parameters). It should be emphasized again that the present invention in particular

- Configuring a controller of a robot to carry out a

Application using the robot (robot application);

- the training of one or more classification (KI) agents to classify a robot application; and

the implementation of a (robot) application using at least one or the robot

or means set up for this purpose, in particular (also) configuring the controller without performing the application, training the

Classification (KI) agents without performing the application, in particular training the classification (KI) agent or agents together with configuring the controller, but without performing the application, performing the

Application with already configured control and / or trained / -n

Classification (AI) agents, i.e. without configuring and / or without that

Training of the classification (KI) agent or agents, as well as the combination of configuration and / or training with the implementation or means (each) set up for this purpose. Correspondingly, a feature of the implementation of the application also includes in particular that the configuration of the control or training of the classification (KI) agent or agents is set up for this or in such a way

it is carried out that this feature is then used when the application is carried out with the configured control or the trained

Classification (AI) agent is realized.

Further advantages and features emerge from the subclaims and the exemplary embodiments. This shows, partly schematically:

1 shows a system according to an embodiment of the present invention;

Fig. 2: parts of the system;

3 shows a method according to an embodiment of the present invention;

4: a method according to a further embodiment of the present invention

Invention; and 5: a visualization of a stochastic parameter model and a robot parameter.

1 shows a system according to an embodiment of the present invention with a robot 1, a (robot) controller 2 that communicates with the robot 1 and a cloud 4, and a data input / output and processing device, in particular a computer 3.

A wizard runs on its user interface that guides a user through one or more of the processes described below:

In a first step of a method according to an embodiment of the present invention (FIG. 3: S10), a robot parameter and a start configuration are recorded. To make motion learning beneficial in a simulation environment

the parameters of the robot as well as the environmental model should be available in the cloud simulation environment as precisely as possible.

With the aid of a so-called asset administration shell (AAS), also called a digital twin, status and management data of the robot 1 are stored. An OPC UA information model is advantageously used for this. The robot's administration shell contains data such as the robot model,

Operating hours, current axis values (to determine a starting position), attached tools, etc. are available and are transferred to the cloud simulation environment. From this, the simulation environment can configure the simulation with regard to the robot (CAD model, dynamic parameters, tools, current axis configuration, possibly changed dynamic parameters due to service life, etc.)

In a second step (FIG. 3: S20) the environmental model is recorded. There are several options to choose from in one version:

- Transfer of a fully modeled CAD model including transformation to the robot coordinate system;

- Detection of the environment by a 3D camera, which is either hand-operated by a human or is mounted on the robot that

- is hand-held or

- runs a defined and collision-free trajectory In the case of manual guidance, it is also possible to record areas that are important for the task, for example a joining target, more precisely and from a small distance.

The environmental model generated in this way is now also transferred to the cloud simulation environment. A simple option here is to also store the data in the administration shell of the robot.

In a modification, the robot cell has an administration shell 10 (cf. FIG. 2), the environmental model and references to other administration shells involved. This means that the robot itself is interchangeable and the whole thing has a more modular structure than if all the information is in the robot's administration shell itself. The "Cell Manager" can then interact with the subcomponents, the

Control the simulation environment 20 (see FIG. 2) and the execution of the learning process.

In a third step (FIG. 3: S30) the learning objective is defined. A cost function is specified so that the reinforcement algorithm knows its goal. In particular, it is possible to specify the target in the guided wizard, for example, by the user guiding the robot to the joining target and repeating this a few times in order to minimize errors.

A manual demonstration is also used in one embodiment, depending on the reinforcement learning algorithm, to initialize the algorithm or to perform inverse reinforcement learning of the cost function. The trajectories of the demonstrations can also be stored in the administration shell.

In a fourth step (FIG. 3: S40), the task is learned in the cloud environment 4, preferably in parallel, by means of the deep reinforcement learning method.

The specific algorithm is advantageous

- Guided Policy Search;

- Soft Q-Learning;

- A3C

or similar. In order to overcome the simulation-reality gap, a

Randomization of the dynamic parameters made. If a vision system is involved, a flexible vision model is learned in one implementation using domain randomization.

A geometric path planner can plan non-contact path elements and, in the case of Guided Policy Search, initialize the Linear Quadratic Gaussian Controllers.

The result of the algorithm is the structure of the neural network and the trained weights of a neural network. In a modification, progressive nets can be used for later fine-tuning. The results of the simulation are sent back to the robot / edge controller.

In a fifth step (FIG. 3: S50) the model is downloaded to the robot or an edge controller.

The trained model can now be played back. In the administration shell of the simulation instance, parameters of the simulation and the learning algorithm can also be made available (e.g. learning rate, number of iterations, etc., which will later be used in

Fine tuning can be used). In particular, the ONNX exchange format, for example, can be used for exchanging the computation graph and the weights.

In an optional sixth step (Fig. 3: S60), the model is fine-tuned on the real system.

Depending on the quality of the simulation, the model is immediately ready-to-use or is further fine-tuned on the real system. This means that the reinforcement learning algorithm is trained further on the real system, with initialization using the weights and other parameters of the reinforcement algorithm being advantageous.

In a seventh step (FIG. 3: S70) the learned task can now be carried out.

FIG. 4 shows a method according to a further embodiment of the present invention, which can be carried out with the system of FIG. In a step S100, a random generator 3a (cf. FIG. 1) provided in this exemplary embodiment and therefore indicated by dashed lines is used to generate a stochastic value of a robot parameter and / or a

Environmental model parameters, for example a two-dimensional target position of a connector 1a guided by a robot in this exemplary embodiment in an area 10 (cf. FIG. 1), on the basis of a predetermined stochastic parameter model, in

Embodiment of a Gaussian distribution predetermined by a user with the aid of the wizard.

Fig. 5 shows an example of how this robot parameter and this

stochastic parameter model can be visualized in an image of this joining application by a marked area in the form of a circle around the mean or expected value of the Gaussian distribution for the two-dimensional target position in the (image of the) surface. The edge of the circle visualizes a maximum value of a deviation from the mean or expected value and the through

different hatchings indicated different coloring, in one

Execution of different (color) brightness, the marked area the respective probability that the target position is at this point.

In a step S200, the application is simulated on the basis of the determined stochastic value, i.e. with a stochastic target position, on the computer 3 or in the cloud 4.

A control agent is trained using reinforcement learning.

In a step S300 it is checked whether the control agent has already been trained sufficiently. If this is not the case (S300: “N”), the random number generator 3a generates a new stochastic target position with which a further simulation is carried out.

If the control agent has trained sufficiently (S300: “Y”), the controller 2 of the robot 1 is configured on the basis of the trained control agent (S400).

In addition, a classification agent is trained with the aid of the simulations carried out, for example anomaly or error detection is learned by machine (S500), ie those performed when training the control agent

Simulations used.

The real application is then carried out with the aid of the robot 1 with the controller configured in step S400 (S600) and in the process or subsequently classified by the anomaly or error detection (S700). The configured control can be trained further.

Although exemplary embodiments have been explained in the preceding description, it should be pointed out that a large number of modifications are possible. It should also be noted that the exemplary

Explanations are only examples that cover the scope of protection that

Applications and structure are not intended to restrict in any way. Rather, the person skilled in the art through the preceding description is a guide for

Implementation of at least one exemplary embodiment given, whereby various changes, in particular with regard to the function and arrangement of the described components, can be made without departing from the scope of protection as it emerges from the claims and their equivalents

Combinations of characteristics results.

Claims

1. A method for performing an application with the aid of at least one robot (1), the method comprising the repeated steps:

- Determining (S100) a stochastic value of at least one

Robot parameters and / or at least one environmental model parameter, in particular on the basis of a predetermined stochastic parameter model and / or with the aid of at least one random generator (3a); and

- Performing (S200) a, in particular multi-stage, simulation of the

Application based on the determined stochastic value;

the steps:

- Training (S200, S500) at least one control agent and / or

at least one classification agent using the simulations using machine learning, in particular training a first one

Control agents and / or first classification agents using first stages of the simulations, and at least one further control agent and / or further classification agents using further stages of the

Simulations; and

- Execution (S600) of the application using the robot;

and at least one of the steps:

- Configuring (S400) a controller of the robot with which the application is carried out in whole or in part, on the basis of the, in particular the, trained control agent; and or

- Classification (S700) of the application with the aid of the, in particular the, trained classification agent.

2. The method according to claim 1, characterized in that the at least one control agent and / or the at least one classification agent a machine-learned anomaly detection and / or machine-learned

Has error detection and / or at least one artificial neural network, in particular the control of the robot is configured based on the structure and / or weightings of the trained network, and / or the at least one control agent and / or the at least one classification agent using Reinforcement learning and / or trained using the robot.

3. The method according to any one of the preceding claims, characterized in that the at least one classification agent classifies the application on the basis of at least one, in particular migrating, time segment and / or during or after performing the application and / or the at least one control agent and / or the at least one classification agent is trained on the basis of at least one state variable that was not measured when the application was carried out.

4. The method according to any one of the preceding claims, characterized in that

- the robot parameter a start pose, at least one intermediate pose

and / or has a target pose of the application and / or a force parameter of a robot-internal force and / or an external force that acts at least temporarily on the robot and / or a kinematic, in particular dynamic, robot structure parameter; and or

the environmental model parameter has a kinematic, in particular dynamic, environmental, in particular load structure parameter; and or

- The robot parameter and / or the environmental model parameter is determined with the aid of a robot-assisted parameter identification.

5. The method according to the preceding claim, characterized in that it is checked whether the start pose, intermediate pose and / or target pose can be reached with the robot.

6. The method according to any one of the preceding claims, characterized in that the stochastic parameter model is specified based on the application and / or a user input, in particular selected from a plurality of different parameter models made available, and / or is marked in a mapping of the application Area is visualized and / or the robot and / or environmental model parameters are specified based on the application and / or a user input, in particular selected from a plurality of different parameters made available, and / or is visualized in a map of the application by a marked area .

7. The method according to any one of the preceding claims, characterized in that the configured control of the robot and / or the machine-learned anomaly detection and / or error detection using at least one further simulation, in particular on the basis of an automated or user specification of a value of at least one robot parameter and / or at least one environmental model parameter is tested and / or further trained with the aid of the robot.

8. The method according to any one of the preceding claims, characterized in that the stochastic parameter model is specified with the aid of machine learning.

9. A method for configuring a controller of a robot (1) for carrying out an application using the robot, the method comprising the repeatedly repeated steps of determining (S100) a stochastic value and carrying out (S200) a simulation and the step of training (S200) at least one control agent according to a method according to claim 1, in particular additionally one or more features of one or more of claims 2 to 8, as well as the step: configuring (S400) the controller of the robot for performing the application on the basis of, in particular the , trained control agents.

10. Method for training at least one classification agent to use

Classification of a robot application, in particular together with a configuration of a controller of a robot (1) for performing the

Application according to a method according to the preceding claim, wherein the method for training comprises the repeatedly repeated steps of determining (S100) a stochastic value and performing (S200) a simulation and the step of training (S500) at least one classification agent, in particular the step of training (S200, S500) at least one

Control agent and at least one classification agent, according to a method according to claim 1, in particular additionally one or more features of one or more of claims 2 to 8.

11. A method for performing an application using at least one robot (1), comprising the step of: performing (S600) the application using the robot, a control of the robot, with which the application is carried out in whole or in part, according to a method according to claim 9 is configured and / or the application is classified with the aid of at least one classification agent which is trained according to a method according to claim 10.

12. The method according to any one of the preceding claims, characterized in that at least one of the method steps has user input support by a software assistant, in particular a user interface guide.

13. The method according to any one of the preceding claims, characterized in that at least one of the method steps is carried out in a cloud (4).

14. System for performing an application using at least one robot (1), the system for performing a method according to one of the

preceding claims is set up and / or has:

Means for repeatedly determining a stochastic value of at least one robot parameter and / or at least one

Environmental model parameters, in particular on the basis of a predetermined one

stochastic parameter model and / or using at least one

Random generator (3a), and performing an, in particular multi-stage, simulation of the application on the basis of the determined stochastic value; and means for training at least one control agent and / or at least one classification agent using the simulations using machine learning, in particular training a first control agent and / or first classification agent using first stages of the simulations, and at least one further control Agents and / or others

Classification agents using further stages of simulations; and / or has:

Means for performing the application with the aid of the robot, wherein a

Control of the robot with which the application is carried out in whole or in part, configured on the basis of the, in particular the, trained control agent and / or the application is classified with the aid of the, in particular the, trained classification agent; and / or has:

Means for configuring a controller of the robot based on the,

in particular, the trained control agent for performing the

Application; and / or has:

Means for classifying the application with the aid of the, in particular the, trained classification agent.

15. Method for configuring a controller (2) of a robot (1) for

Perform a given task, with the steps:

Detecting (S10, S20) at least one robot parameter and at least one environmental model parameter;

Training (S40) an agent using at least one simulation based on the detected robot parameters and environmental model parameters using machine learning based on a predetermined cost function; and

Configuring (S50) the control of the robot based on the trained agent.

16. The method according to claim 16, characterized in that the predetermined

Task has at least one movement of the robot, in particular at least one scheduled environmental contact of the robot.

17. The method according to any one of claims 15 and 16, characterized in that the robot parameter is a kinematic, in particular dynamic,

Robot and / or load model parameters, a current robot pose

and / or operating time; and or

the environmental model parameter a CAD model parameter and / or a

Comprises robot positioning in the environment model; and or

is determined using at least one optical sensor.

18. The method according to claim 17, characterized in that the optical sensor is guided by a person or a, in particular hand-guided, robot.

19. The method according to any one of claims 15 to 18, characterized in that the agent has an artificial neural network, in particular the control of the robot based on the structure and / or weightings of the trained Network is configured and / or the agent is trained using reinforcement learning.

20. The method according to any one of claims 15 to 19, characterized in that the configured control of the robot by means of machine learning,

in particular reinforcement learning, is further trained with the aid of the robot (S60).

21. The method according to any one of claims 15 to 20, characterized in that at least one of the method steps has user input support by a software assistant, in particular a user interface guide, and / or the robot parameters and / or environmental model parameters in an administration shell and / or a data Is stored in the cloud.

22. A method for performing a given task using at least

of a robot, characterized in that a controller of the robot is configured according to a method according to one of claims 15 to 21.

23. System for performing a method according to any one of claims 15 to

22 is set up.

24. Computer program product with a program code, which is stored on a medium that can be read by a computer, for carrying out a method according to one of claims 1 to 13 or 15 to 22.