EP3993959A1 - Exécution d'une application à l'aide d'au moins un robot - Google Patents

Exécution d'une application à l'aide d'au moins un robot

Info

Publication number
EP3993959A1
EP3993959A1 EP20735370.7A EP20735370A EP3993959A1 EP 3993959 A1 EP3993959 A1 EP 3993959A1 EP 20735370 A EP20735370 A EP 20735370A EP 3993959 A1 EP3993959 A1 EP 3993959A1
Authority
EP
European Patent Office
Prior art keywords
robot
application
agent
parameter
trained
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP20735370.7A
Other languages
German (de)
English (en)
Inventor
Manuel KASPAR
Pierre VENET
Jonas SCHWINN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
KUKA Deutschland GmbH
Original Assignee
KUKA Deutschland GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from DE102019209616.6A external-priority patent/DE102019209616A1/de
Priority claimed from DE102020206924.7A external-priority patent/DE102020206924A1/de
Application filed by KUKA Deutschland GmbH filed Critical KUKA Deutschland GmbH
Publication of EP3993959A1 publication Critical patent/EP3993959A1/fr
Pending legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1602—Programme controls characterised by the control system, structure, architecture
    • B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • B25J9/1682—Dual arm manipulator; Coordination of several manipulators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1628—Programme controls characterised by the control loop
    • B25J9/163—Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1671—Programme controls characterised by programming, planning systems for manipulators characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/33—Director till display
    • G05B2219/33037—Learn parameters of network offline, not while controlling system

Definitions

  • the present invention relates in particular to a method for performing an application using at least one robot, a method for configuring a controller of a robot for performing an application or a predetermined task, a method for training at least one classification agent for classifying a robot application, a method for Carrying out a predetermined task with the aid of at least one robot with a correspondingly configured controller and a system and a computer program product for carrying out at least one of these methods.
  • One object of an embodiment of the present invention is to improve the implementation of an application or a given task with the aid of at least one robot.
  • One object of an embodiment of the present invention is to improve the configuration of a controller of the robot for carrying out the application or the specified task.
  • the present invention is to improve a classification of a robot application.
  • One object of an embodiment of the present invention is to improve control of a robot with which an application is carried out.
  • a method for - Configuring a controller of a robot for carrying out an application with the aid of the robot (robot application);
  • At least one classification (KI) agent to classify a robot application
  • the stochastic value for a simulation can be determined before the simulation is carried out and then used in it. Likewise, several stochastic values of the robot and / or the
  • Environmental model parameters are determined in advance and then one of these stochastic values is used for or in one of the simulations.
  • the method comprises the step:
  • At least one control (AI) agent and / or
  • the method comprises the step: single or multiple execution of the (real) application using the robot.
  • control (AI) agents can also use several control (AI) agents.
  • the robot or environmental model parameter (value) is randomized or the simulations with randomized robot or
  • machine learning can be improved in one implementation and more robust and / or faster in one implementation.
  • Robot or environmental model parameters (values) trained agent improve the implementation of the (real) application with the help of the robot, in particular the control of the robot and / or classification of the application, in particular robust (more) and / or flexible (e) l ( he) act.
  • this includes in particular an artificial intelligence agent, in particular one
  • AI agents or a classification (AI) agent.
  • determining a stochastic value comprises generating the value, in particular numerically and / or physically, and can in particular be generating the value.
  • the stochastic values on the basis of which the simulations are carried out are, in particular are, in one embodiment with the aid of at least one
  • Random generator in particular pseudo-random number generator, determined, in particular generated, and / or are stochastically or randomly distributed values, in one embodiment random numbers, in particular pseudo-random numbers, which in one embodiment are determined by the specified stochastic parameter model or satisfy it.
  • the stochastic parameter model has one or more stochastic parameters, in particular minimum, maximum, expected and / or mean value (s), variance (s), standard deviation (s), measure (s) of dispersion or the like , and / or a probability distribution, for example a
  • Gaussian or normal distribution a uniform distribution, or the like.
  • Environmental model parameters specify a minimum and maximum value as well as a uniform distribution and thus a stochastic parameter model, whereby corresponding stochastically (distributed values are then generated with the help of a (pseudo) random number generator and determined in this way on the basis of this predetermined stochastic parameter model and with the aid of this (pseudo) random number generator Likewise, the user and / or the
  • User input support for example, specify a specific Gaussian distribution and thus a different stochastic parameter model, with the help of a (pseudo) random number generator then again generated corresponding stochastically (distributed values and in this way on the basis of this other specified stochastic parameter model and with the aid of this
  • the determined stochastic values are (also) determined by the specified stochastic parameter model, for example limited by minimum and / or maximum value (s), scattered by variance (s) around an expected or mean value or like that.
  • a simulation is understood to mean, in particular, a simulation run or a numerical simulation of the application or its course over time.
  • a multi-stage simulation comprises two or more in one implementation
  • a first control agent is established using first stages or
  • Sections of the simulations and at least one further control agent with the aid of further stages or sections of the simulations and / or a first
  • Classification agent trained using the same or different first stages or sections of the simulations and at least one further classification agent using further stages or sections of the simulations, in the above example a first (control or classification) agent for joining the first Gear using the first simulation stages or simulations of joining the first gear and another (control or classification) agent for joining the further gear using the further simulation stages or simulations of joining the further gear.
  • an initial state for a later simulation stage is determined or specified on the basis of a final state or result of a previous simulation stage, in one embodiment this initial state also varies, in particular randomized, in particular on the basis of a user input or specification, will or can be.
  • a pose of the first gearwheel after its simulated joining can be used as a starting value in the further simulation stage and, if necessary, can be changed and / or randomized by a user.
  • the method comprises the step:
  • the simulations with stochastic or randomized values are used to learn machine control of the robot for performing the (real) application with the aid of the robot or to train one or more agents for this purpose.
  • the implementation of the (real) application can be improved with the aid of the robot, in particular the application can be implemented in a robust (more) and / or flexible (more) manner.
  • a control of the robot with which only a part of the application is or is to be carried out, is configured on the basis of the, in particular the, trained control agent.
  • the application can have one or more sections that are (should) be carried out with a (different) controller of the robot that is or is not configured on the basis of the trained control agent (s), as well as one or more sections, which are (should) be carried out with a controller of the robot, which are based on the trained
  • Control agent is or is configured.
  • a controller within the meaning of the present invention can in particular have, in particular be, a control device and / or a computer program, in particular (computer) program module or part.
  • the simulations with stochastic or randomized values are used in order to machine a classification of the (real) application or to learn one or more
  • the implementation of the (real) application can be improved with the aid of the robot, in particular the application is (more) robust and / or
  • Classification agent (s each) a machine-learned anomaly detection. Additionally or alternatively, in one embodiment the or one or more of the classification agent (s) has / have a machine-learned one
  • an anomaly detection comprises a classification of the application (s) carried out into normal (s) and abnormal applications).
  • the anomaly detection is or is machine learned, in particular only, on the basis of simulated applications labeled as normal and / or the anomaly detection classifies an application as abnormal if it deviates (too greatly) from the simulated applications labeled as normal.
  • error detection includes a classification of the
  • joining the first gear with the aid of the robot in particular on the basis of force and / or posture data of the robot, can be considered to be free of errors, as being attached, but not sufficiently deep and / or jammed, or as are classified as not joined if the force or pose data are sufficiently similar to the courses of appropriately labeled simulated applications and the agent classifies this real application into the corresponding error class.
  • Robot applications are used, as these are (only) difficult to learn by machine using (real) applications carried out with the robot.
  • Classification agent at least one artificial neural network
  • the control of the robot is configured based on the structure and / or weightings of the trained network.
  • control agent (s) and / or the or one or more of the classification agent (s) is trained with the aid of reinforcement learning, in particular deep reinforcement learning.
  • This is particularly suitable for configuring a controller for the robot and for classifying, in particular anomaly and / or error detection, of the application.
  • control agent (s) and / or the or one or more of the classification agent (s) is trained, in particular additionally, with the aid of the robot, in one embodiment based on one or more of the several (real) applications carried out with the help of the robot.
  • the corresponding agent can be used particularly advantageously when performing the real application with the aid of the robot and / or machine learning can be (further) improved.
  • Classification agent (s), trained on the basis of at least one state variable that is not measured when the application is carried out and cannot be measured in an embodiment.
  • Simulations are also calculable state variables, in particular are calculated that are not measured when the application is carried out.
  • the distance between the (first or additional) gear and a stop cannot be measured, for example because there is no corresponding sensor or the space between the gear and the stop is not accessible. In a simulation of the joining, however, this distance can be calculated and then used as a state variable for training, in particular in a quality criterion.
  • Control agents and / or classification agents used quality criterion, in particular a quality or cost function, based on at least one at
  • machine learning can be improved in one implementation and more robust and / or faster in one implementation.
  • Simulations that are (anyway) carried out or used to train at least one control agent, on the basis of which the control of the robot is configured, with which the (real) application is or should be carried out, also used or used for this purpose. use one or more
  • the method comprises
  • classification of the application with the aid of one or more of the trained classification agents
  • control and classification agent are or have been trained using the same simulations, wherein in one
  • the invention uses simulations on the basis of which, in particular by means of reinforcement learning, the controller is configured or with the aid of which the or one or more of the control agent (s) is or are trained, in one embodiment has been or are also used for training at least one machine-learned classification or the or one or more of the classification agents.
  • data in particular state variables, in one embodiment (temporal) state variable curves, in particular trajectories, of the application, in one embodiment of the robot, which are or have been calculated in simulations, in one embodiment, simulations with the aid of which the or one or more of the control agent (s) are or have been trained, stored and the or one or more of the classification agent (s) are trained with the aid of this stored data, in an execution following this
  • these data include poses of one or more robot-fixed references, in particular an end effector, TCPs, robot-guided tool or piece or the like, joint or axis divisions of the robot, internal and / or external forces on the robot, in particular joint and / or Driving forces, frictional forces, contact forces or the like, current variables, in particular
  • Following errors can in particular include force, position and / or speed errors.
  • Simulations in one embodiment simulations with the aid of which the or one or more of the control agent (s) are or have been trained, in particular using the stored data, those simulations or data are selected in which a quality criterion is met, and used to train the anomaly detection, or those simulations or data sorted into different defect classes on the basis of a quality criterion and used to train the defect detection.
  • trajj denotes the data of a simulation i
  • traj ⁇ traj_i ⁇ the amount of all data stored during the simulation
  • the machine-learned classification can in each case be improved in one embodiment, in particular learned more quickly and / or classified more precisely, more robustly and / or more reliably.
  • the one or more of the agent (s) classifies
  • the application on the basis of at least one, migrating in one execution, in particular
  • a continuous and / or serial evaluation is carried out in addition to or as an alternative to an evaluation of the complete application and the agent classifies the application on the basis of this continuous or serial evaluation.
  • Recurrent networks, Markov models or autoregressive networks are particularly suitable for this.
  • machine learning can be improved in one implementation and more efficient and / or faster in one implementation.
  • the one or more of the agent (s) classifies
  • the application while the application is being carried out.
  • this enables a reaction to the result of the classification.
  • the one that is just or currently being carried out corresponds to the result of the classification.
  • Application may be changed based on the classification, in one embodiment a corresponding signal is output in the event of a recognized anomaly and / or a recognized error and / or a movement of the robot is modified and / or a workpiece handled, in particular transported and / or processed during the application sorted out or reworked.
  • the or one or more of the agent (s), in particular the anomaly detection and / or the error detection classifies the application after the application has been carried out.
  • the robot parameter has a one-dimensional or multi-dimensional start pose, one or more one or more one-dimensional or multi-dimensional intermediate poses and / or a one-dimensional or multi-dimensional target pose of the application, in particular of the
  • the simulations of the application are based on stochastic (distributed or generated start,
  • the robot is checked whether (the stochastic value for) the starting pose, intermediate pose (s) and / or target pose can be achieved with the robot, in particular on the basis of a kinematic model of the robot. If the pose or the corresponding stochastic value of the starting pose, intermediate pose (s) and / or target pose can be achieved with the robot, in particular on the basis of a kinematic model of the robot. If the pose or the corresponding stochastic value of the starting pose, intermediate pose (s) and / or target pose can be achieved with the robot, in particular on the basis of a kinematic model of the robot. If the pose or the corresponding stochastic value of the starting pose, intermediate pose (s) and / or target pose can be achieved with the robot, in particular on the basis of a kinematic model of the robot. If the pose or the corresponding stochastic value of the starting pose, intermediate pose (s) and / or target pose can be achieved with the robot, in particular on the basis of a kinematic model of the
  • Robot parameter is not achievable, the value is determined again in one embodiment or until (it is determined that) the pose or the value can be reached with the robot, and then this value as the value determined when the simulation of the application is carried out used.
  • machine learning can be improved in one implementation and more efficient and / or faster in one implementation.
  • a pose within the meaning of the present invention can in particular have, in particular be, a one, two or three-dimensional position and / or one, two or three-dimensional orientation.
  • the robot parameter has a one- or multi-dimensional force parameter of a robot-internal force, in particular at least one axis and / or at least one end effector rigidity and / or damping. This can lead to wear and tear in one version
  • the robot parameter has a one-dimensional or multi-dimensional force parameter of an external force that acts on the robot at least, in one embodiment only temporarily, in particular an external force
  • a force within the meaning of the present invention can in particular have, in particular be, an antiparallel force pair or torque.
  • a force parameter can in particular include a force, but also a rigidity, a damping and / or a friction coefficient or the like.
  • the robot parameter has in one embodiment a one- or multi-dimensional kinematic, in one embodiment dynamic, robot structure parameter, in particular a one-dimensional or multi-dimensional dimension and / or a weight and / or one or multi-dimensional
  • Structural member groups or the like.
  • tolerances between robots of the same model or the like can be taken into account in one embodiment, thereby improving machine learning or the trained agent (s), in particular robust (more) and / or flexible (more), will.
  • the environmental model parameter in one embodiment has a one- or multi-dimensional kinematic, in one embodiment dynamic, environmental, in one embodiment load structure parameter, in particular a one or multi-dimensional pose and / or dimension and / or weight and / or moment of inertia an environmental, in particular load structure,
  • the robot parameters and / or the environmental model parameters are determined with the aid of a robot-supported parameter identification, for example minimum, maximum and / or mean value (s) of or for the stochastic parameter model.
  • the predetermined stochastic one is, in particular becomes
  • Parameter model based on a user input and / or application-specific predetermined, selected in one embodiment from a plurality of different parameter models made available.
  • a user can initially choose one of several items
  • Probability distributions for example a Gaussian distribution, a
  • Probability distribution for example a uniform distribution, for certain gripping applications a different probability distribution, for example a Gaussian distribution, or the like is selected and application-specific minimum and maximum values or the like are specified for this or here.
  • Mixed forms are also possible, in particular one
  • Robot and / or the environmental model parameters based on user input and / or specified application-specifically, selected in one embodiment from a plurality of different parameters made available.
  • Surface normals can be specified or selected as the target pose, whereas for drilling with a robot-guided drill in a flat surface, a one-dimensional distance to the surface along the drill axis is used as the target or
  • the area within the area in which the target position can be (stochastically) can be visualized in an image of the application, for example by a corresponding circular area, the possible orientations or angular positions around the surface normal, for example, by two appropriately twisted cuboids or workpiece avatars in the respective maximum possible deflections.
  • a probability distribution of the stochastic parameter model is visualized by a different color, in one embodiment different (color) brightness, of the marked area, the respective color or brightness (level) depending on the probability that the robot or environmental model parameter has the corresponding value.
  • the area within the area in which the target position can (stochastically) be visualized in an image of the application for example, by a corresponding circular area
  • areas of the circular area in which the target position is more likely for example are or become darker or a first area of the circular area in which the Target position lies with a first probability, for example is or is colored with a first color and / or brightness
  • at least one other area of the circular area in which the target position lies with a different probability is colored with a different color and / or brightness is or will.
  • a particularly suitable parameter model or particularly suitable parameters can be selected, in particular the speed and / or error tolerance of the input can be improved.
  • the configured control of the robot and / or the machine-learned anomaly detection and / or error detection is tested with the aid of at least one further simulation, in particular based on an automated specification or user specification of a value of at least one robot parameter and / or at least one environmental model parameter.
  • the user can pose a workpiece for the
  • test simulation Change the test simulation and then use the test simulation to check whether and how well the configured control or anomaly or error detection (for this) works or performs.
  • a test script can automate further simulation with the trained control agent or agents
  • Reinforcement learning further trained with the help of the robot, in particular on the basis of applications carried out with the help of the robot.
  • a control which is particularly advantageous in practice can be implemented in one embodiment, in particular in combination.
  • the stochastic parameter model is, in particular, specified with the aid of machine learning.
  • a stochastic parameter model is, in particular, specified with the aid of machine learning.
  • Parameter model (KI) agent based on earlier applications carried out with the aid of the robot, which have been classified with the aid of a classification agent trained according to a method described here and / or in which the
  • Control of the robot, with which these applications were carried out have been configured on the basis of a control agent trained according to a method described here, on the basis of the results of the earlier carried out
  • Control agents used stochastic parameter model, which predefined the stochastic parameter model, which is then used in a method described here for performing simulations to train the at least one classification agent, with the help of which a new application is then classified, and / or the at least one control agent is used, with the help of which a controller is then configured to carry out a new application.
  • a particularly advantageous, in particular realistic, stochastic parameter model can be used, preselected in one embodiment, in particular by the user input support or the software assistant.
  • simulated applications can also be used as earlier applications for machine learning for specifying the stochastic parameter model.
  • one or more of the steps include one of the following
  • the robot and / or the environmental model parameter and / or the stochastic parameter model is preselected from a plurality of different parameters or parameter models made available, in particular application-specific and / or by the
  • a particularly suitable parameter model or particularly suitable parameters can be selected, in particular the speed and / or error tolerance of the input can be improved.
  • one or more of the steps of one of the methods described here are carried out in a cloud.
  • this method can advantageously be carried out in parallel and / or quickly and / or in a distributed manner.
  • a controller of a robot for performing a predetermined task can be configured particularly advantageously in one embodiment.
  • the robot has a stationary or mobile, in particular mobile, base and / or a robot arm with at least three, in particular at least six, in one embodiment at least seven joints or joints. (Movement) axes, in one version swivel joints or axes.
  • the present invention is particularly suitable for such robots because of its kinematics, variability and / or complexity.
  • the specified task has at least one movement of the robot, in particular at least one scheduled environmental contact of the robot, that is to say in particular can have robot-assisted gripping and / or joining.
  • the present invention is due to its
  • the robot has parameters
  • a one or more dimensional kinematic, in particular dynamic, load model parameter in particular one or more dimensions, masses, centers of gravity and / or inertia;
  • a current robot pose in particular one or more current axis or
  • the environmental model parameter has a one- or multi-dimensional CAD model parameter and / or a, in particular current, robot positioning in the environmental model and / or is determined with the aid of at least one optical sensor, in particular a camera.
  • this optical sensor is guided, in particular held or carried by a person, in another development by one
  • Robot which for its part travels a programmed or automatically determined path, in particular by means of collision avoidance, or is or is guided manually or by forces exerted manually on the robot.
  • the agent has an artificial neural network.
  • the control of the robot is then configured on the basis of the structure and / or weightings of the trained network, this structure and / or Transferring weights in one execution to the controller of the robot. Additionally or alternatively, the agent is implemented using
  • Reinforcement learning preferably deep reinforcement learning, trained.
  • control of the robot is trained further by means of machine learning, in particular reinforcement learning, preferably deep reinforcement learning, with the aid of the real robot.
  • Environmental model parameters are stored at least temporarily in an administration shell and / or in a data cloud.
  • a controller of the robot in a method for performing a predetermined task with the aid of at least one robot, is configured according to a method described here. Accordingly, in one embodiment, an inventive
  • a system in particular in terms of hardware and / or software, in particular in terms of programming, is set up to carry out one or more methods described here. It shows in a
  • the system comprises:
  • Classification agents using first stages of the simulations, and at least one further control agent and / or further classification agents using further stages of the simulations.
  • system has means for
  • system has means for in one embodiment
  • system has means for
  • Robot with which the application is carried out in whole or in part, is configured on the basis of the, in particular the, trained control agent and / or the
  • Application is classified with the aid of, in particular, the trained classification agent.
  • system or its means has:
  • Error detection and / or at least one artificial neural network and / or means for training at least one control agent and / or at least one classification agent with the aid of reinforcement learning and / or with the aid of the robot; and or
  • Robot is reachable; and or
  • Means for visualizing the stochastic parameter model in an image of the application through a marked area Means for visualizing the stochastic parameter model in an image of the application through a marked area
  • Means for performing at least one of the method steps in a cloud Means for performing at least one of the method steps in a cloud.
  • a means within the meaning of the present invention can be designed in terms of hardware and / or software, in particular one that is data or signal-connected, in particular digital, processing, preferably with a memory and / or bus system. in particular have a microprocessor unit (CPU), graphics card (GPU) or the like, and / or one or more programs or program modules.
  • CPU microprocessor unit
  • GPU graphics card
  • Processing unit can be designed to receive commands as one in one
  • a storage system can have one or more,
  • the program can be designed in such a way that it embodies or is able to carry out one or more of the methods described here in whole or in part, so that the
  • Processing unit can carry out steps of such methods and thus
  • a computer program product can have, in particular a non-volatile, storage medium for storing a program or with a program stored on it Process or one or more of its steps.
  • the method is carried out completely or partially automatically, in particular by the system or its means.
  • the system has the robot.
  • a framework is created that enables motion or task learning to be implemented more efficiently using reinforcement learning methods.
  • parameters of the robot are queried simply and efficiently and / or the environmental model is recorded.
  • this is not carried out in one embodiment on the real system, but in a cloud simulation environment. This can advantageously enable a parallelization of the learning process and thereby an advantageous increase in speed and, in particular, a more robust model (through randomization of parameters).
  • Classification (KI) agents without performing the application, in particular training the classification (KI) agent or agents together with configuring the controller, but without performing the application, performing the
  • AI agents i.e. without configuring and / or without that
  • Training of the classification (KI) agent or agents as well as the combination of configuration and / or training with the implementation or means (each) set up for this purpose.
  • a feature of the implementation of the application also includes in particular that the configuration of the control or training of the classification (KI) agent or agents is set up for this or in such a way
  • FIG. 1 shows a system according to an embodiment of the present invention
  • Fig. 2 parts of the system
  • Invention and 5: a visualization of a stochastic parameter model and a robot parameter.
  • FIG. 1 shows a system according to an embodiment of the present invention with a robot 1, a (robot) controller 2 that communicates with the robot 1 and a cloud 4, and a data input / output and processing device, in particular a computer 3.
  • a wizard runs on its user interface that guides a user through one or more of the processes described below:
  • a robot parameter and a start configuration are recorded.
  • the parameters of the robot as well as the environmental model should be available in the cloud simulation environment as precisely as possible.
  • AAS asset administration shell
  • OPC UA information model is advantageously used for this.
  • the robot's administration shell contains data such as the robot model,
  • the environmental model generated in this way is now also transferred to the cloud simulation environment.
  • a simple option here is to also store the data in the administration shell of the robot.
  • the robot cell has an administration shell 10 (cf. FIG. 2), the environmental model and references to other administration shells involved. This means that the robot itself is interchangeable and the whole thing has a more modular structure than if all the information is in the robot's administration shell itself.
  • the "Cell Manager" can then interact with the subcomponents, the
  • a third step (FIG. 3: S30) the learning objective is defined.
  • a cost function is specified so that the reinforcement algorithm knows its goal.
  • a manual demonstration is also used in one embodiment, depending on the reinforcement learning algorithm, to initialize the algorithm or to perform inverse reinforcement learning of the cost function.
  • the trajectories of the demonstrations can also be stored in the administration shell.
  • a fourth step (FIG. 3: S40), the task is learned in the cloud environment 4, preferably in parallel, by means of the deep reinforcement learning method.
  • Randomization of the dynamic parameters made. If a vision system is involved, a flexible vision model is learned in one implementation using domain randomization.
  • a geometric path planner can plan non-contact path elements and, in the case of Guided Policy Search, initialize the Linear Quadratic Gaussian Controllers.
  • the result of the algorithm is the structure of the neural network and the trained weights of a neural network.
  • progressive nets can be used for later fine-tuning.
  • the results of the simulation are sent back to the robot / edge controller.
  • a fifth step (FIG. 3: S50) the model is downloaded to the robot or an edge controller.
  • the trained model can now be played back.
  • parameters of the simulation and the learning algorithm can also be made available (e.g. learning rate, number of iterations, etc., which will later be used in
  • Fine tuning can be used).
  • the ONNX exchange format for example, can be used for exchanging the computation graph and the weights.
  • the model is immediately ready-to-use or is further fine-tuned on the real system.
  • FIG. 4 shows a method according to a further embodiment of the present invention, which can be carried out with the system of FIG.
  • a random generator 3a (cf. FIG. 1) provided in this exemplary embodiment and therefore indicated by dashed lines is used to generate a stochastic value of a robot parameter and / or a
  • Environmental model parameters for example a two-dimensional target position of a connector 1a guided by a robot in this exemplary embodiment in an area 10 (cf. FIG. 1), on the basis of a predetermined stochastic parameter model, in
  • Embodiment of a Gaussian distribution predetermined by a user with the aid of the wizard Embodiment of a Gaussian distribution predetermined by a user with the aid of the wizard.
  • Fig. 5 shows an example of how this robot parameter and this
  • stochastic parameter model can be visualized in an image of this joining application by a marked area in the form of a circle around the mean or expected value of the Gaussian distribution for the two-dimensional target position in the (image of the) surface.
  • the edge of the circle visualizes a maximum value of a deviation from the mean or expected value and the through
  • a step S200 the application is simulated on the basis of the determined stochastic value, i.e. with a stochastic target position, on the computer 3 or in the cloud 4.
  • a control agent is trained using reinforcement learning.
  • step S300 it is checked whether the control agent has already been trained sufficiently. If this is not the case (S300: “N”), the random number generator 3a generates a new stochastic target position with which a further simulation is carried out.
  • the controller 2 of the robot 1 is configured on the basis of the trained control agent (S400).
  • a classification agent is trained with the aid of the simulations carried out, for example anomaly or error detection is learned by machine (S500), ie those performed when training the control agent
  • the real application is then carried out with the aid of the robot 1 with the controller configured in step S400 (S600) and in the process or subsequently classified by the anomaly or error detection (S700).
  • the configured control can be trained further.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Automation & Control Theory (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Fuzzy Systems (AREA)
  • Feedback Control In General (AREA)
  • Manipulator (AREA)

Abstract

Selon la présente invention, un procédé d'exécution d'une application à l'aide d'au moins un robot (1) comporte les étapes répétées plusieurs fois : la détermination (S100) d'une valeur stochastique d'au moins un paramètre de robot et/ou d'au moins un paramètre de modèle d'environnement, en particulier sur la base d'un modèle de paramètre stochastique prédéfini et/ou à l'aide d'au moins un générateur de nombres aléatoires (3a) ; et l'exécution (S200) d'une simulation, en particulier à plusieurs phases, de l'application sur la base de la valeur stochastique déterminée ; les étapes : l'entraînement (S200, S500) d'au moins un agent de commande et/ou d'au moins un agent de classification à l'aide des simulations au moyen d'un apprentissage automatique, en particulier l'entraînement d'un premier agent de commande et/ou d'un premier agent de classification à l'aide de premières phases des simulations, et d'au moins un autre agent de commande et/ou d'un autre agent de classification à l'aide d'autres phases des simulations ; et l'exécution (S600) de l'application à l'aide du robot ; ainsi qu'au moins une des étapes : la configuration (S400) d'une commande du robot, avec laquelle l'application est exécutée intégralement ou partiellement, sur la base du, en particulier, des agents de commande entraînés ; et/ou la classification (S700) de l'application à l'aide du, en particulier, des agents de classification entraînés.
EP20735370.7A 2019-07-01 2020-06-29 Exécution d'une application à l'aide d'au moins un robot Pending EP3993959A1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE102019209616.6A DE102019209616A1 (de) 2019-07-01 2019-07-01 Durchführen einer vorgegebenen Aufgabe mithilfe wenigstens eines Roboters
DE102020206924.7A DE102020206924A1 (de) 2020-06-03 2020-06-03 Durchführen einer Applikation mithilfe wenigstens eines Roboters
PCT/EP2020/068241 WO2021001312A1 (fr) 2019-07-01 2020-06-29 Exécution d'une application à l'aide d'au moins un robot

Publications (1)

Publication Number Publication Date
EP3993959A1 true EP3993959A1 (fr) 2022-05-11

Family

ID=71401784

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20735370.7A Pending EP3993959A1 (fr) 2019-07-01 2020-06-29 Exécution d'une application à l'aide d'au moins un robot

Country Status (4)

Country Link
US (1) US20220339787A1 (fr)
EP (1) EP3993959A1 (fr)
CN (1) CN114051444B (fr)
WO (1) WO2021001312A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110000785B (zh) * 2019-04-11 2021-12-14 上海交通大学 农业场景无标定机器人运动视觉协同伺服控制方法与设备
DE102021200569A1 (de) 2021-01-22 2022-07-28 Robert Bosch Gesellschaft mit beschränkter Haftung Vorrichtung und Verfahren zum Trainieren eines Gaußprozess-Zustandsraummodells
DE102021126188A1 (de) 2021-10-08 2023-04-13 Dürr Systems Ag Überwachungsverfahren für einen Roboter und zugehörige Roboteranlage
KR20240092243A (ko) * 2022-12-14 2024-06-24 한국전자기술연구원 자산관리쉘 정보 모델을 활용한 장비 연동 및 데이터 시각화 방법

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7822749B2 (en) * 2005-11-28 2010-10-26 Commvault Systems, Inc. Systems and methods for classifying and transferring information in a storage network
US8150101B2 (en) * 2006-11-13 2012-04-03 Cybernet Systems Corporation Orientation invariant object identification using model-based image processing
US9566710B2 (en) * 2011-06-02 2017-02-14 Brain Corporation Apparatus and methods for operating robotic devices using selective state space training
US9764468B2 (en) * 2013-03-15 2017-09-19 Brain Corporation Adaptive predictor apparatus and methods
CN105228712B (zh) * 2013-05-31 2017-04-12 安凯公司 用于操纵、移动和/或重定向组件的移动代理
US9242372B2 (en) * 2013-05-31 2016-01-26 Brain Corporation Adaptive robotic interface apparatus and methods
US9314924B1 (en) * 2013-06-14 2016-04-19 Brain Corporation Predictive robotic controller apparatus and methods
US20150005937A1 (en) * 2013-06-27 2015-01-01 Brain Corporation Action selection apparatus and methods
US9296101B2 (en) * 2013-09-27 2016-03-29 Brain Corporation Robotic control arbitration apparatus and methods
DE102015204641B4 (de) * 2014-06-03 2021-03-25 ArtiMinds Robotics GmbH Verfahren und System zur Programmierung eines Roboters
US9630318B2 (en) * 2014-10-02 2017-04-25 Brain Corporation Feature detection apparatus and methods for training of robotic navigation
US9717387B1 (en) * 2015-02-26 2017-08-01 Brain Corporation Apparatus and methods for programming and training of robotic household appliances
US9840003B2 (en) * 2015-06-24 2017-12-12 Brain Corporation Apparatus and methods for safe navigation of robotic devices
MX2018000942A (es) * 2015-07-24 2018-08-09 Deepmind Tech Ltd Control continuo con aprendizaje de refuerzo profundo.
EP3360086A1 (fr) * 2015-11-12 2018-08-15 Deepmind Technologies Limited Apprentissage de réseaux neuronaux utilisant une mémoire d'expérience hiérarchisée
WO2018071392A1 (fr) * 2016-10-10 2018-04-19 Deepmind Technologies Limited Réseaux neuronaux de sélection d'actions devant être exécutées par un agent robotique
US20190126472A1 (en) * 2017-10-27 2019-05-02 Deepmind Technologies Limited Reinforcement and imitation learning for a task
US20190138880A1 (en) * 2017-11-03 2019-05-09 Drishti Technologies, Inc. Workspace actor selection systems and methods
CN108052004B (zh) * 2017-12-06 2020-11-10 湖北工业大学 基于深度增强学习的工业机械臂自动控制方法
US11340079B1 (en) * 2018-05-21 2022-05-24 AI Incorporated Simultaneous collaboration, localization, and mapping
CN109952582B (zh) * 2018-09-29 2023-07-14 区链通网络有限公司 一种强化学习模型的训练方法、节点、系统及存储介质

Also Published As

Publication number Publication date
CN114051444B (zh) 2024-04-26
US20220339787A1 (en) 2022-10-27
WO2021001312A1 (fr) 2021-01-07
CN114051444A (zh) 2022-02-15

Similar Documents

Publication Publication Date Title
EP3993959A1 (fr) Exécution d'une application à l'aide d'au moins un robot
DE102014108287B4 (de) Schnelles Erlernen durch Nachahmung von Kraftdrehmoment-Aufgaben durch Roboter
DE112019002310B4 (de) Ausführen einer "peg in hole"-aufgabe mit unbekannter neigung
DE112010000775B4 (de) Industrierobotersystem
DE102014103738B3 (de) Visuelle fehlersuche bei roboteraufgaben
DE102015010124B4 (de) Roboterprogramm-Erzeugungsvorrichtung, die Roboterprogramm zum Reduzieren von Stössen der Gelenke des Roboters erzeugt
DE102019119319B4 (de) Abtastsystem, Arbeitssystem, Verfahren zum Anzeigen von Augmented-Reality-Bildern, Verfahren zum Speichern von Augmented-Reality-Bildern und Programme dafür
DE102019109624B4 (de) Roboterbewegungseinlernvorrichtung, Robotersystem und Robotersteuerung
DE102013202378B4 (de) Prozedurales Gedächtnislernen und Robotersteuerung
DE102019122865B4 (de) Erfassungssystem, Arbeitssystem, Anzeigeverfahren für ein Erweiterte-Realität-Bild und Programm
DE112019001256B3 (de) Steuervorrichtung, Arbeitsroboter, Programm und Steuerverfahren
DE102020110252B4 (de) Vibrationsanzeigeeinrichtung, Betriebsprogrammerstellungseinrichtung und System
DE102017001131B4 (de) Verfahren und System zum Betreiben eines Roboters
DE102019203821B3 (de) Verfahren und System zum Durchführen einer Roboterapplikation
DE102019118637A1 (de) Automatische pfadgenerierungsvorrichtung
EP4013574A1 (fr) Système et procédé d'automatisation destinés à la manipulation de produits
EP3760390A1 (fr) Exécution d'une tâche prédéterminée à l'aide d'au moins un robot
DE102017007359B4 (de) Verfahren und System zum Überprüfen und/oder Modifizieren eines Arbeitsprozesses eines Roboters
DE102019205651B3 (de) Verfahren und System zum Ausführen von Roboterapplikationen
DE112019007579B4 (de) Numerische-Steuerung-Vorrichtung und Maschinelles-Lernen-Gerät
DE102018112370B4 (de) Richtungsabhängige Kollisionsdetektion für einen Robotermanipulator
EP3731043A1 (fr) Procédé d'évaluation de la sécurité, programme informatique, support d'enregistrement lisible par machine et dispositif d'évaluation de la sécurité
DE102020206924A1 (de) Durchführen einer Applikation mithilfe wenigstens eines Roboters
WO2017080649A2 (fr) Robots sensibles grâce à la présence de capteurs situés devant le premier axe de robot
DE102020103852B4 (de) Erzeugen und Optimieren eines Steuerprogramms für einen Robotermanipulator

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20211129

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230528

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20240221