DE102022119202A1 - Method and system for coordinating collaborative robots with payload/load/priority awareness and collaborative robots - Google Patents

Abstract

The invention creates a method for determining a mission plan and/or mission parameters for an autonomous mobile and/or collaborative robot, which is set up to accommodate one or more payloads, taking into account priority data of various tasks to be accomplished by the robot, comprising the following steps: Receiving, in particular incoming by the robot, in particular starting from a facility communication point, payload metadata for each payload of the robot, in particular regarding payload properties, payload target determinations, mission context data for an order involving the payload and/or combinations thereof, Determine a mission plan and/or mission parameters for the robot to execute missions based on the missions and payload metadata.

Description

TECHNICAL FIELD

The present invention relates generally to the technical field of robotics, autonomous mobile robots (AMRs) and in particular collaborative robots (cobots) and safety and precision technology in the workplace and in assembly plants.

BACKGROUND

Cobot and/or AMR systems have been used in more and more facilities in recent years. This has led to human workers increasingly sharing the same space with cobot systems. Such robots are designed to be particularly collaborative. There are often numerous such cobots in a system, with the individual cobots carrying out one or more tasks.

A robot often includes a robotic device with multiple joint actuators and at least one end effector.

The patent document US2017291315A1 discloses a system for determining and promoting the safety of a robot payload, comprising: a controller; and a robot controllable by the control unit, the robot comprising: at least one payload area configured to carry a payload; a sensor configured to detect at least a payload mass and a payload distribution, the sensor being further configured to alert the controller regarding the payload mass and/or the payload distribution, the controller being further configured to, in response to the alarm, the Determine the safety of the payload and/or promote the safety of the payload.

The patent document US20200009727A1 discloses a system and method for detecting and correcting the position of the robot payload. The method includes moving a payload through a motion path near at least one sensor. Capturing edges of the payload so that at least three points are captured on at least two edges. Detecting a position when the at least one sensor detects at least one edge of the payload.

The systems and procedures US20210247776A1 use distributed ledger technology (DLT) to enable decentralized control of cooperative tasks performed by a variety of robots. Characteristics of the variety of robots can be stored in a distributed ledger, which can be provided by a blockchain or a distributed database system. When a service request is received, a set of tasks can be identified for providing the requested service, and the robot characteristics recorded in the distributed ledger can be used to determine a list of robot candidates whose characteristics match the set of tasks. A smart contract can be used to select one or more candidate robots to perform the task and to verify whether the selected robot(s) completed the task successfully. State information related to the operation of the selected robot(s) can be monitored to verify task completion.

However, the state of the art has the disadvantage that the payloads are only evaluated with regard to their geometric and/or gravitational properties.

The state of the art also has the disadvantage that the cobots often have difficulty coordinating their own tasks and often hinder each other when carrying out them. In particular, priorities associated with payloads as well as the behavior of other robots at a certain point in time are regularly not taken into account, which means that important work is carried out too late and the overall efficiency of the cobot system is reduced.

It is therefore an object of the present invention to further develop collaborative and/or AMR robot systems in such a way that they work precisely and safely, especially in shared work environments with people. A high level of precision and flexibility of the robots should be guaranteed and costs should be minimized as best as possible without compromising on the quality of the product and the results achieved with it.

SUMMARY

The present invention provides the method according to claim 1. Accordingly, a method for determining a mission plan and / or mission parameters for an autonomous mobile and / or collaborative robot is provided, which is set up to accommodate one or more payloads, taking into account priority data from different ones Orders to be carried out by robots, comprising the following steps: Receiving, in particular incoming, by the robot, in particular outgoing from a facility communication point, payload metadata per payload of the robot, in particular on payload properties, payload target determinations, mission context data on an order involving the payload and/or combinations thereof, defining a mission plan and/or mission parameters for the robot Execute jobs based on the jobs and payload metadata.

This makes the robot aware of its payloads. It can take priorities into account, for example in the order order, as well as, for example, pay particular attention to the nature, weight, shape, fragility, durability, etc. of certain payloads. By using this technology, the robot also behaves more “intuitively” from a human perspective. For example, emergencies are given higher priority.

He does all of this autonomously. He is therefore not dependent on a “central dictate”, i.e. a centrally made decision, from a “Central Facility Communications Point” or similar.

The overall efficiency of the system increases through the use of such autonomously deciding robots.

The payload metadata can be recorded by sensors, for example. For example, these sensors may or may not be arranged on the robots/cobots. In one example, they are not and the data is collected centrally, through a Central Facility Communications Point or similar. The data collected about its payloads that is relevant to the respective cobot is then communicated to it so that the cobot can decide autonomously. The calculation steps required for this can, for example, also be outsourced so that they do not take place on the cobot itself, for example in a computing cloud. What is important, however, is that from a logical point of view the cobot decides and can/may decide autonomously. For example, no specific constraints are communicated to the cobot that restrict it as to which orders must be completed first, for example.

The more “intuitive” behavior of the cobots improves the system dynamics of the entire system, the more cobots/AMRs according to the invention participate in the system.

The invention further creates a cobot or AMR with payload and/or priority awareness according to claim 6. Accordingly, an autonomous mobile and/or collaborative robot, in particular cobot, with payload and/or priority awareness is provided, comprising: a payload holder for receiving a or multiple payloads, an interface for receiving orders, an interface for receiving payload metadata per payload, in particular on payload properties, mission context data for an order involving the payload and / or combinations thereof, a computing unit for determining a mission plan and /or mission parameters for the robot to execute the orders based on the orders and the payload metadata.

In addition, the invention creates the cobot/AMR system according to claim 10. Accordingly, a system is provided which comprises: one or more autonomous mobile and/or collaborative robots, sensors, arranged on at least one of the robots and/or arranged outside the robots , at least one central or decentralized computing unit, in particular computing cloud, wherein the sensors are set up to record data on payloads carried by the robots, to process them by a computing unit and to provide them in the form of payload metadata and/or to the respective robots communicate.

An AI (e.g. a machine learning model) can be used to generate the mission parameters and/or mission plan from the payload metadata. These models produce particularly “intuitive” behavior, which also replicates human analog behavior/decisions as best as possible - and at the same time is less error-prone than real human behavior (keyword: “human error”).

Example parameters that can be set are the order of orders and/or locations to be visited, but also navigation and/or kinematic variables, such as maximum permitted angles, speeds and/or (positive and negative) accelerations and corresponding acceleration values. and brake profiles. However, the invention in no way intends to limit itself with regard to these parameters.

The robots can communicate their autonomous decisions and plans, for example in the form of mission plans and/or mission parameters, to a central facility management. They can also communicate relevant information to the other robots, especially nearby other robots. This allows the robots to have additional coordinated behavior that is coordinated with each other.

The invention makes a lasting contribution to technical progress in the technical field of collaborative robots (autonomous mobile robots), especially in the context of comprehensive robot systems that can be managed and scaled well and efficiently.

BRIEF DESCRIPTION OF THE DRAWINGS

• 1: ein Ablauf-/Flussdiagramm für ein erfindungsgemäßes Robotersystem (Ausführungsform),
• 2: ein Ablauf-/Flussdiagramm für beispielhafte festzulegende Variablen auf Grundlage von beispielhaften Nutzlast-Metadaten im Rahmen eines erfindungsgemäßen Robotersystems (Ausführungsform),
• 3: eine beispielhafte geometrische Route eines erfindungsgemäßen Roboters (Ausführungsform).

Preferred embodiments of the present disclosure are described below with reference to the following figures:

• 1 : a sequence/flow diagram for a robot system according to the invention (embodiment),
• 2 : a sequence/flow diagram for exemplary variables to be determined based on exemplary payload metadata in the context of a robot system according to the invention (embodiment),
• 3 : an exemplary geometric route of a robot according to the invention (embodiment).

DETAILED DESCRIPTION

The 1 - 3 describe some aspects of the present invention using an embodiment.

The 1 shows a sequence/flow diagram for a robot system according to the invention (embodiment).

An external sensor device 102 collects data. For example, these are sensors in the facility. The sensors sense the cobots and the payloads they manage. An extraction of payload metadata 104 can take place based on the raw data 103 obtained.

Communication steps can, for example, take place using the facility communication point 101 (Facility Communications Access Point).

The payload metadata 105 (payload information), which is relevant for a robot, is thus sent to the robot 106 to determine a mission plan and/or mission parameters.

Example payload metadata 105 includes product serial numbers, weight, weight distribution, destination, shelf life, delivery and/or unloading requirements, etc.

The 2 shows a process/flow diagram for exemplary variables to be determined based on exemplary payload metadata in the context of a robot system according to the invention (embodiment).

The payload metadata 105 can be processed by the robot/cobot. They can contain individual data records for different payloads, symbolized here by the data records 110a, 110b, 110c. Hierarchical structures, multiple data sets and/or (partial) copies of data sets can also be provided, each of which is adapted to its intended purpose (symbolized here by data sets 111 and 112).

For example, according to the invention, mission parameters 120, navigation parameters 121 and kinematic parameters 122 are generated.

The 3 shows an exemplary geometric route of a robot according to the invention (embodiment).

The AMR/Cobot 131 is equipped with three payloads 130a-c. Using the invention, it autonomously comes to the conclusion that the payload 130b enjoys special priority and must therefore be processed first in the order order (“P1”).

Therefore, the AMR/Cobot follows route 140b. It does this even though route 140b is longer than the shorter route 140a. After analyzing the payload metadata, the other orders “P3” are not urgent enough to deny priority to the order “P1”.

In a preferred embodiment, this weighing, using the payload metadata, was carried out using artificial intelligence, in particular a machine learning model. As a result, the robots make objective and rational decisions, which are more precise and less error-prone than human ones, but also reflect and replicate “intuitive” human behavior particularly well.

The embodiments described schematically here can be further developed by numerous details of the invention, in particular those already described above.

Although some aspects have been described in the context of a device, it is clear that these aspects also represent a description of the corresponding method, where a block or a device corresponds to a method step or a function of a method step. Analogous to this, aspects that arise within the framework of a Method step are described, also a description of a corresponding block or element or a property of a corresponding device.

Embodiments of the invention can be implemented in a computer system. The computing system may be a local computing device (e.g., personal computer, laptop, tablet computer, or cell phone) with one or more processors and one or more storage devices, or may be a distributed computing system (e.g., a cloud computing system with one or more processors or one or more Storage devices distributed at various locations, for example at a local client and/or one or more remote server farms and/or data centers). The computer system may include any circuit or combination of circuits. In one embodiment, the computer system may include one or more processors, which may be of any type. As used herein, processor may mean any type of computing circuit such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set microprocessor (CISC), a reduced instruction set microprocessor (RISC), a very long instruction word (Very Long Instruction Word; VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system may include a custom-built circuit, an application-specific integrated circuit (ASIC), or the like, such as one or more circuits (e.g., a communications circuit) for use with wireless devices such as . B. cell phones, tablet computers, laptop computers, walkie-talkies and similar electronic systems. The computer system may include one or more storage devices, which may include one or more storage elements suitable for the particular application, such as a main memory in the form of a random access memory (RAM), one or more hard drives, and/or a or multiple drives that handle removable media such as CDs, flash memory cards, DVDs, and the like. The computer system may also include a display device, one or more speakers, and a keyboard and/or controller, which may include a mouse, trackball, touch screen, voice recognition device, or any other device that allows a system user to enter information into the computer system and information to receive from the same.

Some or all of the method steps may be performed by (or using) a hardware device, such as a processor, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the key method steps may be performed by such a device.

Depending on particular implementation requirements, embodiments of the invention may be implemented in hardware or software. The implementation may be performed with a non-volatile storage medium such as a digital storage medium such as a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM and EPROM, an EEPROM or a FLASH memory in which electronically readable control signals are stored, which interact (or can interact) with a programmable computer system in such a way that the respective method is carried out. Therefore, the digital storage medium can be computer readable.

Some embodiments according to the invention include a data carrier with electronically readable control signals that can interact with a programmable computer system to perform one of the methods described herein.

In general, embodiments of the present invention may be implemented as a computer program product with program code, the program code being effective for executing one of the methods when the computer program product is running on a computer. The program code can, for example, be stored on a machine-readable medium.

Further exemplary embodiments include the computer program for carrying out one of the methods described herein, which is stored on a machine-readable medium.

In other words, an embodiment of the present invention is therefore a computer program having program code for carrying out one of the methods described herein when the computer program runs on a computer.

A further embodiment of the present invention is therefore a storage medium (or a data carrier or a computer-readable one Medium) comprising a computer program stored thereon for carrying out one of the methods described herein when executed by a processor. The data carrier, digital storage medium or recorded medium is usually tangible and/or non-seamless. Another embodiment of the present invention is an apparatus as described herein that includes a processor and the storage medium.

A further exemplary embodiment of the invention is therefore a data stream or a signal sequence that represents the computer program for carrying out one of the methods described herein. The data stream or the signal sequence can, for example, be configured so that they are transmitted via a data communication connection, for example via the Internet.

Another embodiment includes a processing means, for example a computer or a programmable logic device, configured or adapted to carry out any of the methods described herein.

Another embodiment includes a computer on which the computer program for executing one of the methods described herein is installed.

Another embodiment according to the invention includes an apparatus or system configured to transmit (e.g., electronically or optically) to a receiver a computer program for carrying out one of the methods described herein. The receiver may be, for example, a computer, a mobile device, a storage device, or the like. The device or system may, for example, include a file server for transmitting the computer program to the recipient.

In some embodiments, a programmable logic device (e.g., a field programmable gate array, FPGA) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are preferably performed by each hardware device.

Embodiments may be based on using artificial intelligence, in particular a machine learning model or machine learning algorithm. Machine learning can refer to algorithms and statistical models that computer systems can use to perform a specific task without using explicit instructions, rather than relying on models and inference. In machine learning, for example, instead of a rule-based transformation of data, a transformation of data can be used that can be derived from an analysis of historical and/or training data. For example, the content of images can be analyzed using a machine learning model or using a machine learning algorithm. In order for the machine learning model to analyze the content of an image, the machine learning model can be trained using training images as input and training content information as output. By training the machine learning model with a large number of training images and/or training sequences (e.g. words or sentences) and associated training content information (e.g. labels or annotations), the machine learning model “learns” the content of the images recognize so that the content of images not included in the training data can be recognized using the machine learning model. The same principle can be used for other types of sensor data as well: By training a machine learning model using training sensor data and a desired output, the machine learning model "learns" a transformation between the sensor data and the output, which can be used to create a Provide output based on non-training sensor data provided to the machine learning model. The provided data (e.g. sensor data, metadata and/or image data) can be pre-processed to obtain a feature vector, which is used as input for the machine learning model.

Machine learning models can be trained using training input data. The examples above use a training method called “supervised learning.” In supervised learning, the machine learning model is trained using a plurality of training samples, where each sample may include a plurality of input data values and a plurality of desired output values, that is, each training sample is assigned a desired output value. By specifying both training samples and desired output values, the machine learning model “learns” what output value to provide based on an input sample that is similar to the samples provided during training. In addition to supervised learning, semi-supervised learning can also be used. In semi-supervised learning, some of the training samples are missing desired output value. Supervised learning can be based on a supervised learning algorithm (e.g. a classification algorithm, a regression algorithm, or a similarity learning algorithm). Classification algorithms can be used when the outputs are restricted to a limited set of values (categorical variables), i.e. the input is classified as one from the limited set of values. Regression algorithms can be used when the outputs show any numerical value (within a range). Similarity learning algorithms can be similar to both classification and regression algorithms, but are based on learning from examples using a similarity function that measures how similar or related two objects are. In addition to supervised learning or semi-supervised learning, unsupervised learning can be used to train the machine learning model. In unsupervised learning, input data may be provided (only) and an unsupervised learning algorithm may be used to find structure in the input data (e.g. by grouping or clustering the input data, finding commonalities in the data). Clustering is the assignment of input data comprising a plurality of input values into subsets (clusters) such that input values within the same cluster are similar according to one or more (predefined) similarity criteria, while they are dissimilar to input values comprised in other clusters.

Reinforcement learning is a third group of machine learning algorithms. In other words, reinforcement learning can be used to train the machine learning model. Reinforcement learning involves training one or more software actors (so-called “software agents”) to perform actions in an environment.

Based on the actions taken, a reward is calculated. Reinforcement learning is based on training the one or more software agents to select the actions in such a way that the cumulative reward is increased, resulting in software agents that become better at the task they are given (as by increasing rewards proven).

Furthermore, some techniques can be applied to some of the machine learning algorithms. For example, feature learning can be used. In other words, the machine learning model may be at least partially trained using feature learning and/or the machine learning algorithm may include a feature learning component. Feature learning algorithms, called representation learning algorithms, can preserve the information in their input but transform it in such a way that it becomes useful, often as a preprocessing stage before performing classification or prediction. For example, feature learning can be based on principal component analysis or cluster analysis.

In some examples, anomaly detection (i.e., outlier detection) may be used, which aims to provide identification of input values that raise suspicion because they differ significantly from the majority of input and training data. In other words, the machine learning model may be trained at least in part using anomaly detection, and/or the machine learning algorithm may include an anomaly detection component.

In some examples, the machine learning algorithm may use a decision tree as a prediction model. In other words, the machine learning model can be based on a decision tree. In a decision tree, the observations about an item (e.g., a set of input values) can be represented by the branches of the decision tree, and an output value corresponding to the item can be represented by the leaves of the decision tree. Decision trees can support both discrete values and continuous values as output values. When discrete values are used, the decision tree can be called a classification tree; when continuous values are used, the decision tree can be called a regression tree.

Association rules are another technique that can be used in machine learning algorithms. In other words, the machine learning model can be based on one or more association rules. Association rules are created by identifying relationships between variables in large sets of data. The machine learning algorithm may identify and/or utilize one or more ratio rules that represent the knowledge derived from the data. The rules can e.g. B. be used to store, manipulate or apply knowledge.

Machine learning algorithms are usually based on a machine learning model. In other words, the term “machine learning algorithm” can mean a set of instructions that can be used to train a machine to create, train or use a learning model. The term “machine learning model” may mean a data structure and/or a set of rules that represents the learned knowledge (e.g. based on the training performed by the machine learning algorithm). In embodiments, the use of a machine learning algorithm may imply the use of an underlying machine learning model (or a plurality of underlying machine learning models). The use of a machine learning model may imply that the machine learning model and/or the data structure/set of rules that is/are the machine learning model is trained by a machine learning algorithm.

For example, the machine learning model can be an artificial neural network (ANN). ANNs are systems inspired by biological neural networks such as those found in a retina or brain. ANNs include a plurality of interconnected nodes and a plurality of connections, called edges, between the nodes. There are usually three types of nodes, input nodes that receive input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node can represent an artificial neuron. Each edge can send information from one node to another. The output of a node can be defined as a (nonlinear) function of the inputs (e.g. the sum of its inputs). A node's inputs can be used in the function based on a "weight" of the edge or node providing the input. The weight of nodes and/or edges can be adjusted in the learning process. In other words, training an artificial neural network may include adjusting the weights of the nodes and/or edges of the artificial neural network, i.e. H. to achieve a desired output for a given input.

Alternatively, the machine learning model can be a support vector machine, a random forest model or a gradient boosting model. Support Vector Machines are supervised learning models with associated learning algorithms that can be used to analyze data (e.g. in a classification or regression analysis). Support vector machines can be trained by providing input with a plurality of training input values belonging to one of two categories. The support vector machine can be trained to assign a new input value to either category. Alternatively, the machine learning model may be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network can represent a set of random variables and their conditional dependencies using a directed acyclic graph. Alternatively, the machine learning model may be based on a genetic algorithm, which is a search algorithm and heuristic technique that mimics the process of natural selection.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2017291315 A1 [0004]
• US 20200009727 A1 [0005]
• US 20210247776 A1 [0006]

Claims (13)

Method for determining a mission plan and/or mission parameters for an autonomous mobile and/or collaborative robot, which is set up to accommodate one or more payloads, taking into account priority data of various tasks to be accomplished by the robot, comprising the following steps: - Receiving, in particular incoming by the robot, in particular starting from a facility communication point, payload metadata per payload of the robot, in particular regarding payload properties, payload target determinations, mission context data for an order involving the payload and/or combinations thereof , - Set a mission plan and/or mission parameters for the robot to execute missions based on the missions and payload metadata. Procedure according to Claim 1 , wherein the step of determining a mission plan and/or mission parameters includes use of an artificial intelligence, in particular a machine learning model or machine learning algorithm, wherein at least one payload metadata of the payload metadata is entered into the artificial intelligence. Procedure according to Claim 2 , wherein the at least one mission plan and / or mission parameters are provided by the artificial intelligence, in particular an order of execution of the orders is provided by the artificial intelligence. Method for operating a robot with payload and/or priority awareness, comprising the method for determining a mission plan and/or mission parameters according to one of Claims 1 or 2 , further comprising the following steps: - receiving at least one order by the robot - receiving at least one payload by the robot - executing the orders by the robot based on the mission plan and / or mission parameters. Method according to one of the preceding claims, further comprising a step of communicating a mission plan and/or mission parameters to at least one other robot, in particular a closest robot, and/or to a facility communication point, in particular to a robot central location. Autonomous mobile and/or collaborative robot, in particular cobot, with payload and/or priority awareness, comprising: - A payload holder for holding one or more payloads - An interface for receiving orders - An interface for receiving payload metadata per payload, in particular on payload properties, mission context data on an order involving the payload and/or combinations thereof - A computing unit for setting a mission plan and/or mission parameters for the robot to execute the orders based on the orders and the payload metadata. Autonomous mobile and/or collaborative robot Claim 6 , wherein the computing unit determines: - Mission parameters, in particular a sequence of places to be visited and / or order sequences, based on the payload metadata - Navigation parameters, in particular a maximum speed, an acceleration and / or braking profile, based on the payload metadata - Kinematic parameters , in particular a maximum inclination and / or a minimum and maximum braking profile, based on the payload metadata Autonomous mobile and/or collaborative robot Claim 6 or 7 , whereby the computing unit carries out the determination locally on the robot, in particular without accepting further restrictions through a device communication point and/or taking them into account when determining. Autonomous mobile and/or collaborative robot according to one of the Claims 7 - 8th , wherein the mission plan and/or mission parameters are communicated to a facility communications point after determination. System comprising: - one or more autonomous mobile and/or collaborative robots - sensors, arranged on at least one of the robots and/or arranged outside the robots - at least one central or decentralized computing unit, in particular computing cloud, the sensors being set up for this purpose , to collect data on payloads carried by the robots, to process them by a computing unit and in the form of Provide payload metadata and/or communicate with the respective robots. System after Claim 10 , which is set up to allow the robots to carry out tasks autonomously based on the payload metadata. Procedure for operating a system Claim 10 or 11 with at least one autonomous mobile and/or collaborative robot, in particular cobot, according to one of Claims 6 - 9 , wherein at least one of the at least one robot is set up to carry out a method according to one of Claims 1 - 5 to effect. A computer program comprising instructions which, when the program is executed by a computer, cause it to carry out the method according to any of the Claims 1 - 5 or 12 to carry out.

Images