WO2022242966A1 - Robot fleet management method and system using a graph neural network - Google Patents

Robot fleet management method and system using a graph neural network Download PDF

Info

Publication number
WO2022242966A1
WO2022242966A1 PCT/EP2022/059955 EP2022059955W WO2022242966A1 WO 2022242966 A1 WO2022242966 A1 WO 2022242966A1 EP 2022059955 W EP2022059955 W EP 2022059955W WO 2022242966 A1 WO2022242966 A1 WO 2022242966A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
robot
neural network
graph
route
Prior art date
Application number
PCT/EP2022/059955
Other languages
French (fr)
Inventor
Michael Colin HOY
Dimitrios Panagiotis GEROMICHALOS
Rahul Singh
Original Assignee
Continental Automotive Technologies GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive Technologies GmbH filed Critical Continental Automotive Technologies GmbH
Priority to DE112022002704.5T priority Critical patent/DE112022002704T5/en
Publication of WO2022242966A1 publication Critical patent/WO2022242966A1/en

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0027—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement involving a plurality of vehicles, e.g. fleet or convoy travelling
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning

Definitions

  • the invention relates to a computer implemented robot management method and a robot management system.
  • the invention further relates to a graph neural network adapted for use in said method and system.
  • Fleet management systems usually incorporate functions like task allocation, constraint satisfaction and cooperative path planning, and are in widespread use across many logistic industries. These are traditionally solved using a combination of meta-heuristic search algorithms (e.g., Tabu search, simulated annealing, deconfliction based planning) and graph search algorithms (e.g., A * planning). For complicated, multifaceted problems, additional engineering work is typically needed to develop and tune application specific metaheuristics algorithms.
  • meta-heuristic search algorithms e.g., Tabu search, simulated annealing, deconfliction based planning
  • graph search algorithms e.g., A * planning
  • Another idea is the use of reinforcement learning to train a neural network to learn policies that are directly optimized for each problem type. For example, a solution is to use multi-agent RL, and have each robot independently learn a navigation policy that causes optimized cooperative behavior to emerge.
  • ICLR 2021 discloses a graph neural network architecture over which reinforcement learning can be performed to obtain online policies for multi-robot task allocation problems.
  • US 10474 141 B2 discloses a method and an apparatus for controlling movement of transporting devices.
  • Transport containers are stored in stacks arranged in a facility having pathways arranged in a grid-like structure above the stacks.
  • the transporting devices are configured to operate on the grid-like structure.
  • US 10 089 586 B2 discloses a job management system for processing job requests in an automated physical environment, such as a factory, hospital, order processing facility or office building, wherein the job requests are handled by a fleet of autonomously - navigating mobile robots.
  • the invention provides a computer implemented robot management method for operating a robot management system that controls at least one robot in an environment, the method comprising: a) providing an input map of the environment, the input map having a plurality of nodes and at least one connecting edge that connects two nodes, each node indicating a location that the robot is able to visit within the environment, each connecting edge indicating a route that the robot is able to take between connected nodes through the environment; b) collating the current state of the robot management system, the current state of the robot management system including at least one node state of each node, at least one edge state of each edge, and at least one robot state of each robot; c) compiling a list of route constraints, wherein the route constraints relate to information indicative for properties of a route that the robot is able to take to a node; d) initializing a graph neural network system, wherein the graph neural network system is configured for determining one single-step command for each robot and the graph neural network system comprises a main graph autoencoder network
  • step c), step d) or step e) are performed multiple times.
  • steps c) and d), steps c) and e), or steps d) and e) are performed multiple times.
  • steps c), d), and e) are performed multiple times.
  • steps c), d), and e) are performed multiple times.
  • an intermediate state is propagated from one iteration to the next iteration. This intermediate state is handled similarly within the message passing graph neural network.
  • step e) a hard attention mask is generated and step e) is performed only on a subgraph of the graph neural network system, wherein the subgraph is determined by the hard attention mask.
  • the hard attention mask is generated by randomly selecting one of the route constraints; determining a subgraph of the graph neural network system to which the route constraint relates; updating the internal neural state of the graph neural network system and/or updating the single-step command at each node for the robot.
  • the subgraph is determined such that the deviation of the subgraph from the shortest route is below a predetermined threshold.
  • step e it is randomly selected, whether in step e), the whole graph neural network system or a portion of the graph neural network system is executed.
  • each single-step command is determined after the graph neural network system was iteratively executed for a predetermined number of iterations.
  • the graph neural network system is configured as a message passing graph neural network system.
  • the main graph autoencoder network includes a first fully connected neural network for each node
  • each auxiliary graph autoencoder network includes a second fully connected neural network that is associated with a particular route constraint.
  • step e) the node state is passed to the first and second fully connected networks and the route constraint is only passed to the associated second fully connected neural network.
  • the main graph autoencoder network includes a first aggregator for each node, and the output of any, some, or all of the first and second fully connected neural networks is passed to the first aggregator.
  • the graph neural network system comprises a second aggregator for each node that is associated with that node, wherein the second aggregator of any node receives the output of any, some, or all of the first aggregators of the unassociated other nodes.
  • the main graph autoencoder network includes a third fully connected neural network for each node.
  • each auxiliary graph autoencoder includes a fourth fully connected neural network associated with a particular route constraint.
  • the third fully connected network receives the output of the first aggregator and/or the second aggregator that is associated with that node.
  • each fourth fully connected neural network receives the output of the first aggregator and/or the second aggregator that is associated with that node.
  • the main graph autoencoder network includes a third aggregator for each node, wherein for each node the output of any, some, or all of the third and fourth fully connected neural networks is passed to the third aggregator of that node, and this output is passed on as output of the graph neural network system.
  • any, some, or all of the fourth fully connected networks of that node may be passed on as additional output.
  • the node state includes a pick-up indicator, that indicates that a package is to be picked up from that node.
  • the node state includes a drop-off indicator, that indicates that a package is to be dropped off at that node.
  • the node state includes a battery charger indicator, that indicates that a battery charger is available at that node.
  • the note state includes a capacity indicator, that indicates whether or how much robots or packages that node can still accommodate.
  • the edge state includes an edge capacity indicator, that indicates whether or how much robots are able to travel along that edge, preferably without delay or congestion.
  • the edge state includes a distance indicator, that is indicative of the physical distance between the two nodes connected by that edge.
  • the robot state includes a carrying capacity indicator, that is indicative whether or how much freight that robot is able to carry.
  • the robot state includes a battery status indicator, that is indicative of the charge status of the robot batter, in particular whether the robot needs to go to a charging station, i.e., a node with a battery charger.
  • the invention further provides a robot management system for controlling a plurality of robots in an environment, the environment comprising a plurality of locations that are connected by at least one route, the robot management system comprising at least one robot that is configured to received commands generated by a computer system, and a computer system that is configured to perform one, some, or all of the steps of the method previously described.
  • the invention further provides a computer program, a computer readable storage medium or a data carrier signal comprising instructions which, upon execution by a computer system, cause the computer system to carry out one, some or all of the steps of the previously described method.
  • the invention further provides a computer implemented graph neural network system configured for controlling robots in an environment, the environment comprising a plurality of locations represented by nodes that are connected by at least one route that is represented by at least one connecting edge, the environment further comprising route constraints relating to information indicative for properties of a route that the robot is able to take to a node, the graph neural network system comprising a main graph autoencoder network for each node characterized by an auxiliary graph autoencoder network for each route constraint.
  • the graph neural network system is configured as a message passing graph neural network system.
  • the main graph autoencoder network includes a first fully connected neural network for each node
  • each auxiliary graph autoencoder network includes a second fully connected neural network that is associated with a particular route constraint.
  • the graph neural network system is configured such that the node state is passable to the first and second fully connected networks and the route constraint is only passable to the associated second fully connected neural network.
  • the main graph autoencoder network includes a first aggregator for each node, and the output of any, some, or all of the first and second fully connected neural networks is passable to the first aggregator.
  • the graph neural network system comprises a second aggregator for each node that is associated with that node, wherein the second aggregator of any node is configured to receive the output of any, some, or all of the first aggregators of the unassociated other nodes.
  • the main graph autoencoder network includes a third fully connected neural network for each node.
  • each auxiliary graph autoencoder includes a fourth fully connected neural network associated with a particular route constraint.
  • the third fully connected network is configured to, for each node, receive the output of the first aggregator and/or the second aggregator that is associated with that node.
  • each fourth fully connected neural network is configured to, for each node, receive the output of the first aggregator and/or the second aggregator that is associated with that node.
  • the main graph autoencoder network includes a third aggregator for each node, wherein for each node the output of any, some, or all of the third and fourth fully connected neural networks is passable to the third aggregator of that node, and this output is able to be passed on as output of the graph neural network system.
  • the output of any, some, or all of the fourth fully connected networks of that node are passable on as additional output.
  • the node state includes a pick-up indicator, that indicates that a package is to be picked up from that node.
  • the node state includes a drop-off indicator, that indicates that a package is to be dropped off at that node.
  • the node state includes a battery charger indicator, that indicates that a battery charger is available at that node.
  • the note state includes a capacity indicator, that indicates whether or how much robots or packages that node can still accommodate.
  • the edge state includes an edge capacity indicator, that indicates whether or how much robots are able to travel along that edge, preferably without delay or congestion.
  • the edge state includes a distance indicator, that is indicative of the physical distance between the two nodes connected by that edge.
  • the robot state includes a carrying capacity indicator, that is indicative whether or how much freight that robot is able to carry.
  • the robot state includes a battery status indicator, that is indicative of the charge status of the robot batter, in particular whether the robot needs to go to a charging station, i.e., a node with a battery charger.
  • the invention aims to coordinate robots on highly detailed world maps (described as graphs with many nodes and edges) or environments. For such applications, there is only little work applying reinforcement learning with neural networks.
  • Multi-agent RL approaches require a separate neural network instance for each agent, and thus have a limited capacity for each individual network to accept input information. For example, for large graphs, it would be computationally infeasible for every agent to analyze the entire graph individually. The solution to this is only to provide coarse, high level information to the neural network inputs, which makes certain types of applications difficult.
  • the instant invention proposes a new graph neural network architecture for solving arbitrary task allocation, constraint satisfaction and cooperative routing problem (s).
  • s arbitrary task allocation, constraint satisfaction and cooperative routing problem
  • the current state of the system is collated. This includes the state of each graph node, which may include information about the state of robots currently positioned at each graph nodes. Some input data may be computed for each node, such as, whether there are pickups or drop-offs at the node, whether there is a battery charger at the node, what the robot carrying capacity is (if there is a robot on the node), or what the battery status is (if there is a robot on the node).
  • Route constraints are typically relating to a particular route. For instance, a route constraint can define where the robot goes next, in order to arrive at its goal node. As an example, two robots that have picked up items, each require route constraints to ensure that the items are routed to the correct locations.
  • route constraints are route constraints. Those are referred to as non route constraints. For example, if a robot requires use of a charger, it is not relevant which charger is used; in general, any charger can be used.
  • Route constraints that are calculated as input date for each node include, for example, the distance to the robot, the distance to the goal node, the scoring of the route (if multiple routes are available). Scoring can be done based on any property of the route. Usually, this will be the time the robot takes for completing the route, including possible detours due to congestion, for example.
  • the system may have additional constraints, such as the robots have to be in a charged state.
  • the state information is presented to the input of a graph autoencoder network. For each route constraint, a separate auxiliary graph autoencoder to initialized.
  • the initialized graph autoencoder network is executed.
  • any graph neural network structure can be used, although the description will focus on an example of a message passing graph neural network.
  • the graph autoencoder network was trained for the particular deployment environment or world map using reinforcement learning. Since, in general the underlying system model is known, the network may be trained in a simulator using a range of techniques such as Q-learning, stochastic value gradients, or AlphaGo style policy improvement.
  • the graph autoencoder network During execution the graph autoencoder network generates/determines a single-step command for each node and robot. The single-step commands are then sent to the corresponding robots.
  • the steps so far can be referred to “node level attention”, since each node, robot and route attribute is considered and as a result, each node and robot are given a result from the graph autoencoder network.
  • route level attention an additional network mechanism which is referred to as “route level attention”.
  • the steps described above have run the network once so far, however, here the network may run multiple times, each time potentially only computing a portion or subgraph of the neural network, e.g., by using a hard attention mask.
  • the invention proposes to randomly select one of the route constraints. Next, the subgraph “near” the shortest route is isolated. Thus, a small deviation between the isolated subgraph and the shortest route is allowed.
  • the graph neural network is executed on the isolated subgraph and the output of the graph neural network is updated, which can include values for the internal neural state and/or interim actions for the robot at each node.
  • Fig. 1 depicts an environment in which robots are navigating
  • Fig. 2 depicts an autoencoder
  • Fig. 3 depicts an updated environment
  • Fig. 4 depicts a block diagram of a graph neural network.
  • Fig. 1 schematically depicts a robot management system 10.
  • the robot management system 10 is configured to control a plurality of robots 12 in an environment 14.
  • the environment 14 comprises a plurality of locations 16.
  • the robots 12 are capable of travelling between two locations 16 via a path 18.
  • the environment 14 is abstracted into a machine-readable format.
  • the environment 14 is represented as a route graph 20 that includes a plurality of nodes 22 and at least one connecting edge 24.
  • Each node 22 represents one of the physical locations 16 in the physical environment 14.
  • Each connecting edge 24 connects two nodes 22 and represents a physical path 18, that one of the robots 12 can take between two physical locations 16.
  • connecting edge 24 is represented as a straight line
  • the corresponding physical path 18 need not be straight and can further include curves or ramps or other non-straight elements.
  • Each robot 12 has associated with it a robot state.
  • the robot state contains information indicative of the respective operational state of the robot 12.
  • the robot state can include information about the current battery charge, remaining battery charge, estimated charge consumption, the carrying capacity, the current robot position (in physical space and/or in graph space), and any other property that describes the operational situation of the robot 12.
  • each node 22 has associated with it a node state, which can include information about the location 16 that the corresponding node 22 represents.
  • the information may include, whether a package 26 is to be picked up from the node 22, whether or which of the packages 26 is to be dropped off at the node 22, whether the node 22 has a charging capability, i.e. the location 16 has a charging dock 28, and whether the node 22 is capable of accepting a robot 12 (i.e. whether the location 16 has enough space to accommodate a robot 12).
  • Each of the connecting edges 24 has associated with it an edge state, which contains information relating to properties of the specific connecting edge 24, i.e. the path 18 which is represented by the connecting edge 24. This information may include the capacity of the connecting edge 24 (for example, how many robots 12 can occupy the connecting edge 24 at the same time), the distance between the two nodes 22 that are connected by the specific connecting edge 24, or other properties of the connecting edge 24 as well as the associated path 18.
  • Fig. 1 depicts an example for a current state of the robot management system 10. Therein, a first robot 30 should go to the charging dock 28. A second robot 32 and a third robot 34 have each picked up a package 26 and should deliver it.
  • the second robot 32 has picked up the package 26 from a first pickup 36 and the third robot 34 has picked a package 26 from a second pick up 38.
  • the first robot 30 and the third robot 34 are still at their respective starting nodes 40.
  • the second robot 32 has already moved from its starting node 40 to an intermediate node 42. None of the robots 30, 32, 34 are at their respective goal nodes 44.
  • the first robot 30 has as its goal node 44 the charging dock 28.
  • the second robot 32 has as its goal node 44 a first drop-off 46.
  • the third robot 34 has as its goal node 44 a second drop-off 48.
  • the previously described configuration of the robot management system 10 is known as the system state and includes the various robot, node, and edge states.
  • the robot management method operates the robot management system 10 such that each robot 12 receives a single-step command 50, which instructs the respective robot 12 what action it should perform next.
  • the route constraints relate to information that is indicative for properties of a route that each robot 12 is able to take towards its goal node 44.
  • the route may be composed of one or more paths 18.
  • the route constraints may include the total distance to travel to the goal node, the (estimated) time to arrival, the route capacity, or any other information that is relevant to the journey of the respective robot 12 along the route.
  • a graph neural network system 52 is initialized based on the current state of the robot management system 10 and the list of route constraints.
  • the graph neural network system 52 comprises a main graph auto encoder network 54.
  • the graph neural network system 52 further comprises a separate auxiliary graph auto encoder network 56 for each route constraint within the list of route constraints.
  • a plurality of first routes 58 that are associated with the third robot 34 initialize a first of the auxiliary graph auto encoder networks 56.
  • the plurality of first routes 58 preferably include all possible routes for the third robot 34 from its current node 22 to its goal node 44, wherein the first routes 58 are selected such that they deviate from the shortest possible route by not more than a predetermined deviation.
  • the third robot 34 can either go directly from the second pick up 38 to the second drop-off 48 or can go on a detour 60 via another node 22.
  • a second of the auxiliary graph auto encoder networks 56 is initialized based on a plurality of second routes 62, which are routes that the second robot 32 is able to take from its current node 22 to its goal node 44.
  • the second routes 62 are selected in the same manner as the first routes 58 so that they do not deviate by more than a predetermined amount from the shortest route.
  • another auxiliary graph auto encoder network 56 may be initialized for the first robot 30 (not shown here).
  • the graph neural network system 52 has been trained by methods known per se in the art.
  • the training on the environment 14 is preferably done by simulator and can use a variety of techniques, such as cue-learning, stochastic value gradients, or policy improvement.
  • the graph neural network system 52 After the graph neural network system 52 is initialized, it is executed and results in a set of single-step commands 50 that are associated with each node 22 and corresponding robot 12.
  • the first robot 30 receives an instruction to move north to the next node 22, i.e. to the node 22 that has the charging dock 28.
  • the second robot 32 has received an instruction to also go north to the next node 22 in order to avoid collision with the first robot 30 while still moving closer to the first drop-off 46.
  • the third robot 34 is instructed to move north towards its goal node 44 which is the second drop-off 48.
  • each node 22 and each connecting edge 24 can only accommodate a single robot 12 at a time.
  • the graph neural network system 52 has managed the movement of the robots 12 such that each robot 12 is closer to its goal node 44, while simultaneously avoiding that any connecting edge 24 and any node 22 is occupied by two robots 12 at the same time.
  • the graph neural network system 52 is configured as message passing graph neural network system.
  • Each node state 54 and each route constraint information 66 is input as initial information (far left in Fig. 4).
  • the main graph auto encoder network 54 comprises a first fully connected neural network 68 or first FCN 68 (hereinafter, “fully connected neural network” is referred to as “FCN”).
  • each auxiliary graph auto encoder network 56 comprises a second FCN 70 for each route constraint information 66.
  • the node state 64 is passed to both the first and second FCNs 68, 70.
  • the route constraint information 66 is only passed to the second FCN 70 of its corresponding auxiliary graph auto encoder network 56.
  • the main graph auto encoder network 54 further comprises a first aggregator 72 associated with each node state 64.
  • the output of the first and second FCNs 68, 70 is passed on to the first aggregator 72.
  • the output of the second FCN 70 is additionally passed on without aggregation.
  • the graph neural network system 52 further comprises a second aggregator 74 for each node state 64.
  • the second aggregator 74 of a specific node 22 is connected to the first aggregator 72 of the other nodes 22.
  • the main graph auto encoder network 54 comprises a third FCN 76.
  • the auxiliary graph auto encoder network 56 comprises a fourth FCN 78 for each route constraint information 66.
  • the third FCN 76 receives the output from the first and second aggregators 72, 74.
  • the fourth FCN 78 receives the output of the second aggregator 74 and the output of their corresponding second FCN 70.
  • the fourth FCN 78 of one route constraint information 66 does not receive the output from the second FCN 70 of any other route constraint information 66.
  • the main graph auto encoder network 54 comprises a third aggregator 80 that receives the output from the third and fourth FCNs 76, 78.
  • the graph neural network system 52 has a first output 82, which corresponds to the output of the third aggregator 80. Furthermore, the graph neural network system 52 has a second output 84 for each route constraint information 66. The first output 82 corresponds to the output of the third aggregator 80 and the second output 84 corresponds to the output of the fourth FCN 78 of the corresponding route constraint information 66.
  • the graph neural network system 52 with this architecture is able to generate the single-step commands 50 for each robot 12 based on the system state and an arbitrary number of route constraints.
  • the invention provides a robot management system 10 that utilizes a graph neural network system 52.
  • the graph neural network system 52 has a main graph auto encoder network 54, which deals with the system state as a whole and further comprises an auxiliary graph auto encoder network 56 for each route constraint of the robot management system 10, in order to generate commands 50 for the robots 12.
  • the robots 12 move about an environment 14 based on the single-step commands 50.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Medical Informatics (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

In order to improve scalability and manage an arbitrary number of route constraints, the invention provides a robot management system (10) that utilizes a graph neural network system (52). The graph neural network system (52) has a main graph auto encoder network (54), which deals with the system state as a whole and further comprises an auxiliary graph auto encoder network (56) for each route constraint of the robot management system (10), in order to generate commands (50) for the robots (12). The robots (12) move about an environment (14) based on the single-step commands (50).

Description

DESCRIPTION
Robot fleet management method and system using a graph neural network
TECHNICAL FIELD
The invention relates to a computer implemented robot management method and a robot management system. The invention further relates to a graph neural network adapted for use in said method and system.
BACKGROUND
Fleet management systems usually incorporate functions like task allocation, constraint satisfaction and cooperative path planning, and are in widespread use across many logistic industries. These are traditionally solved using a combination of meta-heuristic search algorithms (e.g., Tabu search, simulated annealing, deconfliction based planning) and graph search algorithms (e.g., A* planning). For complicated, multifaceted problems, additional engineering work is typically needed to develop and tune application specific metaheuristics algorithms.
Another idea is the use of reinforcement learning to train a neural network to learn policies that are directly optimized for each problem type. For example, a solution is to use multi-agent RL, and have each robot independently learn a navigation policy that causes optimized cooperative behavior to emerge.
Steve Paul et al. , “Learning to Solve Multi-Robot Task Allocation with a Covariant- Attention based Neural Architecture”, ICLR 2021, discloses a graph neural network architecture over which reinforcement learning can be performed to obtain online policies for multi-robot task allocation problems.
US 10474 141 B2 discloses a method and an apparatus for controlling movement of transporting devices. Transport containers are stored in stacks arranged in a facility having pathways arranged in a grid-like structure above the stacks. The transporting devices are configured to operate on the grid-like structure. US 10 089 586 B2 discloses a job management system for processing job requests in an automated physical environment, such as a factory, hospital, order processing facility or office building, wherein the job requests are handled by a fleet of autonomously - navigating mobile robots.
SUMMARY OF THE INVENTION
It is the object of the invention to improve robot fleet management, especially in highly detailed environment maps.
The invention provides a computer implemented robot management method for operating a robot management system that controls at least one robot in an environment, the method comprising: a) providing an input map of the environment, the input map having a plurality of nodes and at least one connecting edge that connects two nodes, each node indicating a location that the robot is able to visit within the environment, each connecting edge indicating a route that the robot is able to take between connected nodes through the environment; b) collating the current state of the robot management system, the current state of the robot management system including at least one node state of each node, at least one edge state of each edge, and at least one robot state of each robot; c) compiling a list of route constraints, wherein the route constraints relate to information indicative for properties of a route that the robot is able to take to a node; d) initializing a graph neural network system, wherein the graph neural network system is configured for determining one single-step command for each robot and the graph neural network system comprises a main graph autoencoder network and at least one auxiliary graph autoencoder network for each route constraint of the list of route constraints, the main graph autoencoder network being initialized based on the current state of the robot management system obtained in step b), wherein each separate auxiliary graph autoencoder is initialized based on the respective route constraint; e) executing the graph neural network system, so as to determine a single-step command for each robot, the single-step command being configured to control a next action of the corresponding robot; and f) transmitting each single-step command obtained in step e) to the corresponding robot, in order to cause the robot to perform its next action according to the single- step command.
Preferably, step c), step d) or step e) are performed multiple times. Preferably, steps c) and d), steps c) and e), or steps d) and e) are performed multiple times. Preferably, steps c), d), and e) are performed multiple times. Preferably, in case of multiple performance of the steps, an intermediate state is propagated from one iteration to the next iteration. This intermediate state is handled similarly within the message passing graph neural network.
Preferably, before step e), a hard attention mask is generated and step e) is performed only on a subgraph of the graph neural network system, wherein the subgraph is determined by the hard attention mask.
Preferably, the hard attention mask is generated by randomly selecting one of the route constraints; determining a subgraph of the graph neural network system to which the route constraint relates; updating the internal neural state of the graph neural network system and/or updating the single-step command at each node for the robot.
Preferably, the subgraph is determined such that the deviation of the subgraph from the shortest route is below a predetermined threshold.
Preferably, it is randomly selected, whether in step e), the whole graph neural network system or a portion of the graph neural network system is executed.
Preferably, in step e), each single-step command is determined after the graph neural network system was iteratively executed for a predetermined number of iterations. Preferably, the graph neural network system is configured as a message passing graph neural network system. Preferably, the main graph autoencoder network includes a first fully connected neural network for each node
Preferably, each auxiliary graph autoencoder network includes a second fully connected neural network that is associated with a particular route constraint.
Preferably, in step e) the node state is passed to the first and second fully connected networks and the route constraint is only passed to the associated second fully connected neural network.
Preferably, the main graph autoencoder network includes a first aggregator for each node, and the output of any, some, or all of the first and second fully connected neural networks is passed to the first aggregator.
Preferably, the graph neural network system comprises a second aggregator for each node that is associated with that node, wherein the second aggregator of any node receives the output of any, some, or all of the first aggregators of the unassociated other nodes.
Preferably, the main graph autoencoder network includes a third fully connected neural network for each node.
Preferably, each auxiliary graph autoencoder includes a fourth fully connected neural network associated with a particular route constraint.
Preferably, for each node the third fully connected network receives the output of the first aggregator and/or the second aggregator that is associated with that node. Preferably, for each node each fourth fully connected neural network receives the output of the first aggregator and/or the second aggregator that is associated with that node.
Preferably, the main graph autoencoder network includes a third aggregator for each node, wherein for each node the output of any, some, or all of the third and fourth fully connected neural networks is passed to the third aggregator of that node, and this output is passed on as output of the graph neural network system.
Preferably, in addition the output of any, some, or all of the fourth fully connected networks of that node may be passed on as additional output.
Preferably the node state includes a pick-up indicator, that indicates that a package is to be picked up from that node. Preferably, the node state includes a drop-off indicator, that indicates that a package is to be dropped off at that node. Preferably, the node state includes a battery charger indicator, that indicates that a battery charger is available at that node. Preferably, the note state includes a capacity indicator, that indicates whether or how much robots or packages that node can still accommodate.
Preferably, the edge state includes an edge capacity indicator, that indicates whether or how much robots are able to travel along that edge, preferably without delay or congestion. Preferably, the edge state includes a distance indicator, that is indicative of the physical distance between the two nodes connected by that edge.
Preferably, the robot state includes a carrying capacity indicator, that is indicative whether or how much freight that robot is able to carry. Preferably, the robot state includes a battery status indicator, that is indicative of the charge status of the robot batter, in particular whether the robot needs to go to a charging station, i.e., a node with a battery charger.
The invention further provides a robot management system for controlling a plurality of robots in an environment, the environment comprising a plurality of locations that are connected by at least one route, the robot management system comprising at least one robot that is configured to received commands generated by a computer system, and a computer system that is configured to perform one, some, or all of the steps of the method previously described.
The invention further provides a computer program, a computer readable storage medium or a data carrier signal comprising instructions which, upon execution by a computer system, cause the computer system to carry out one, some or all of the steps of the previously described method.
The invention further provides a computer implemented graph neural network system configured for controlling robots in an environment, the environment comprising a plurality of locations represented by nodes that are connected by at least one route that is represented by at least one connecting edge, the environment further comprising route constraints relating to information indicative for properties of a route that the robot is able to take to a node, the graph neural network system comprising a main graph autoencoder network for each node characterized by an auxiliary graph autoencoder network for each route constraint.
Preferably, the graph neural network system is configured as a message passing graph neural network system. Preferably, the main graph autoencoder network includes a first fully connected neural network for each node
Preferably, each auxiliary graph autoencoder network includes a second fully connected neural network that is associated with a particular route constraint.
Preferably, the graph neural network system is configured such that the node state is passable to the first and second fully connected networks and the route constraint is only passable to the associated second fully connected neural network.
Preferably, the main graph autoencoder network includes a first aggregator for each node, and the output of any, some, or all of the first and second fully connected neural networks is passable to the first aggregator.
Preferably, the graph neural network system comprises a second aggregator for each node that is associated with that node, wherein the second aggregator of any node is configured to receive the output of any, some, or all of the first aggregators of the unassociated other nodes.
Preferably, the main graph autoencoder network includes a third fully connected neural network for each node. Preferably, each auxiliary graph autoencoder includes a fourth fully connected neural network associated with a particular route constraint.
Preferably, the third fully connected network is configured to, for each node, receive the output of the first aggregator and/or the second aggregator that is associated with that node. Preferably, each fourth fully connected neural network is configured to, for each node, receive the output of the first aggregator and/or the second aggregator that is associated with that node.
Preferably, the main graph autoencoder network includes a third aggregator for each node, wherein for each node the output of any, some, or all of the third and fourth fully connected neural networks is passable to the third aggregator of that node, and this output is able to be passed on as output of the graph neural network system.
Preferably, in addition the output of any, some, or all of the fourth fully connected networks of that node are passable on as additional output.
Preferably the node state includes a pick-up indicator, that indicates that a package is to be picked up from that node. Preferably, the node state includes a drop-off indicator, that indicates that a package is to be dropped off at that node. Preferably, the node state includes a battery charger indicator, that indicates that a battery charger is available at that node. Preferably, the note state includes a capacity indicator, that indicates whether or how much robots or packages that node can still accommodate.
Preferably, the edge state includes an edge capacity indicator, that indicates whether or how much robots are able to travel along that edge, preferably without delay or congestion. Preferably, the edge state includes a distance indicator, that is indicative of the physical distance between the two nodes connected by that edge.
Preferably, the robot state includes a carrying capacity indicator, that is indicative whether or how much freight that robot is able to carry. Preferably, the robot state includes a battery status indicator, that is indicative of the charge status of the robot batter, in particular whether the robot needs to go to a charging station, i.e., a node with a battery charger.
The invention aims to coordinate robots on highly detailed world maps (described as graphs with many nodes and edges) or environments. For such applications, there is only little work applying reinforcement learning with neural networks.
Traditional search algorithms either have long computation times (e.g., for general search algorithms), or require much engineering effort (e.g., for hand-tuned heuristics). Multi-agent RL approaches require a separate neural network instance for each agent, and thus have a limited capacity for each individual network to accept input information. For example, for large graphs, it would be computationally infeasible for every agent to analyze the entire graph individually. The solution to this is only to provide coarse, high level information to the neural network inputs, which makes certain types of applications difficult.
The instant invention proposes a new graph neural network architecture for solving arbitrary task allocation, constraint satisfaction and cooperative routing problem (s). By designing a novel network structure, we can accept an arbitrary and variable number of route constraints as input. This allows us to use a single neural network comprised of subnetworks to make decisions for the entire fleet, which has better computational scalability.
The solution described has in general the following steps.
Initially the current state of the system is collated. This includes the state of each graph node, which may include information about the state of robots currently positioned at each graph nodes. Some input data may be computed for each node, such as, whether there are pickups or drop-offs at the node, whether there is a battery charger at the node, what the robot carrying capacity is (if there is a robot on the node), or what the battery status is (if there is a robot on the node).
Furthermore, a list of route constraints is compiled. Route constraints are typically relating to a particular route. For instance, a route constraint can define where the robot goes next, in order to arrive at its goal node. As an example, two robots that have picked up items, each require route constraints to ensure that the items are routed to the correct locations.
Not all constraints, however, are route constraints. Those are referred to as non route constraints. For example, if a robot requires use of a charger, it is not relevant which charger is used; in general, any charger can be used.
Route constraints that are calculated as input date for each node include, for example, the distance to the robot, the distance to the goal node, the scoring of the route (if multiple routes are available). Scoring can be done based on any property of the route. Mostly, this will be the time the robot takes for completing the route, including possible detours due to congestion, for example.
The system may have additional constraints, such as the robots have to be in a charged state.
After all the system information about the nodes, edges, and robots is compiled, the state information is presented to the input of a graph autoencoder network. For each route constraint, a separate auxiliary graph autoencoder to initialized.
Now, the initialized graph autoencoder network is executed. Basically, any graph neural network structure can be used, although the description will focus on an example of a message passing graph neural network.
The graph autoencoder network was trained for the particular deployment environment or world map using reinforcement learning. Since, in general the underlying system model is known, the network may be trained in a simulator using a range of techniques such as Q-learning, stochastic value gradients, or AlphaGo style policy improvement.
During execution the graph autoencoder network generates/determines a single-step command for each node and robot. The single-step commands are then sent to the corresponding robots. The steps so far can be referred to “node level attention”, since each node, robot and route attribute is considered and as a result, each node and robot are given a result from the graph autoencoder network.
In addition to these steps, one may make use of an additional network mechanism which is referred to as “route level attention”.
The steps described above have run the network once so far, however, here the network may run multiple times, each time potentially only computing a portion or subgraph of the neural network, e.g., by using a hard attention mask.
In order to generate the attention mask, the invention proposes to randomly select one of the route constraints. Next, the subgraph “near” the shortest route is isolated. Thus, a small deviation between the isolated subgraph and the shortest route is allowed.
Then, the graph neural network is executed on the isolated subgraph and the output of the graph neural network is updated, which can include values for the internal neural state and/or interim actions for the robot at each node.
For each iteration, it can be randomly decided whether to execute the neural network over the whole graph or only over a portion of the graph, i.e., a subgraph, as defined above.
Once some number of predetermined iterations are complete, the actions for the robot at each node can be finalized.
BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments of the invention are described in more detail with reference to the accompanying schematic drawings.
Fig. 1 depicts an environment in which robots are navigating; Fig. 2 depicts an autoencoder;
Fig. 3 depicts an updated environment; and
Fig. 4 depicts a block diagram of a graph neural network.
DETAILED DESCRIPTION OF EMBODIMENT
Fig. 1 schematically depicts a robot management system 10. The robot management system 10 is configured to control a plurality of robots 12 in an environment 14. The environment 14 comprises a plurality of locations 16. The robots 12 are capable of travelling between two locations 16 via a path 18.
The environment 14 is abstracted into a machine-readable format. The environment 14 is represented as a route graph 20 that includes a plurality of nodes 22 and at least one connecting edge 24. Each node 22 represents one of the physical locations 16 in the physical environment 14. Each connecting edge 24 connects two nodes 22 and represents a physical path 18, that one of the robots 12 can take between two physical locations 16.
It should be noted that while the connecting edge 24 is represented as a straight line, the corresponding physical path 18 need not be straight and can further include curves or ramps or other non-straight elements.
Each robot 12 has associated with it a robot state. The robot state contains information indicative of the respective operational state of the robot 12. The robot state can include information about the current battery charge, remaining battery charge, estimated charge consumption, the carrying capacity, the current robot position (in physical space and/or in graph space), and any other property that describes the operational situation of the robot 12.
Furthermore, each node 22 has associated with it a node state, which can include information about the location 16 that the corresponding node 22 represents. The information may include, whether a package 26 is to be picked up from the node 22, whether or which of the packages 26 is to be dropped off at the node 22, whether the node 22 has a charging capability, i.e. the location 16 has a charging dock 28, and whether the node 22 is capable of accepting a robot 12 (i.e. whether the location 16 has enough space to accommodate a robot 12).
Each of the connecting edges 24 has associated with it an edge state, which contains information relating to properties of the specific connecting edge 24, i.e. the path 18 which is represented by the connecting edge 24. This information may include the capacity of the connecting edge 24 (for example, how many robots 12 can occupy the connecting edge 24 at the same time), the distance between the two nodes 22 that are connected by the specific connecting edge 24, or other properties of the connecting edge 24 as well as the associated path 18.
Fig. 1 depicts an example for a current state of the robot management system 10. Therein, a first robot 30 should go to the charging dock 28. A second robot 32 and a third robot 34 have each picked up a package 26 and should deliver it.
The second robot 32 has picked up the package 26 from a first pickup 36 and the third robot 34 has picked a package 26 from a second pick up 38.
In this example, the first robot 30 and the third robot 34 are still at their respective starting nodes 40. The second robot 32 has already moved from its starting node 40 to an intermediate node 42. None of the robots 30, 32, 34 are at their respective goal nodes 44.
The first robot 30 has as its goal node 44 the charging dock 28. The second robot 32 has as its goal node 44 a first drop-off 46. The third robot 34 has as its goal node 44 a second drop-off 48.
The previously described configuration of the robot management system 10 is known as the system state and includes the various robot, node, and edge states.
Subsequently, an embodiment of a robot management method is described. The robot management method operates the robot management system 10 such that each robot 12 receives a single-step command 50, which instructs the respective robot 12 what action it should perform next.
As an initial step, the system state (as exemplified in Fig. 1) is collated. As a next step, a list of route constraints is compiled. The route constraints relate to information that is indicative for properties of a route that each robot 12 is able to take towards its goal node 44. The route may be composed of one or more paths 18. The route constraints may include the total distance to travel to the goal node, the (estimated) time to arrival, the route capacity, or any other information that is relevant to the journey of the respective robot 12 along the route.
Referring now to Fig. 2, a graph neural network system 52 is initialized based on the current state of the robot management system 10 and the list of route constraints. As depicted in Fig. 2, the graph neural network system 52 comprises a main graph auto encoder network 54. The graph neural network system 52 further comprises a separate auxiliary graph auto encoder network 56 for each route constraint within the list of route constraints.
A plurality of first routes 58 that are associated with the third robot 34 initialize a first of the auxiliary graph auto encoder networks 56. The plurality of first routes 58 preferably include all possible routes for the third robot 34 from its current node 22 to its goal node 44, wherein the first routes 58 are selected such that they deviate from the shortest possible route by not more than a predetermined deviation. In this case, the third robot 34 can either go directly from the second pick up 38 to the second drop-off 48 or can go on a detour 60 via another node 22.
A second of the auxiliary graph auto encoder networks 56 is initialized based on a plurality of second routes 62, which are routes that the second robot 32 is able to take from its current node 22 to its goal node 44. The second routes 62 are selected in the same manner as the first routes 58 so that they do not deviate by more than a predetermined amount from the shortest route. In the same manner, another auxiliary graph auto encoder network 56 may be initialized for the first robot 30 (not shown here).
The graph neural network system 52 has been trained by methods known per se in the art. The training on the environment 14 is preferably done by simulator and can use a variety of techniques, such as cue-learning, stochastic value gradients, or policy improvement.
After the graph neural network system 52 is initialized, it is executed and results in a set of single-step commands 50 that are associated with each node 22 and corresponding robot 12.
As depicted in Fig. 3, the first robot 30 receives an instruction to move north to the next node 22, i.e. to the node 22 that has the charging dock 28.
The second robot 32 has received an instruction to also go north to the next node 22 in order to avoid collision with the first robot 30 while still moving closer to the first drop-off 46. The third robot 34 is instructed to move north towards its goal node 44 which is the second drop-off 48.
For the purposes of illustration it is assumed that each node 22 and each connecting edge 24 can only accommodate a single robot 12 at a time. Flere, the graph neural network system 52 has managed the movement of the robots 12 such that each robot 12 is closer to its goal node 44, while simultaneously avoiding that any connecting edge 24 and any node 22 is occupied by two robots 12 at the same time.
Referring to Fig. 4, an example of the configuration of the graph neural network system 52 is described in more detail.
The graph neural network system 52 is configured as message passing graph neural network system. Each node state 54 and each route constraint information 66 is input as initial information (far left in Fig. 4). The main graph auto encoder network 54 comprises a first fully connected neural network 68 or first FCN 68 (hereinafter, “fully connected neural network” is referred to as “FCN”). Furthermore, each auxiliary graph auto encoder network 56 comprises a second FCN 70 for each route constraint information 66.
The node state 64 is passed to both the first and second FCNs 68, 70. The route constraint information 66 is only passed to the second FCN 70 of its corresponding auxiliary graph auto encoder network 56.
The main graph auto encoder network 54 further comprises a first aggregator 72 associated with each node state 64. The output of the first and second FCNs 68, 70 is passed on to the first aggregator 72. Furthermore, the output of the second FCN 70 is additionally passed on without aggregation.
The graph neural network system 52 further comprises a second aggregator 74 for each node state 64. The second aggregator 74 of a specific node 22 is connected to the first aggregator 72 of the other nodes 22.
The main graph auto encoder network 54 comprises a third FCN 76. The auxiliary graph auto encoder network 56 comprises a fourth FCN 78 for each route constraint information 66.
The third FCN 76 receives the output from the first and second aggregators 72, 74. The fourth FCN 78 receives the output of the second aggregator 74 and the output of their corresponding second FCN 70. The fourth FCN 78 of one route constraint information 66 does not receive the output from the second FCN 70 of any other route constraint information 66.
Finally, the main graph auto encoder network 54 comprises a third aggregator 80 that receives the output from the third and fourth FCNs 76, 78.
The graph neural network system 52 has a first output 82, which corresponds to the output of the third aggregator 80. Furthermore, the graph neural network system 52 has a second output 84 for each route constraint information 66. The first output 82 corresponds to the output of the third aggregator 80 and the second output 84 corresponds to the output of the fourth FCN 78 of the corresponding route constraint information 66.
The graph neural network system 52 with this architecture is able to generate the single-step commands 50 for each robot 12 based on the system state and an arbitrary number of route constraints.
In order to improve scalability and manage an arbitrary number of route constraints, the invention provides a robot management system 10 that utilizes a graph neural network system 52. The graph neural network system 52 has a main graph auto encoder network 54, which deals with the system state as a whole and further comprises an auxiliary graph auto encoder network 56 for each route constraint of the robot management system 10, in order to generate commands 50 for the robots 12. The robots 12 move about an environment 14 based on the single-step commands 50.
REFERENCE SIGNS 10 robot management system 12 robot
14 environment
16 location
18 path
20 route graph
22 node
24 connecting edge
26 package
28 charging dock
30 first robot
32 second robot
34 third robot
36 first pickup
38 second pickup
40 starting node
42 intermediate node
44 goal node
46 first drop-off
48 second drop-off
50 single-step command
52 graph neural network system
54 main graph auto encoder network
56 auxiliary graph auto encoder network
58 first route
60 detour
62 second route
64 node state
66 constraint info
68 first FCN
70 second FCN
72 first aggregator
74 second aggregator third FCN fourth FCN third aggregator first output second output

Claims

1. A computer implemented robot management method for operating a robot management system (10) that controls at least one robot (12) in an environment (14), the method comprising: a) providing an input map of the environment (14), the input map having a plurality of nodes (22) and at least one connecting edge (24) that connects two nodes (22), each node (22) indicating a location (16) that the robot (12) is able to visit within the environment (14), each connecting edge (24) indicating a route that the robot (12) is able to take between connected nodes (22) through the environment (14); b) collating a current state of the robot management system (10), the current state of the robot management system (10) including at least one node state (64) of each node (22), at least one edge state of each edge (24), and at least one robot state of each robot (12); c) compiling a list of route constraints, wherein the route constraints relate to information indicative for properties of a route that the robot (12) is able to take to a node (22); d) initializing a graph neural network system (52), wherein the graph neural network system (52) is configured for determining one single-step command (50) for each robot (12) and the graph neural network system (52) comprises a main graph autoencoder network (54) and at least one auxiliary graph autoencoder network (56) for each route constraint of the list of route constraints, the main graph autoencoder network (54) being initialized based on the current state of the robot management system (10) obtained in step b), wherein each separate auxiliary graph autoencoder (56) is initialized based on the respective route constraint; e) executing the graph neural network system (52), so as to determine a single-step command (50) for each robot (12), the single-step command (50) being configured to control a next action of the corresponding robot (12); and f) transmitting each single-step command (50) obtained in step e) to the corresponding robot, in order to cause the robot (12) to perform the next action according to the single-step command (50).
2. The method according to claim 1 , characteri zed i n that any combination of the steps c), d) and e) is performed multiple times.
3. The method according to any of the preceding claims, characterized in that, before step e), a hard attention mask is generated, and step e) is performed only on a subgraph of the graph neural network system (52), wherein the subgraph is determined by the hard attention mask.
4. The method according to claim 3, characte rized i n that the hard attention mask is generated by randomly selecting one of the route constraints; determining a subgraph of the graph neural network system (52) to which the route constraint relates; updating the internal neural state of the graph neural network system (52) and/or updating the single-step command (50) at each node (22) for the robot (12).
5. The method according to claim 4, characte rized i n that the subgraph is determined such that the deviation of the subgraph from the shortest route is below a predetermined threshold.
6. The method according to any of the preceding claims, characterized in that it is randomly selected, whether in step e), the whole graph neural network system (52) or a portion of the graph neural network system (52) is executed.
7. The method according to any of the preceding claims, characterized i n that, in step e), each single-step command (50) is determined after the graph neural network system (52) was iteratively executed for a predetermined number of iterations.
8. The method according to any of the preceding claims, characte rized i n that the graph neural network system (52) is configured as a message passing graph neural network system (52), wherein the main graph autoencoder network (54) includes a first fully connected neural network for each node (22), wherein each auxiliary graph autoencoder network (56) includes a second fully connected neural network that is associated with a particular route constraint, wherein in step e) the node (22) state is passed to the first and second fully connected networks and the route constraint is only passed to the associated second fully connected neural network.
9. The method according to claim 8, ch aracte rized i n th at the main graph autoencoder network includes a first aggregator (72) for each node (22), and the output of any, some, or all of the first and second fully connected neural networks is passed to the first aggregator (72).
10. The method according to claim 9, ch aracte rized i n that the graph neural network system (52) comprises a second aggregator (74) for each node (22) that is associated with that node (22), wherein the second aggregator (74) of any node (22) receives the output of any, some, or all of the first aggregators (72) of the unassociated other nodes (22).
11. The method according to claim 10, characterized i n th at the main graph autoencoder network (54) includes a third fully connected neural network for each node (22), wherein each auxiliary graph autoencoder (56) includes a fourth fully connected neural network associated with a particular route constraint, wherein for each node (22) the third fully connected network receives the output of the first aggregator (72) and/or the second aggregator (74) that is associated with that node (22), wherein for each node (22) each fourth fully connected neural network receives the output of the first aggregator (72) and/or the second aggregator (74) that is associated with that node (22).
12. The method according to claim 11 , characte rized i n that the main graph autoencoder network (54) includes a third aggregator (80) for each node (22), wherein for each node (22) the output of any, some, or all of the third and fourth fully connected neural networks is passed to the third aggregator (80) of that node (22), and this output is passed on as output of the graph neural network system (52).
13. A robot management system (10) for controlling a plurality of robots in an environment (14), the environment (14) comprising a plurality of locations (16) that are connected by at least one route, the robot management system (10) comprising at least one robot (12) that is configured to received commands generated by a computer system, characterized by a computer system that is configured to perform all of the steps of the method according to any of the claims 1 to 12.
14. A computer program, a computer readable storage medium or a data carrier signal comprising instructions which, upon execution by a computer system, cause the computer system to carry out all of the steps of the method according to any of the claims 1 to 12.
15. A computer implemented graph neural network system (52) configured for controlling robots in an environment (14) , the environment (14) comprising a plurality of locations (16) represented by nodes (22) that are connected by at least one route that is represented by at least one connecting edge (24), the environment (14) further comprising route constraints relating to information indicative for properties of a route that the robot (12) is able to take to a node (22), the graph neural network system (52) comprising a main graph autoencoder network (54) for each node characterized by an auxiliary graph autoencoder network (56) for each route constraint.
PCT/EP2022/059955 2021-05-20 2022-04-13 Robot fleet management method and system using a graph neural network WO2022242966A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE112022002704.5T DE112022002704T5 (en) 2021-05-20 2022-04-13 Method and system for managing a robot fleet using a graph neural network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB2107204.6A GB2606752A (en) 2021-05-20 2021-05-20 Robot fleet management method and system using a graph neural network
GB2107204.6 2021-05-20

Publications (1)

Publication Number Publication Date
WO2022242966A1 true WO2022242966A1 (en) 2022-11-24

Family

ID=76637714

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/059955 WO2022242966A1 (en) 2021-05-20 2022-04-13 Robot fleet management method and system using a graph neural network

Country Status (3)

Country Link
DE (1) DE112022002704T5 (en)
GB (1) GB2606752A (en)
WO (1) WO2022242966A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10089586B2 (en) 2012-02-08 2018-10-02 Omron Adept Technologies, Inc. Job management system for a fleet of autonomous mobile robots
US10474141B2 (en) 2014-06-03 2019-11-12 Ocado Innovation Limited Methods, systems and apparatus for controlling movement of transporting devices
US20210004966A1 (en) * 2019-07-03 2021-01-07 Robert Bosch Gmbh Method for the Assessment of Possible Trajectories

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4661838B2 (en) * 2007-07-18 2011-03-30 トヨタ自動車株式会社 Route planning apparatus and method, cost evaluation apparatus, and moving body
US20190120640A1 (en) * 2017-10-19 2019-04-25 rideOS Autonomous vehicle routing
US10809732B2 (en) * 2018-09-25 2020-10-20 Mitsubishi Electric Research Laboratories, Inc. Deterministic path planning for controlling vehicle movement
US11803184B2 (en) * 2018-12-18 2023-10-31 Motional Ad Llc Methods for generating maps using hyper-graph data structures
US10814871B2 (en) * 2018-12-26 2020-10-27 Gm Cruise Holdings Llc Computing system for assigning maneuver labels to autonomous vehicle sensor data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10089586B2 (en) 2012-02-08 2018-10-02 Omron Adept Technologies, Inc. Job management system for a fleet of autonomous mobile robots
US10474141B2 (en) 2014-06-03 2019-11-12 Ocado Innovation Limited Methods, systems and apparatus for controlling movement of transporting devices
US20210004966A1 (en) * 2019-07-03 2021-01-07 Robert Bosch Gmbh Method for the Assessment of Possible Trajectories

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
EKATERINA TOLSTAYA ET AL: "Multi-Robot Coverage and Exploration using Spatial Graph Neural Networks", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 24 March 2021 (2021-03-24), XP081898382 *
SAMBARAN BANDYOPADHYAY ET AL: "Robust Hierarchical Graph Classification with Subgraph Attention", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 19 July 2020 (2020-07-19), XP081724887 *
STEVE PAUL ET AL.: "Learning to Solve Multi-Robot Task Allocation with a Covariant-Attention based Neural Architecture", ICLR, 2021

Also Published As

Publication number Publication date
GB2606752A (en) 2022-11-23
DE112022002704T5 (en) 2024-04-18
GB202107204D0 (en) 2021-07-07

Similar Documents

Publication Publication Date Title
US10655975B2 (en) System and method for routing optimization
JP4138541B2 (en) Distributed path planning apparatus and method, and distributed path planning program
Jeon et al. Routing automated guided vehicles in container terminals through the Q-learning technique
CN107203190A (en) A kind of inertial navigation AGV dispatching methods and system based on pahtfinder hard
WO2020165739A1 (en) Safeguarding resources of physical entites in a shared environment
CN110146103B (en) Unmanned equipment path planning method considering target trend and energy supply
EP4141599B1 (en) Multi-robot route planning
Zhou et al. Anisotropic Q-learning and waiting estimation based real-time routing for automated guided vehicles at container terminals
Roselli et al. A compositional algorithm for the conflict-free electric vehicle routing problem
Huang et al. Drone stations-aided beyond-battery-lifetime flight planning for parcel delivery
Bøgh et al. Distributed fleet management in noisy environments via model-predictive control
WO2022242966A1 (en) Robot fleet management method and system using a graph neural network
US20230075128A1 (en) Column generation methods and systems for routing and scheduling in robotic and vehicular applications
CN114415690A (en) Multi-agent path planning method, navigation server and readable storage medium
CN114237259A (en) Multi-agent path planning method based on floating resources, navigation server and readable storage medium
KR20230082753A (en) Server, method and computer program for providing route information for logistics
Zhang et al. Multi-AGVs pathfinding based on improved jump point search in logistic center
Miklić et al. A modular control system for warehouse automation-algorithms and simulations in USARSim
JP2022537938A (en) Method and system for performing trajectory planning under known environment
Abolhoseini et al. Dynamic routing with ant system and memory-based decision-making process
Rahili et al. Distributed adaptive reinforcement learning: A method for optimal routing
Kloetzer et al. On a class of multi-robot task allocation problems
Verma et al. Traffic management of multi-AGV systems by improved dynamic resource reservation
Wang et al. [Retracted] Biobjective UAV/UGV Collaborative Rendezvous Planning in Persistent Intelligent Task‐Based Wireless Communication
Jiang et al. Adaptive dynamic programming for multi-driver order dispatching at large-scale

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22722278

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 112022002704

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22722278

Country of ref document: EP

Kind code of ref document: A1