WO2023081634A1

WO2023081634A1 - Method of creating self-assembling, transient analytic and learning agents in a message-driven architecture

Info

Publication number: WO2023081634A1
Application number: PCT/US2022/079029
Authority: WO
Inventors: William House; Anthony HOWCROFT; Yizhao Chen
Original assignee: Swarm Engineering, Inc.
Priority date: 2021-11-05
Filing date: 2022-11-01
Publication date: 2023-05-11

Abstract

A message-driven multi-agent system (20) which includes a message broker (24) for routing messages and notifications between agents. The individual agents (22) each have a generic agent controller module which loads and executes a persona module and an algorithm module. The individual agents (22) may load one of several different types of persona modules, including a free agent persona, a worker persona, a supervisor persona and a paramount persona. The system includes a code generator service (28) that supplies the individual agents with algorithms necessary for their assigned tasks. The individual agents can chain the algorithms, run them in parallel, or in some combination thereof. A database gateway (30) provides agents the ability to read/write data and the code service generator the ability to save and recall code templates which form the algorithms used by the agents. An outbound channel service (32) is used to communicate notifications and results to an operator or other user of the system.

Description

METHOD OF CREATING SELF- ASSEMBLING, TRANSIENT ANALYTIC AND LEARNING AGENTS IN A MESSAGE-DRIVEN ARCHITECTURE

Cross Reference to Related Applications

[0001] This application claims priority of U.S. patent application serial no. 63/276,222 filed on Nov. 5, 2021 entitled METHOD OF CREATING SELF-ASSEMBLING, TRANSIENT ANALYTIC AND LEARNING AGENTS IN A MESSAGE-DRIVEN ARCHITECTURE, the disclosure of which is hereby incorporated herein by reference.

BACKGROUND

[0002] The present invention relates to software agents and, more particularly, to multi-agent systems.

[0003] Multi-agent systems are used in a variety of different domains such as robotics, distributed control, collaborative decision systems and other domains having to address a complex mixture of different tasks. The agents in such systems may have the ability to develop solutions on their own using some form of artificial intelligence such as machine learning.

[0004] Known examples of multi-agent systems are described in US Pat. 7,260,635; US Pat. 8,543,280; US Pat. 10,049,162; US Pat. 10,200,884; US Pat. 10,235,642; US Pat. 10,797,737; US Pat. 10,926,410; US Pat. 10,977,551; US Pat. 11,171,974; and US Pub. 2020/0160168, the disclosures of each of which are incorporated herein by reference.

[0005] While known multi-agent systems are useftd for addressing problems which are difficult for individual agent and monolithic systems to solve, further improvements in multiagent systems remains desirable.

SUMMARY

[0006] The present invention provides a message-driven multi-agent system which addresses one or more problems in known multi-agent systems to thereby reduce the addressed problems. [0007] To accomplish this, the inventors of the present application identified a number of preconceived needs for the implementation of a multi-agent system that increased the resources and difficulty of developing and operating a multi-agent system thereby impeding the development of such systems that could be overcome or reduced by the use of a message-driven architecture as described herein. [0008] For example, there is a perceived need to deploy a unique, separate agent to represent each real-world asset or process being modeled. This can lead to the over-allocation of computer resources especially in large, complex problem domains.

[0009] There is also a perceived need to hard-code agent logic and algorithms so that each such hard-coded agent may only be utilized for specialized use cases.

[0010] There is a perceived need to devise complex message routing protocols so that only agents capable of performing the requested actions will receive the requests.

[0011] When using a modem, cloud-based micro-service architecture, there is also a perceived need to preconfigure a separate agent container for each permutation of logic and algorithms.

[0012] The system described herein uses an agent structure that decouples many of the dependencies that these perceived needs are based on and thereby provides a multi-agent architecture that is more readily developed and implemented than one based upon such perceived needs.

[0013] The system described herein provides such de-coupling by using a message-driven architecture with a plurality of flexible agents.

[0014] The invention comprises, in one embodiment thereof, a multi-agent system that includes a plurality of agents; a message broker which routes notifications and messages between agents; a code generator service which communicates algorithms to individual agents upon the request of the agent; and wherein each agent includes a persona module and an algorithm module, the persona module being configured to rim one of a plurality of distinct personas wherein each of the plurality of distinct personas is configured to define the relationship of the agent with other agents and wherein the algorithm module is configured to execute algorithms supplied by the code generator service.

[0015] Any one of the different embodiments of the multi-agent system may include a database gateway wherein each of the agents has the ability to read and write to a database through the database gateway and the code service generator has the ability to save and recall code templates, each code template defining an algorithm, through the database gateway.

[0016] Any one of the different embodiments of the multi-agent system may also include an outbound channel service module which forwards messages to entities outside of the multi-agent system. [0017] Any one of the different embodiments of the multi-agent system may be configured such that each of the plurality of agents has a controller module which, in response to a message from the message broker, (i) loads a selected one of the plurality of distinct personas and (ii) requests and loads a selected algorithm from the code generator service. In such embodiments, each of the plurality of agents may have an identical controller module.

[0018] Any one of the different embodiments of the multi-agent system may be configured such that the plurality of distinct personas includes: (i) a free agent persona wherein the agent is configured to perform a requested job without interacting with other agents; (ii) a worker persona wherein the agent is configured to execute a unit of work at the request of another agent; (iii) a supervisor persona wherein the agent is configured to create a team of worker agents and delegate work to them; and (iv) a paramount persona wherein the agent is configured to create a team of supervisor and/or other paramount agents and delegate work to them.

[0019] Any one of the different embodiments of the multi-agent system may be configured such that the algorithm module of each agent is configured to execute one or more algorithms wherein the algorithms which are chained, or parallel, or some combination of both chained and parallel.

BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above mentioned and other features of this invention, and the manner of attaining them, will become more apparent and the invention itself will be better understood by reference to the following description of an embodiment of the invention taken in conjunction with the accompanying drawings, wherein:

[0021] FIG. 1 is a schematic view showing a message-driven multi-agent system using a single agent to execute an analytic job.

[0022] FIG. 2 is a schematic view of the system of FIG. 1 where one agent supervises a pool of worker agents when executing a job.

[0023] FIG. 3 is a schematic view of the system of FIG. 1 where a paramount agent supervises a plurality of supervisory agents who in turn supervise worker agents when executing a job.

[0024] Corresponding reference characters indicate corresponding parts throughout the several views. Although the exemplification set out herein illustrates an embodiment of the invention, in one form, the embodiment disclosed below is not intended to be exhaustive or to be construed as limiting the scope of the invention to the precise form disclosed. DETAILED DESCRIPTION

[0025] A multi-agent system (MAS) 20 having a message-driven architecture is shown in FIGS. 1-3 taking on different configurations of agents to execute different types of jobs. In MAS 20, there are a plurality of separate software agents 22. The individual agents 22 are not each permanently assigned to model a separate unique real-world asset or process. They may, however, model a particular real-world asset or process while a need exists for an agent to do so but MAS 20 does not pre-assign an agent to each asset or process to be modeled by the system wherein that agent is then permanently assigned to that asset or process for the life of the system. [0026] Instead, the message-driven architecture of MAS 20 uses a message broker 24 and messages which contain a unique identifier which is used to read the complete problem specification, along with necessary parameters. The launching of such a message is indicated by the Launch Message arrow in FIG. 1. A system operator 26, e.g., a human, provides the problem specification and necessary parameters and initiates the job as indicated by the Run Job arrow in FIG. 1.

[0027] Messages may be addressed to specific agents 22, to some or all of the agents 22 for parallel or coordinated execution, or, to a single anonymous agent 22 on a “first read” basis. In FIG. 1, MAS 20 is executing an analytic job which requires only a single agent 22. In FIG. 1, the depicted agent 22 is described as a Flagent Free Agent which is a play on words because the plurality of agents may also be referred to as a “flock” of agents and Flagent is a euphemism for a single agent out of the flock. It is also acting as a “free agent” and acts alone to perform the assigned task.

[0028] The individual agents 22 may also take on other process roles, e.g., some agents may take on a worker or operational role working under another agent while others take on a supervisory role. This allows the agents 22 to form one or more ad hoc teams which collaborate to solve a problem. FIGS. 2 and 3 illustrate examples where MAS 20 employs multiple agents, some of which are acting in a supervisory role while others act in a worker role.

[0029] Individual agents 22 act according to behavior modules. A module defining a particular behavior may be stored in the memory of the individual agent 22, it may be stored on a local disk, or it can be assembled at runtime using a code generation service 28. When a message is received by an agent 22, the agent 22 determines if the necessary behavior or behaviors are already loaded into memory and ready to execute, if the behavior(s) must be read from a local disk into memory, or if the code generation service must be called to acquire the necessary software modules for the behavior(s). This system allows individual agents 22 to acquire new and/or updated capabilities using the code generation service 28 without any system downtime or a separate continuous delivery/continuous integration pipeline (CD/CI pipeline).

[0030] MAS 20 includes a number of system components in addition to the individual agents 22. These components include the message broker 24, the code generation service 28, an agent controller, a persona module, an algorithm module, a distributed in-memory blackboard, a database gateway 30 and an outbound channel service 32.

[0031] In the illustrated embodiment, message broker 24 is a fast, lightweight message broker that supports Advanced Message Queuing Protocol (AMQP).

[0032] Each agent 22 has a controller module that handles message queue subscriptions and publishing, retrieval of the problem definition, loading and/or acquiring of necessary modules and execution of the loaded persona module. The controller module is generic for all agents 22. Variation between individual agents 22 is achieved by loading persona and algorithm modules not by using different controller modules.

[0033] One of the modules that the controller module loads in an agent when executing a job is a persona module. The illustrated MAS 20 has four different personas that can be loaded into an agent 22. One of the personas is a “Free Agent” persona. The Free Agent persona is loaded when the agent 22 will work by itself to perform the received request. FIG. 1 illustrates an agent 22 having a Free Agent persona performing a task by itself.

[0034] FIG. 2 illustrates a situation where two other personas are being utilized to execute the job. These two personas are a Worker persona and a Supervisor persona. A Worker agent executes a unit of work for a Supervisor agent and returns the results to the Supervisor agent. A Supervisor agent creates an ad hoc team of Worker agents and delegates work to them. The Supervisor agent then receives the results of that delegated work and evaluates those results. In FIG. 2, the individual agents 22A that have loaded a Worker persona module are labelled Flagent Worker and the agent 22B that has loaded a Supervisor persona module is labelled Flagent Queen.

[0035] FIG. 3 illustrates a situation where two levels of supervisory agents are employed. In the example of FIG. 3, MAS 20 employs a Paramount agent 22C which creates an ad hoc team of Supervisors 22B wherein each of the plurality of Supervisor agents 22B has its own team of Worker agents 22A. The Paramount agent 22C receives and evaluates the results of each of the Supervisor agents 22B. It is noted that a Paramount agent 22C can also create an ad hoc teams of other Paramount agents 22C such that there can further levels of hierarchy in the agent structure. In FIG. 3, the Paramount agent 22C is labelled Flagent Empress.

[0036] The persona module is also responsible for marshaling input data and parameters and executing the algorithm module(s). The algorithm modules may be chained, used for parallel processing of partitioned input data or any combination of chained and parallel processing. The persona module of the agent receives the algorithm module results and formats the results for return to the caller or designated recipient of the results.

[0037] The use of the persona module to dynamically assemble and execute the code/algorithm modules allows a single agent to perform complex tasks. In some conventional multi-agent systems, each individual agent might be limited to the performance of only one or a small number of algorithm modules, in which case, such a system would require the use of a much larger number of agents for some tasks in comparison to MAS 20 which utilizes a persona module that allows for the dynamic assembly of multiple algorithm modules.

[0038] The algorithm module(s) of each individual agent wrap an algorithm with boilerplate code for integrating with the persona module. As a result, MAS 20 is algorithm-agnostic and can deploy algorithms from a variety of sources, written in a variety of languages, such as algorithms from Google, Microsoft, academic researchers and other commercial and non-commercial sources.

[0039] The code generation service 28 facilitates the dynamic configuration of the algorithm modules by the individual agents. The code generation service 28 operates by selecting and configuring a code template for a particular purpose and providing it to an agent in response to a request. A code template is a file with optional embedded template commands that are used for logic and data dependency injection operations. The code template is provided with the necessary logic and data operations and forwarded to the requesting agent which then incorporates them into its algorithm module.

[0040] The agents also use a distributed in-memory blackboard to save process state during execution. To enhance the performance of MAS 20, a high-performance, memory-first, distributed database is preferred for this purpose. [0041] MAS 20 also includes a database gateway 30 which integrates a relational database management system with MAS 20 to thereby provide easy access to client data. The agents may read and write data to a database through gateway 30. Data read from the database may form some or all of the input data for processes performed by the agents. The code generator service 28 also utilizes the database gateway 30 to access code templates stored in a database. The code generator service and agents may also write data to the database. The data written to the database may be intermediate data that is subsequently read in further performance of the processes being conducted by MAS 20, it may be output data usefid to the operator 26 or other entity, or it may be data usefill for subsequent jobs performed by MAS 20.

[0042] MAS 20 also includes an outbound channel service 32 which communicates data to the operator 26 and/or other entities distinct from MAS 20. The operator 26 and other entities receiving such communications may be humans or a non-human entity/operation, e.g., a robotic system, which is distinct from MAS 20. Outbound channel service 32 forms an interface between MAS 20 and one or more outbound communication channels, such as a telecommunication and/or internet connection, whereby results obtained by MAS 20 are communicated to the operator 26 or other entity. Such results can be communicated via email or SMS (Short Message Service) messages or other suitable means of communication.

[0043] As can be seen from the figures, message broker 24 communicates messages from the operator 26 to one or more of the agents 22. Message broker 24 provides notifications and messages to the operator 26 through the outbound channel service 32. Message broker 24 also acts as an intermediary for all messages and notifications between different agents. The agents do, however, have direct read/write capabilities with the database gateway 30 and direct communications with the code generator service 28. Although not shown, operator 26 may have access to some or all of the database which is tied to database gateway 30. It is also noted that the code generator service 28 communicates directly with the database gateway 30 and individual agents 22 rather than through message broker 24.

[0044] While this invention has been described as having an exemplary design, the present invention may be further modified within the spirit and scope of this disclosure. This application is therefore intended to cover any variations, uses, or adaptations of the invention using its general principles.

Claims

WHAT IS CLAIMED IS:

1. A multi-agent system (20) that comprises: a plurality of agents (22); a message broker (24) which routes notifications and messages between agents; a code generator service (28) which communicates algorithms to individual agents upon the request of the agent; and wherein each agent (22) includes a persona module and an algorithm module, the persona module being configured to run one of a plurality of distinct personas wherein each of the plurality of distinct personas is configured to define the relationship of the agent with other agents and wherein the algorithm module is configured to execute algorithms supplied by the code generator service.

2. The multi-agent system of claim 1 further comprising a database gateway (30) wherein each of the agents (22) has the ability to read and write to a database through the database gateway and the code service generator (28) has the ability to save and recall code templates, each code template defining an algorithm, through the database gateway.

3. The multi-agent system of claim 2 further comprising an outbound channel service module (32) which forwards messages to entities outside of the multi-agent system.

4. The multi-agent system of any one of claims 1 through 3 wherein each of the plurality of agents (22) has a controller module which, in response to a message from the message broker, (i) loads a selected one of the plurality of distinct personas and (ii) requests and loads a selected algorithm from the code generator service (28).

5. The multi-agent system of claim 4 wherein each of the plurality of agents (22) has an identical controller module.

6. The multi-agent system of claim 4 wherein the plurality of distinct personas includes: a free agent persona wherein the agent is configured to perform a requested job without -8- interacting with other agents; a worker persona wherein the agent is configured to execute a unit of work at the request of another agent; a supervisor persona wherein the agent is configured to create a team of worker agents and delegate work to them; and a paramount persona wherein the agent is configured to create a team of supervisor and/or other paramount agents and delegate work to them.

7. The multi-agent system of claim 6 wherein the algorithm module of each agent is configured to execute one or more algorithms wherein the algorithms are chained, or parallel, or both chained and parallel.

-9-