CN111738372A - Distributed multi-agent space-time feature extraction method and behavior decision method - Google Patents
Distributed multi-agent space-time feature extraction method and behavior decision method Download PDFInfo
- Publication number
- CN111738372A CN111738372A CN202010873794.8A CN202010873794A CN111738372A CN 111738372 A CN111738372 A CN 111738372A CN 202010873794 A CN202010873794 A CN 202010873794A CN 111738372 A CN111738372 A CN 111738372A
- Authority
- CN
- China
- Prior art keywords
- agent
- space
- vector
- time
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 93
- 238000000605 extraction Methods 0.000 title claims abstract description 84
- 239000003795 chemical substances by application Substances 0.000 claims abstract description 258
- 239000013598 vector Substances 0.000 claims abstract description 164
- 230000003542 behavioural effect Effects 0.000 claims abstract description 5
- 230000006399 behavior Effects 0.000 claims description 43
- 238000004364 calculation method Methods 0.000 claims description 34
- 238000013528 artificial neural network Methods 0.000 claims description 31
- 238000010586 diagram Methods 0.000 claims description 23
- 230000007787 long-term memory Effects 0.000 claims description 23
- 230000006403 short-term memory Effects 0.000 claims description 23
- 230000015654 memory Effects 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 15
- 238000004891 communication Methods 0.000 claims description 12
- 230000007613 environmental effect Effects 0.000 claims description 8
- 230000007246 mechanism Effects 0.000 claims description 7
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 230000002787 reinforcement Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 4
- 230000002441 reversible effect Effects 0.000 claims description 4
- 101100410811 Mus musculus Pxt1 gene Proteins 0.000 claims description 3
- 230000001133 acceleration Effects 0.000 claims description 3
- 230000003213 activating effect Effects 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 3
- 238000010606 normalization Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 16
- 238000003860 storage Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007123 defense Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 239000012754 barrier agent Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000011273 social behavior Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a distributed multi-agent space-time feature extraction method and a behavior decision method. The behavior decision method comprises the following steps: intelligent agent for acquiring current moment and previous momentsThe state information which can be sensed constructs a space-time state vector; inputting the space-time state vector into the network generation layer of the graph and outputting the intelligent agentThe original feature vector of (2); inputting the original feature vector into a spatial feature extraction layer, and outputting a spatial relationship feature vector; inputting the space relation characteristic vector into a space-time characteristic extraction layer, and outputting the space-time relation characteristic vector; calculating an agent based on the obtained spatiotemporal relationship feature vectorsA behavior decision at the current moment; updating time step, calculating next time agentSpatiotemporal features and behavioral decisions. The invention realizes the extraction of the distributed space-time characteristic relation of the multi-agent system under the constraints of complex environment, time-varying topology, limited resources and the like, and improves the self-adaptive capacity and performance robustness of the multi-agent system in large-scale complex tasks.
Description
Technical Field
The invention belongs to the field of multi-agent and group intelligence, and particularly relates to a distributed multi-agent space-time feature extraction method and a behavior decision method.
Background
The multi-agent system has the advantages of distributivity, simplicity, flexibility, robustness and the like, provides a brand-new solution for a plurality of complex problems with great challenges, and the group intelligence represented by the multi-agent system is also one of five intelligent forms of the key development established in 'new generation artificial intelligence development planning' in China. With the rapid development of micro-nano electronics, computing platforms, autonomous control and other emerging technologies, a multi-agent system composed of unmanned autonomous platforms such as unmanned aerial vehicles and unmanned vehicles is increasingly applied in important scenes concerning national civilization and national defense safety. The unmanned autonomous multi-agent system can quickly form area coverage in a networked, distributed and collaborative mode, achieves cluster resource optimization scheduling, improves task completion rate and response speed, can serve as a normalized deployment system to serve the fields of mountain patrol, disaster early warning, environment monitoring, regional logistics and the like on the one hand, and can also serve as a quick response system for emergencies to provide the capabilities of quick material scheduling, disaster monitoring evaluation, communication guarantee support and the like under the scenes such as epidemic prevention and control, sudden disasters, large-scale active personal defense and the like on the other hand.
The behavior decision of the multi-agent mainly comprises a centralized mode and a distributed mode. The centralized decision-making has a central decision-making point, all information is gathered to the central node through a communication network, behavior decision-making instructions of all the intelligent agents are calculated through a centralized planning decision-making algorithm, and then the decision-making instructions of all the intelligent agents are issued to each intelligent agent through the communication network for execution. The centralized mode has high requirement on the reliability of a communication network and a central node, and has larger behavior delay, and the intelligent agent is difficult to realize self-adaptive autonomous behavior decision along with the change of tasks and environments when facing to the actual application scene, thereby greatly limiting the exertion of the intelligent synergistic effect of the multi-intelligent-agent system. In an actual scene, a multi-agent system often covers a large range and is difficult to form a centralized network, a single agent often only has limited environment sensing capability, communication capability and behavior capability, and the communication topology connection relationship among the agents will change all the time in a dynamic task, so that distributed decision brings better self-adaption capability and task performance for the multi-agent system in complex environments and tasks.
When an agent makes a decision, the decision-making basis is the state information of the current task and environment, and when large-scale clustering and complex tasks/environments are faced, how to abstract and reduce the state information by effective means and further extract the space-time characteristic relationship between the agent and between the agent and the task environment elements is a key for ensuring the multi-agent system to realize abstract understanding of the task and the environment and further realize autonomous decision-making and intelligent control.
The graph attention network is a machine learning method which is just emerging in recent years, multiple problems in reality are abstracted into a graph structure, a graph neural network is adopted for feature extraction, an attention mechanism is further adopted for realizing fusion of different feature representation spaces, and related technologies are gradually verified and applied in scenes such as social networks, traffic road condition forecast and the like. On the other hand, the long-term and short-term memory network is an important recurrent neural network and is widely applied to the processing of timing problems, particularly in the fields of voice recognition, semantic analysis and the like. In particular, the long and short term memory network with peepholes has the characteristic of repeatedly refining the historical state, thereby having better performance when dealing with complex state and time-varying systems. The essential spatial topological structure of the multi-agent system and the time sequence dependency of tasks enable the graph attention network and the long-short term memory network with peepholes to have natural application advantages on the multi-agent, however, due to the rising of the related technology, the application of the graph attention network and the long-short term memory network on an unmanned autonomous multi-agent system composed of an unmanned aerial vehicle, an unmanned vehicle and the like is rarely reported at present, particularly, the two networks are combined for extracting the spatiotemporal relation characteristics of the multi-agent system, and the method has important prospective and innovative significance in the machine learning field and the multi-agent system field.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problem of efficient spatiotemporal feature relation extraction of a multi-agent system in dynamic and complex tasks, a first aspect of the present invention provides a distributed multi-agent spatiotemporal feature extraction method, which includes the following steps:
step S100, at the time ,Based onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the number is a preset historical state number;
step S200 based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,a selected feature space dimension;
step S300 based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a The spatial feature relation extraction layer is constructed in a mode that a graph attention network module and a full-connection network module are stacked alternately;
step S400, acquiring timeFront sideIndividual moment intelligent agentSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector of。
In some preferred embodiments, step S100 "agent at each moment in timeObservable spatial state vectors "including agent self states, task goal states, observable other agent states, observable environmental element states;
the self state of the intelligent agent comprises the self position, speed and acceleration state of the intelligent agent;
the task target state comprises a target position and a speed state;
the observable other agent states include observable location, speed states of other agents;
the observable environmental element states include observable position and velocity states of obstacles in the environment, and position states of no-traffic zones in the environment.
In some preferred embodiments, the graph network generation layer in step S200 is composed of a plurality of layers of fully-connected neural networks.
In some preferred embodiments, the method for extracting spatial feature relationship based on the spatial feature relationship extraction layer in step S300 includes:
step S310, using the agentOf the original feature vector0 And the original feature vectors of all neighboring agents0 Obtaining, as input, a spatial relationship feature vector by a first graph attention network module1 (ii) a Wherein,,as an agentA set of neighbor agents capable of direct communication; 0 , 0 ;
step S320, in1 For input, obtaining space relation characteristic vector through first full-connection network module2 ;
Step S330, based on the space relation feature vector obtained in the step S320, through the stacked attention network module and the full-connection network module, the method of the step S310 and the step S320 is adopted to iteratively calculate the secondSpatial relationship feature vector of order
f2-1 、 f2 (ii) a Wherein,,the number of stacked layers of the power network module and the fully connected network module is noted for the figure;
step S340, in the iterative computationThe second timeBased on k2(-1) By the method of step S310, throughObtaining space relation characteristic vector by drawing attention network module k2-1 (ii) a Splicing vector value0 ,2 ,4 ,…, k2(-1) , k2-1 ]Input the firstFully connecting network modules to obtainSpatial relationship feature vector k2 As time of dayLower intelligent agentFinal output based on the spatial feature relationship extraction layer。
In some preferred embodiments, the spatial relationship feature vector1 The acquisition method comprises the following steps:
step S311, using a learnable matrixPair relation feature vector0 、0 Linear transformation is carried out and spliced into a new relation characteristic vector0 =[ 0 , 0 ](ii) a Will be provided with0 Inputting a full-connection neural network and outputting an intelligent agentFor intelligent agentAttention coefficient of(ii) a Obtaining an agentFor intelligent agentAttention normalization coefficient of,
Step S312, adopting a multi-head attention mechanism for the first timeCalculating the normalized attention coefficient by the method of step S311(ii) a Intelligent agent under fusion of computing multi-head attention mechanismSpatial relationship feature vector of1 ,
Wherein,to keep track of the number of the first hours of attention,the function is activated for the sigmoid and,is as followsAttention is drawn to the chosen linear transformation matrix,representing a concatenation operation of the vectors.
In some preferred embodiments, the long-short term memory network with peephole based on graph volume operation comprisesThe long and short term memory network units are connected in series and provided with peepholes;
and an input gate, a forgetting gate and an output gate of the long and short term memory network unit are constructed based on a graph convolution neural network.
In some preferred embodiments, the spatiotemporal relationship feature vectorThe acquisition method comprises the following steps:
the long-short term memory network unit near the output end is recorded asAnd increase the serial number in the reverse direction;
first, theA long and short term memory network unitThe cell state of (A) is recorded asThe output is a space-time relation feature vectorInput is asTemporal spatial relationship feature vectorThe first stepp+1 unitOutput space-time relation feature vectorAnd cell state thereof(ii) a Wherein,;
based on cell state notationFeature vector of space-time relation、、Obtaining spatiotemporal relation feature vectors through the long-short term memory network。
In some preferred embodiments, the spatiotemporal relationship feature vectorThe calculation method comprises the following steps:
step 401, willTemporal spatial relationship feature vectorThe first stepA unitNetwork element status ofAnd the firstA unitOutput space-time relation feature vectorInputting the input data into a forgetting gate based on a neural network adopting graph convolution, and calculating to obtain an forgetting gate output variable,
Wherein, the graph convolution operation is represented,andthe weight coefficient matrix and the bias of the forgetting gate diagram convolutional neural network respectively,activating a function for sigmoid;
step 402, willTemporal spatial relationship feature vectorThe first stepA unitOutput space-time relation feature vectorAnd cell state thereofInputting to an input gate based on the use of a convolutional neural network, and comparing the cell statesUpdating, wherein the calculation formula is as follows:
wherein,is the output of the input gate or gates,in order to be a transitional state of the cell,、the weight coefficient matrix corresponding to the neural network is convolved for the input gate map,、the corresponding bias of the neural network is convolved for the input gate map,for the purpose of the tanh activation function,is the Hadamard product;
at step 403, willTemporal spatial relationship feature vectorThe first stepA unitSpace-time of outputRelational feature vectorAnd step 402 updatedA unitNetwork element status ofInput to an output gate based on a neural network using convolution of a graph to obtainA unitOutput space-time relation feature vectorThe calculation formula is as follows:
wherein,is the output variable of the output gate,、and (4) convolution of the weight coefficient matrix and the bias corresponding to the neural network for the output gate graph.
In some preferred embodiments, all of the agents in the distributed multi-agent share learnable parameters.
The invention provides a distributed multi-agent behavior decision method in a second aspect, and the method is based on the above method for extracting the space-time characteristics of the distributed multi-agent to obtain the agentAt the current momentSpace-time relation characteristic vector ofComputing agents using model-knowledge-driven methods or reinforcement learning data-driven methodsAt the current momentBehavioral decision set of(ii) a Wherein,,is the chosen decision space dimension.
In some preferred embodiments of the distributed multi-agent behavioral decision method, all agents in the distributed multi-agent share learnable parameters.
The invention provides a distributed multi-agent space-time feature extraction system, which comprises a state vector acquisition module, an original feature generation module, a space relation calculation module and a space-time relation calculation module;
the state vector acquisition module is configured to acquire the state vector at the momentBased onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the number is a preset historical state number;
the primitive feature generation module configured to generate a primitive feature based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,a selected feature space dimension;
the spatial relationship calculation module is configured to calculate the spatial relationship based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a The spatial feature relation extraction layer is constructed in a mode that a graph attention network module and a full-connection network module are stacked alternately;
the time-space relation calculation module is configured to obtain timeFront sideIndividual moment intelligent agentSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector of。
The fourth aspect of the invention provides a distributed multi-agent behavior decision system, which is based on the above-mentioned distributed multi-agent space-time feature extraction system and further comprises a behavior decision calculation module;
the behavioral decision calculation module is configured to be based on an agentAt the current momentSpace-time relation characteristic vector ofComputing agents using model-knowledge-driven methods or reinforcement learning data-driven methodsAt the current momentBehavioral decision set of(ii) a Wherein,,is the chosen decision space dimension.
In a fifth aspect of the present invention, a storage device is proposed, in which a plurality of programs are stored, said programs being adapted to be loaded and executed by a processor to implement the above-mentioned distributed multi-agent spatiotemporal feature extraction method, and/or the above-mentioned distributed multi-agent behaviour decision method.
In a sixth aspect of the present invention, a processing apparatus is provided, which includes a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; the program is suitable to be loaded and executed by a processor to realize the above-mentioned distributed multi-agent space-time feature extraction method and/or the above-mentioned distributed multi-agent behavior decision method.
The invention has the beneficial effects that:
by adopting a distributed characteristic extraction and behavior decision mode, compared with a centralized mode, the method is closer to the actual application scene of a large-scale multi-agent system, and can fully exert the application advantages of the multi-agent system such as distribution, networking and synergy; the spatial-temporal characteristic relation contained in the multi-agent system is extracted through the graph attention mechanism and the long and short memory networks, so that an important basis can be provided for the follow-up intelligent behavior decision of the multi-agent system, the agent can realize the autonomous behavior decision in dynamic and complex tasks, and the characteristic extraction layer is constructed by adopting parameter learnable models such as the graph neural network and the long and short memory networks with peepholes, so that the extraction of hidden characteristics and changed characteristics in the agent can be realized, and the task and environment adaptability of the agent is improved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a flow chart of a distributed multi-agent behavior decision method according to an embodiment of the present invention;
FIG. 2 is a diagram of a distributed multi-agent spatiotemporal feature extraction method for a single agent in an embodiment of the inventionGenerating a layer schematic diagram;
FIG. 3 is a diagram of a distributed multi-agent spatiotemporal feature extraction method for a single agent in an embodiment of the inventionSchematic diagram of the spatial feature extraction layer;
FIG. 4 is a diagram of a distributed multi-agent spatiotemporal feature extraction method for a single agent in an embodiment of the inventionThe space-time feature extraction layer schematic diagram of (1);
FIG. 5 is a diagram of a distributed multi-agent spatiotemporal feature extraction method for a single agent in an embodiment of the inventionThe 1 st long-short term memory network unit structure diagram in the space-time characteristic extraction layer;
FIG. 6 is a schematic diagram of intelligent agent collaboration behavior in an embodiment of the invention
FIG. 7 is a block diagram of a distributed multi-agent behavior decision system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
The invention provides a novel multi-agent space-time feature extraction method, which aims at the problems of high-efficiency space-time feature relation extraction and intelligent decision making of a multi-agent system in dynamic and complex tasks, adopts a graph structure to express the space-time relation between an agent individual and the environment, further adopts a graph neural network and an attention mechanism to extract the space feature relation, adopts a long-short term memory network with peepholes to realize the time feature relation extraction, and finally realizes the autonomous space-time feature relation extraction and the intelligent behavior decision making of the agent in the dynamic and complex tasks.
The invention discloses a distributed multi-agent space-time feature extraction method, which comprises the following steps:
step S100, at the time ,Based onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the number is a preset historical state number;
step S200 based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,a selected feature space dimension;
step S300 based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a The spatial feature relation extraction layer is constructed in a mode that a graph attention network module and a full-connection network module are stacked alternately;
step S400, acquiring timeFront sideIndividual moment intelligenceSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector of。
In order to more clearly explain the present invention, the following will expand the detailed description of the steps in an embodiment of the distributed multi-agent spatio-temporal feature extraction method according to the present invention with reference to the drawings.
In one embodiment of the present invention, the device comprisesIndividual agent's multi-agent system relies on method specification, the first of which() The method for extracting space-time characteristics of an agent comprises steps S100-S400, as shown in FIG. 1 (in FIG. 1, a distributed multi-agent behavior decision is shown)The flow diagram of the policy method includes the content of the distributed multi-agent spatio-temporal feature extraction method, so this embodiment only describes the relevant content, and other parts are described in the following corresponding embodiments, such as distributed sharing parameters, agent behavior decision based on spatio-temporal relation feature vectors, and updating time steps to return to the next time calculation in step S100).
Step S100, at the current timeBased onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the preset historical state number is an adjustable non-negative integer.
In this step, each discrete moment agent is obtainedObservable spatial states including agent's own state, task goal state, observable states of other agents, observable environmental element statesState; the state of the agent itself includes but is not limited to the state of the agent's own position, velocity, acceleration; task goal states include, but are not limited to, goal position, speed state; other agent states that may be observed include, but are not limited to, the location, speed state of other agents that may be observed; the observable environmental element states include, but are not limited to, observable position, velocity state of obstacles in the environment, position state of no-pass zones in the environment, and other environmental state information affecting multi-agent system tasks.
Step S200 based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,is the selected feature space dimension.
In this step, the graph network generation layer is realized by using a multilayer fully-connected neural network.
FIG. 2 shows a method for extracting spatiotemporal features of a distributed multi-agent space-time feature of a single agentA schematic layer diagram is generated. Spatial state vectorAfter being input into the network generation layer of the graph, the graph passes throughLayer is connected completelyThe neural network is connected for extracting the original features, and the network input is the digitalized state information, so that the features do not need to be extracted by adopting a convolutional network and the like, and the fitting capacity of the original feature vector to the complex intelligent agent relation is enhanced by the multilayer full-connection network.
Step S300 based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a The spatial feature relation extraction layer is constructed in a mode that a graph attention network module and a full-connection network module are stacked alternately.
In this embodiment, the attention network module uses a multi-head attention mechanism to implement feature aggregation of any agent with respect to surrounding agents and environmental information under different attentions.
FIG. 3 shows a distributed multi-agent spatiotemporal feature extraction method for a single agent according to the present inventionAs shown in the figure, the method for extracting a spatial feature relationship based on the spatial feature relationship extraction layer in this embodiment includes:
step S310, using the agentOf the original feature vector0 And all neighbors intelligenceOriginal feature vector of a volume0 Obtaining, as input, a spatial relationship feature vector by a first graph attention network module1 (ii) a Wherein,,as an agentA set of neighbor agents capable of direct communication; 0 , 0 ,0 transmission to agents via a communication network。
Intelligent agentSet of directly communicable neighbor agentsThe obtaining method may be: setting a determination parameter of observable distanceWhen another agent is presentAnd an agentIs less thanThen, it is determined。
In this embodiment, the spatial relationship feature vector1 The acquisition method comprises the following steps:
step S311, using a learnable matrixPair relation feature vector0 、0 Linear transformation is carried out and spliced into a new relation characteristic vector0 =[ 0 , 0 ](ii) a Will be provided with0 Inputting a full-connection neural network and outputting an intelligent agentFor intelligent agentAttention coefficient of(ii) a Obtaining an agentFor intelligent agentAttention normalization coefficient ofThe formula is
Step S312, adopting a multi-head attention mechanism for the first timeCalculating the normalized attention coefficient by the method of step S311(ii) a Intelligent calculation under fusion of multi-head attention mechanismBodySpatial relationship feature vector of1 The formula is
Wherein,to keep track of the number of the first hours of attention,the function is activated for the sigmoid and,is as followsAttention is drawn to the chosen linear transformation matrix,representing a concatenation operation of the vectors.
Spatial relationship feature vector y (Is odd and) The calculation of (2) can also be performed by adopting the methods of step (311) and step (312), wherein1 Become into y Will be0 、0 、0 Is changed into y-1 、
Step S320, in1 For input, obtaining space relation characteristic vector through first full-connection network module2 ,2 。
In this step, the fully-connected network module is composed of a plurality of fully-connected neural network layers, and performs enhanced representation and dimension compression on the features.
Step S330, based on the space relation characteristic vector obtained in the step S320, through the stacked graph attention network module and the full connection network module, adoptingStep S310, step S320 method iteration calculationSpatial relationship feature vector of order f2-1 、 f2 (ii) a Wherein,,the number of stacked layers of network modules and fully connected network modules is noted.
Step S340, in the iterative computationThen, based on k2(-1) By the method of step S310, throughObtaining space relation characteristic vector by drawing attention network module k2-1 (ii) a Splicing vector value0 ,2 ,4 ,…, k2(-1) , k2-1 ]Input the firstFully connecting network modules to obtainSpatial relationship feature vector k2 As time of dayLower intelligent agentFinal output based on the spatial feature relationship extraction layer。
In this embodiment, steps S330 and S340 are processes of performing iterative computations by using the methods of steps S310 and S320 based on the alternating stack graph attention network module and the fully-connected network module, and each stack of the graph attention network module and the fully-connected network module is taken as a unit, which is included in this embodimentA unit, therefore, needs to be performedAnd (3) iterative calculation:
The second iteration adopts the graph attention network module and the full-connection network module in the second unit to obtain the feature vector2 And spatial relationship feature vectors corresponding to all neighbor agents2 (,2 Transmission to agents via a communication network) Adopting the methods of step S310 and step S320 as input to respectively obtain the space relation feature vectors3 、4 ;
Repeating the steps in the same wayThe method operations of sub (including the first and second iterations) steps 310, 320, where the second stepSecond space closingIs a feature vector f2-1 、 f2 ,;
In the first placeAt the time of the second iteration, based on k2(-1) By the method of step S310, throughObtaining space relation characteristic vector by drawing attention network module k2-1 (ii) a Then the original feature vector is processed0 The space relation feature vector obtained by the method of the step 3022 ,4 ,…, k2(-1) And the feature vectors obtained after the last operation step 301 k2-1 Splicing is carried out to obtain the spliced characteristic vector0 ,2 ,4 ,…, k2(-1) , k2-1 ]Inputting the data into the last full-connection network module and outputting the space relation characteristic vector k2 I.e. the current timeLower intelligent agentFinal output of the spatial feature relationship extraction layer, denoted as k2 。
Step S400, acquiring timeFront side(Adjustable positive integer) time agentSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector ofWherein。
FIG. 4 is a diagram illustrating a distributed multi-agent spatiotemporal feature extraction method for a single agent according to the present inventionThe space-time feature extraction layer diagram of (1). In this embodiment, the long and short term memory network with peephole based on graph convolution operation comprisesThe long and short term memory network units are connected in series and provided with peepholes; the input gate, forgetting gate and output gate of the long and short term memory network unit are constructed based on the graph convolution neural network.
Space-time relation feature vectorThe acquisition method comprises the following steps: the long-short term memory network unit near the output end is recorded asAnd increase the serial number in the reverse direction; first, theA long and short term memory network unitThe cell state of (A) is recorded asThe output is a space-time relation feature vectorInput is asTemporal spatial relationship feature vectorThe first stepp+1 unitOutput space-time relation feature vectorAnd cell state thereof(ii) a Wherein,;
based on cell state notationFeature vector of space-time relation、、Obtaining spatiotemporal relation feature vectors through the long-short term memory network。
In this embodiment, the structure diagram of the 1 st long/short term memory network unit is shown in fig. 5, wherein the dotted line part represents the state of the previous long/short term memory network unit as the signal source. First, theSpace-time relation characteristic vector output by long and short term memory network unitThe calculation method comprises the following steps:
step 401, willTemporal spatial relationship feature vectorThe first stepA unitNetwork element status ofAnd the firstA unitOutput space-time relation feature vectorInputting the input data into a forgetting gate based on a neural network adopting graph convolution, and calculating to obtain an forgetting gate output variable,
Wherein, the graph convolution operation is represented,andthe weight coefficient matrix and the bias of the forgetting gate diagram convolutional neural network respectively,activating a function for sigmoid;
step 402, willTemporal spatial relationship feature vectorThe first stepA unitOutput space-time relation feature vectorAnd cell state thereofInputting to an input gate based on the use of a convolutional neural network, and comparing the cell statesUpdating, wherein the calculation formula is as follows:
wherein,is the output of the input gate or gates,in order to be a transitional state of the cell,、the weight coefficient matrix corresponding to the neural network is convolved for the input gate map,、the corresponding bias of the neural network is convolved for the input gate map,for the purpose of the tanh activation function,is the Hadamard product;
at step 403, willTemporal spatial relationship feature vectorThe first stepA unitOutput space-time relation feature vectorAnd step 402 updatedA unitNetwork element status ofInput to an output gate based on a neural network using convolution of a graph to obtainA unitOutput space-time relation feature vectorThe calculation formula is as follows:
wherein,is the output variable of the output gate,、and (4) convolution of the weight coefficient matrix and the bias corresponding to the neural network for the output gate graph.
A distributed multi-agent behavior decision method according to a second embodiment of the present invention, as shown in fig. 1, based on the above-mentioned distributed multi-agent spatiotemporal feature extraction method, further includes step S500: obtaining an agentAt the current momentSpace-time relation characteristic vector ofThe method based on model knowledge driving or the method based on reinforcement learning data driving (preferably adopting Actor-criticic architecture to train and learn the behavior of the intelligent agent) is adopted to calculate the intelligent agentAt the current momentBehavioral decision set of(ii) a Wherein,,is the chosen decision space dimension.
In the embodiments of the above-mentioned method for extracting spatiotemporal features of distributed multi-agent and the method for deciding behaviors of distributed multi-agent, the distributed shared parameters of all agents in distributed multi-agent may include a graph network generation layer in step S200, a spatial feature extraction layer in step S300, learnable parameters of relevant neural networks in the spatiotemporal feature extraction layer in step S400, and control and decision parameters for making a behavior decision of an agent based on the extracted spatiotemporal features.
FIG. 6 is a diagram illustrating intelligent collaboration behavior of agents in an embodiment of the invention. Including 8 enclosure agents (little black circle), 1 target agent (little triangle), 2 barrier agents (pentagon) in this embodiment, for clearer demonstration, the little black circle of agent is connected through the lines in the picture. The target of the enclosure intelligent agent is to form an enclosure for the target intelligent agent, in the process, collision between the target intelligent agent and the obstacle and collision between the target intelligent agent and other enclosure intelligent agents are avoided, and the target intelligent agent tries to get rid of the enclosure. In an embodiment, four phases are included: in a series of processes, the enclosure intelligent agent adopts the method provided by the invention to extract space-time characteristics, and on the basis, an Actor-Critic architecture is adopted to make a behavior decision, a target intelligent agent adopts a traditional artificial potential field method to make a behavior decision, and an obstacle intelligent agent is set as a static obstacle. In a certain time, the captive intelligent bodies learn the cooperative behaviors, the achievement realizes the enclosure of the target intelligent bodies, and the self-adaptive and distributed cooperative advantages of the method provided by the invention in the process of coping with complex and dynamic multi-intelligent-body behavior decision are shown.
The description in the above embodiment is a data processing method at the current time, and at the next timeWill be based onAnd (5) extracting the time-space characteristics and generating behavior decision according to the method, namely updating the time step, returning to the step S100, and calculating the next time.
The description in the embodiment is a process of space-time feature extraction and behavior decision generation of the intelligent agents, and each intelligent agent adopts the method to calculate at the same time; each intelligent agent can independently calculate, and the calculation result can be sent to each intelligent agent after the calculation of each intelligent agent is carried out through the cloud platform.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the above-described distributed multi-agent behavior decision method may refer to the corresponding process in the foregoing embodiment of the distributed multi-agent spatiotemporal feature extraction method, and details are not repeated herein.
The distributed multi-agent space-time feature extraction system comprises a state vector acquisition module, an original feature generation module, a space relation calculation module and a space-time relation calculation module;
the state vector acquisition module is configured to acquire the state vector at the moment ,Based onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the number is a preset historical state number;
the primitive feature generation module configured to generate a primitive feature based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,a selected feature space dimension;
the spatial relationship calculation module is configured to calculate the spatial relationship based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a The spatial feature relation extraction layer is constructed in a mode that a graph attention network module and a full-connection network module are stacked alternately;
the time-space relation calculation module is configured to obtain timeFront sideIndividual moment intelligent agentSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector of。
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the above-described distributed multi-agent spatiotemporal feature extraction system may refer to the corresponding process in the foregoing embodiment of the distributed multi-agent spatiotemporal feature extraction method, and will not be described herein again.
The distributed multi-agent behavior decision system of the fourth embodiment of the invention is based on the above-mentioned distributed multi-agent space-time feature extraction system, and also includes a behavior decision calculation module;
the behavioral decision calculation module is configured to be based on an agentAt the current momentSpace-time relation characteristic vector ofComputing agents using model-knowledge-driven methods or reinforcement learning data-driven methodsAt the current momentBehavioral decision set of(ii) a Wherein,,is the chosen decision space dimension.
The flow chart of this embodiment is shown in fig. 7.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the above-described distributed multi-agent behavior decision system may refer to the corresponding process in the foregoing embodiment of the distributed multi-agent behavior decision method, and details are not repeated herein.
It should be noted that, the distributed multi-agent spatiotemporal feature extraction system and the distributed multi-agent behavior decision system provided in the foregoing embodiments are only exemplified by the division of the above functional modules, and in practical applications, the above functions may be allocated by different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiments may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the above described functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
A storage device according to a fifth embodiment of the present invention stores a plurality of programs, which are suitable for being loaded and executed by a processor to implement the above-mentioned method for extracting spatiotemporal features of distributed multi-agent, and/or the above-mentioned method for deciding the behavior of distributed multi-agent.
A processing apparatus according to a sixth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; the program is suitable to be loaded and executed by a processor to realize the above-mentioned distributed multi-agent space-time feature extraction method and/or the above-mentioned distributed multi-agent behavior decision method.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication section, and/or installed from a removable medium. The computer program, when executed by a Central Processing Unit (CPU), performs the above-described functions defined in the method of the present application. It should be noted that the computer readable medium mentioned above in the present application may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (13)
1. A distributed multi-agent space-time feature extraction method is characterized by comprising the following steps:
step S100, at the time ,Based onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the number is a preset historical state number;
step S200 based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,a selected feature space dimension;
step S300 based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a The spatial feature relation extraction layer is constructed in a mode that a graph attention network module and a full-connection network module are stacked alternately;
step S400, acquiring timeFront sideIndividual moment intelligent agentSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector of。
2. The distributed multi-agent spatiotemporal feature extraction method as defined in claim 1, wherein "agent at each moment of time" in step S100Observable spatial state vectors "including agent's own state, task goal state, other intelligence observableEnergy state, observable environmental element state;
the self state of the intelligent agent comprises the self position, speed and acceleration state of the intelligent agent;
the task target state comprises a target position and a speed state; the observable other agent states include observable location, speed states of other agents;
the observable environmental element states include observable position and velocity states of obstacles in the environment, and position states of no-traffic zones in the environment.
3. The distributed multi-agent spatiotemporal feature extraction method as defined in claim 1, wherein the graph network generation layer in step S200 is composed of a plurality of layers of fully connected neural networks.
4. The method for extracting spatiotemporal features of distributed multi-agent as claimed in claim 1, wherein the method for extracting spatial feature relationship based on the spatial feature relationship extraction layer in step S300 comprises:
step S310, using the agentOf the original feature vector0 And the original feature vectors of all neighboring agents0 Obtaining, as input, a spatial relationship feature vector by a first graph attention network module1 (ii) a Wherein,,as an agentA set of neighbor agents capable of direct communication; 0 , 0 ;
step S320, in1 For input, obtaining space relation characteristic vector through first full-connection network module2 ;
Step S330, based on the space relation feature vector obtained in the step S320, through the stacked attention network module and the full-connection network module, the method of the step S310 and the step S320 is adopted to iteratively calculate the secondSpatial relationship feature vector of order
f2-1 、 f 2 (ii) a Wherein,,the number of stacked layers of the power network module and the fully connected network module is noted for the figure;
step S340, in the iterative computationThen, based on k2(-1) By the method of step S310, throughObtaining space relation characteristic vector by drawing attention network module k2-1 (ii) a Splicing vector value0 ,2 ,4 ,…, k2(-1) , k2-1 ]Input the firstFully connecting network modules to obtainSpatial relationship feature vector k 2 As time of dayLower intelligent agentFinal output based on the spatial feature relationship extraction layer。
5. The distributed multi-agent spatiotemporal feature extraction method as defined in claim 4, wherein the spatial relationship feature vector1 The acquisition method comprises the following steps:
step S311, using a learnable matrixPair relation feature vector0 、0 Linear transformation is carried out and spliced into a new relation characteristic vector0 =[ 0 , 0 ](ii) a Will be provided with0 Inputting a full-connection neural network and outputting an intelligent agentFor intelligent agentAttention coefficient of(ii) a Obtaining an agentFor intelligent agentAttention normalization coefficient of,
Step S312, adopting a multi-head attention mechanism for the first timeCalculating the normalized attention coefficient by the method of step S311(ii) a Intelligent agent under fusion of computing multi-head attention mechanismSpatial relationship feature vector of1 ,
6. The distributed multi-agent spatiotemporal feature extraction method as defined in claim 1, wherein the graph convolution operation based long and short term memory network with peepholes comprisesThe long and short term memory network units are connected in series and provided with peepholes; and an input gate, a forgetting gate and an output gate of the long and short term memory network unit are constructed based on a graph convolution neural network.
7. The distributed multi-agent spatiotemporal feature extraction method as claimed in claim 6, wherein the spatiotemporal relationship feature vectorThe acquisition method comprises the following steps:
the long-short term memory network unit near the output end is recorded asAnd increase the serial number in the reverse direction;
first, theA long and short term memory network unitThe cell state of (A) is recorded asThe output is a space-time relation feature vectorInput is asTemporal spatial relationship feature vectorThe first stepp+1 unitOutput space-time relation feature vectorAnd cell state thereof(ii) a Wherein,;
8. The distributed multi-agent spatiotemporal feature extraction method as defined in claim 7, wherein the spatiotemporal relationship is characterizedEigenvectorThe calculation method comprises the following steps:
step 401, willTemporal spatial relationship feature vectorThe first stepA unitNetwork element status ofAnd the firstA unitOutput space-time relation feature vectorInputting the input data into a forgetting gate based on a neural network adopting graph convolution, and calculating to obtain an forgetting gate output variable,
Wherein,the operation of the convolution of the graph is shown,andthe weight coefficient matrix and the bias of the forgetting gate diagram convolutional neural network respectively,activating a function for sigmoid;
step 402, willTemporal spatial relationship feature vectorThe first stepA unitOutput space-time relation feature vectorAnd cell state thereofInputting to an input gate based on the use of a convolutional neural network, and comparing the cell statesUpdating, wherein the calculation formula is as follows:
wherein,is the output of the input gate or gates,in order to be a transitional state of the cell,、the weight coefficient matrix corresponding to the neural network is convolved for the input gate map,、the corresponding bias of the neural network is convolved for the input gate map,for the purpose of the tanh activation function,is the Hadamard product;
at step 403, willTemporal spatial relationship feature vectorThe first stepA unitOutput space-time relation feature vectorAnd step 402 updatedA unitNetwork element status ofInput to an output gate based on a neural network using convolution of a graph to obtainA unitOutput space-time relation feature vectorThe calculation formula is as follows:
9. A distributed multi-agent spatiotemporal feature extraction method according to any of claims 1-8, characterized in that all agents in the distributed multi-agent share learnable parameters.
10. A distributed multi-agent behavior decision method, characterized in that an agent is obtained based on the distributed multi-agent spatiotemporal feature extraction method of any one of claims 1 to 9At the current momentSpace-time relation characteristic vector ofComputing agents using model-knowledge-driven methods or reinforcement learning data-driven methodsAt the current momentBehavioral decision set of(ii) a Wherein,,is the chosen decision space dimension.
11. A distributed multi-agent behavior decision method as defined in claim 10, wherein all agents in the distributed multi-agent share learnable parameters.
12. A distributed multi-agent space-time feature extraction system is characterized by comprising a state vector acquisition module, an original feature generation module, a space relation calculation module and a space-time relation calculation module;
the state vector acquisition module is configured to acquire the state vector at the moment ,Based onAgent for each moment from momentObservable spatial state vectorSplicing to obtain agentsAt the current momentSpace-time state vector of(ii) a Wherein,the number is a preset historical state number;
the primitive feature generation module configured to generate a primitive feature based onObtaining agents by generating layers through a graph networkOf the original feature vector(ii) a Wherein,,a selected feature space dimension;
the spatial relationship calculation module is configured to calculate the spatial relationship based onObtaining the intelligent agent through the spatial characteristic relation extraction layerAt the current momentSpace relation characteristic vector of(ii) a Wherein the spatial feature relation extraction layer adopts a graphThe attention network module and the full-connection network module are constructed in an alternate stacking mode;
the time-space relation calculation module is configured to obtain timeFront sideIndividual moment intelligent agentSpatial relationship feature vector ofWill be、Inputting the data into a time-space feature extraction layer, outputting an intelligent agent by adopting a long-term and short-term memory network with peepholes based on graph convolution operationAt the current momentSpace-time relation characteristic vector of。
13. A distributed multi-agent behavior decision making system, based on the distributed multi-agent spatiotemporal feature extraction system of claim 12, further comprising a behavior decision calculation module; the behavioral decision calculation module is configured to be based on an agentAt the current momentSpace-time relation characteristic vector ofComputing agents using model-knowledge-driven methods or reinforcement learning data-driven methodsAt the current momentBehavioral decision set of(ii) a Wherein,,is the chosen decision space dimension.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010873794.8A CN111738372B (en) | 2020-08-26 | 2020-08-26 | Distributed multi-agent space-time feature extraction method and behavior decision method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010873794.8A CN111738372B (en) | 2020-08-26 | 2020-08-26 | Distributed multi-agent space-time feature extraction method and behavior decision method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111738372A true CN111738372A (en) | 2020-10-02 |
CN111738372B CN111738372B (en) | 2020-11-17 |
Family
ID=72658807
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010873794.8A Active CN111738372B (en) | 2020-08-26 | 2020-08-26 | Distributed multi-agent space-time feature extraction method and behavior decision method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111738372B (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112203291A (en) * | 2020-12-03 | 2021-01-08 | 中国科学院自动化研究所 | Cluster control method for area coverage and connectivity maintenance based on knowledge embedding |
CN113253684A (en) * | 2021-05-31 | 2021-08-13 | 杭州蓝芯科技有限公司 | Multi-AGV (automatic guided vehicle) scheduling method and device based on graph convolution neural network and electronic equipment |
CN113296502A (en) * | 2021-05-08 | 2021-08-24 | 华东师范大学 | Multi-robot collaborative navigation method based on hierarchical relation graph learning in dynamic environment |
CN114118400A (en) * | 2021-10-11 | 2022-03-01 | 中国科学院自动化研究所 | Concentration network-based cluster countermeasure method and device |
CN114371719A (en) * | 2021-12-09 | 2022-04-19 | 湖南国天电子科技有限公司 | SAC-based autonomous control method for underwater robot |
CN115268481A (en) * | 2022-07-06 | 2022-11-01 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle countermeasure strategy decision method and system |
CN115439932A (en) * | 2022-09-02 | 2022-12-06 | 四元生命工程有限公司 | Behavior and information detection method taking regular tetrahedron framework as logic thinking |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190251358A1 (en) * | 2017-03-30 | 2019-08-15 | Hrl Laboratories, Llc | System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering |
CN111401557A (en) * | 2020-06-03 | 2020-07-10 | 超参数科技(深圳)有限公司 | Agent decision making method, AI model training method, server and medium |
-
2020
- 2020-08-26 CN CN202010873794.8A patent/CN111738372B/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190251358A1 (en) * | 2017-03-30 | 2019-08-15 | Hrl Laboratories, Llc | System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering |
CN111401557A (en) * | 2020-06-03 | 2020-07-10 | 超参数科技(深圳)有限公司 | Agent decision making method, AI model training method, server and medium |
Non-Patent Citations (2)
Title |
---|
PAULA FRAGA-LAMAS等: "A Review on IoT Deep Learning UAV Systems for Autonomous Obstacle Detection and Collision Avoidance", 《REMOTE SENSING》 * |
刘铭: "信息演化网络在分布式协同决策中的研究与应用", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112203291A (en) * | 2020-12-03 | 2021-01-08 | 中国科学院自动化研究所 | Cluster control method for area coverage and connectivity maintenance based on knowledge embedding |
CN113296502A (en) * | 2021-05-08 | 2021-08-24 | 华东师范大学 | Multi-robot collaborative navigation method based on hierarchical relation graph learning in dynamic environment |
CN113253684A (en) * | 2021-05-31 | 2021-08-13 | 杭州蓝芯科技有限公司 | Multi-AGV (automatic guided vehicle) scheduling method and device based on graph convolution neural network and electronic equipment |
CN113253684B (en) * | 2021-05-31 | 2021-09-21 | 杭州蓝芯科技有限公司 | Multi-AGV (automatic guided vehicle) scheduling method and device based on graph convolution neural network and electronic equipment |
CN114118400A (en) * | 2021-10-11 | 2022-03-01 | 中国科学院自动化研究所 | Concentration network-based cluster countermeasure method and device |
CN114118400B (en) * | 2021-10-11 | 2023-01-03 | 中国科学院自动化研究所 | Concentration network-based cluster countermeasure method and device |
CN114371719A (en) * | 2021-12-09 | 2022-04-19 | 湖南国天电子科技有限公司 | SAC-based autonomous control method for underwater robot |
CN114371719B (en) * | 2021-12-09 | 2023-08-08 | 湖南国天电子科技有限公司 | SAC-based autonomous control method for underwater robot |
CN115268481A (en) * | 2022-07-06 | 2022-11-01 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle countermeasure strategy decision method and system |
CN115439932A (en) * | 2022-09-02 | 2022-12-06 | 四元生命工程有限公司 | Behavior and information detection method taking regular tetrahedron framework as logic thinking |
CN115439932B (en) * | 2022-09-02 | 2023-10-10 | 四元生命工程有限公司 | Behavior and information detection method taking regular tetrahedral architecture as logic thinking |
Also Published As
Publication number | Publication date |
---|---|
CN111738372B (en) | 2020-11-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111738372B (en) | Distributed multi-agent space-time feature extraction method and behavior decision method | |
Tzafestas | Synergy of IoT and AI in modern society: The robotics and automation case | |
Odonkor et al. | Distributed operation of collaborating unmanned aerial vehicles for time-sensitive oil spill mapping | |
CN110737968B (en) | Crowd trajectory prediction method and system based on deep convolutional long and short memory network | |
Hussein et al. | A bi-directional agent-based pedestrian microscopic model | |
CN112686281A (en) | Vehicle track prediction method based on space-time attention and multi-stage LSTM information expression | |
CN113159283B (en) | Model training method based on federal transfer learning and computing node | |
CN113313947A (en) | Road condition evaluation method of short-term traffic prediction graph convolution network | |
CN114519932B (en) | Regional traffic condition integrated prediction method based on space-time relation extraction | |
Wei et al. | Learning motion rules from real data: Neural network for crowd simulation | |
Gollapalli et al. | A Neuro-Fuzzy Approach to Road Traffic Congestion Prediction. | |
CN113191241A (en) | Model training method and related equipment | |
CN113253733A (en) | Navigation obstacle avoidance method, device and system based on learning and fusion | |
Roldán et al. | Swarmcity project: Can an aerial swarm monitor traffic in a smart city? | |
CN114639233A (en) | Congestion state prediction method and device, electronic equipment and storage medium | |
CN112115744B (en) | Point cloud data processing method and device, computer storage medium and electronic equipment | |
Li et al. | Multi-mechanism swarm optimization for multi-UAV task assignment and path planning in transmission line inspection under multi-wind field | |
Xu et al. | Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Survey | |
CN113537267A (en) | Method and device for generating countermeasure sample, storage medium and electronic equipment | |
CN107225571A (en) | Motion planning and robot control method and apparatus, robot | |
Jolfaei et al. | Guest editorial introduction to the special issue on deep learning models for safe and secure intelligent transportation systems | |
CN111814915B (en) | Multi-agent space-time feature extraction method and system and behavior decision method and system | |
Lei et al. | Digital twin‐based multi‐objective autonomous vehicle navigation approach as applied in infrastructure construction | |
CN115762147A (en) | Traffic flow prediction method based on adaptive graph attention neural network | |
Skulimowski et al. | Communication quality in anticipatory vehicle swarms: A simulation-based model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |