CN113689576A

CN113689576A - Multi-agent scenario planning method under virtual fire scene

Info

Publication number: CN113689576A
Application number: CN202110978546.4A
Authority: CN
Inventors: 周文; 蒋文英; 张晨; 陈思源; 接标; 卞维新
Original assignee: Anhui Normal University
Current assignee: Anhui Normal University
Priority date: 2021-08-24
Filing date: 2021-08-24
Publication date: 2021-11-23

Abstract

The invention discloses a multi-agent path planning method under a virtual fire scene, which comprises the following steps: step 1: establishing a corresponding environment model based on the selected real disaster environment, extracting coordinate information in a scene and generating a three-dimensional scene corresponding to a real scene map; step 2: creating character model skeleton animation, regarding the character model skeleton animation as an intelligent agent, realizing group intelligent agent path planning based on a DQN algorithm, and adjusting a motion path by the intelligent agent according to a corresponding planned path; and step 3: the virtual reality device shows the escape process in an immersive manner, so that the disaster escape exercise effect with low cost and no risk is realized. The invention has the advantages that: the method can simulate various emergency situations after a fire disaster occurs, thereby vividly virtualizing the actions taken by crowds in possible emergency states and providing a brand-new solution for emergency management exercises. Due to the adoption of the computer virtual reality scene mode, the disaster exercise effect with low cost, zero risk, vividness and reliability can be realized.

Description

Multi-agent scenario planning method under virtual fire scene

Technical Field

The invention relates to the field of fire modeling simulation, in particular to a method for planning a group of intelligent agent scenes in a virtual fire scene.

Background

In real life, an emergency may occur at any time and any place. Particularly, in public areas where a large number of people are gathered, people are often scared once disaster accidents happen in the places, and vicious events such as crowd congestion and trampling are easily caused, so that unnecessary casualties are finally caused. Empirical data indicate that the main cause of casualties is not usually the actual disaster, but rather anxiety and impulsive behavior of the population in panic, which occurs largely due to lack of targeted daily exercise or protocol. To minimize casualties, many organizations in today's society engage in emergency preparation and provide realistic exercise training.

It is of utmost importance for emergency management personnel how to prepare for a disaster event that has not yet occurred. For people in which a large number are affected, appropriate safety measures mean a difference between life and death. However, emergency situations and their associated safety measures must be determined depending on the specific environment in which they are located, and the extraordinary, unpredictable, typically catastrophic and uncontrollable nature of the emergency presents significant difficulties for emergency decision-making and management, and many emergency situations must often be considered. The cost of testing these multiple scenarios is often prohibitive, and large-scale disaster scenarios are difficult to reproduce in real life. In addition, the scene of the emergency exercise is often too simple, the participants are in a form, and the emergency exercise slowly develops towards entertainment and amateur, so that the expected effect is difficult to achieve. Meanwhile, unnecessary casualties often occur in the exercise process due to carelessness in management and the situation of the participators, which lead to small exercise scale and unexpected effect.

In recent years, with the development of virtual reality VR technology, especially the popularization and use of portable and head-mounted VR equipment, a new realization form is provided for disaster exercises under the situation, the exercise cost can be reduced, unnecessary casualties in the exercise process can be avoided, and meanwhile, a solution with immersion is provided for developing large-scale crowd-intensive disaster scene exercises. Therefore, the virtual reality technology VR is adopted to replace real crowd disaster exercises, so that the virtual reality system has a plurality of remarkable advantages, but a plurality of bottleneck problems are urgently needed to be solved, including virtual realization of real scenes, exploration of crowd intelligence and the like. In recent years, with the rapid development of artificial intelligence technology AI, a new thought and direction are provided for the exploration of group intelligence, therefore, the invention combines AI and VR technologies to establish a group intelligent body path planning method under a large-scale virtual disaster scene, realizes the virtualization of the escape exercise process of people in the disaster scene and assists large-scale disaster exercise. The invention selects common fire as a disaster to deduce.

Therefore, the invention provides a group intelligent agent path planning algorithm under a large-scale virtual fire scene.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a multi-agent path planning method under a virtual fire scene, which simulates the behaviors possibly taken by people in various emergency states under the fire scene through simulation planning and simulation of various emergency conditions after the fire occurs, provides data support for emergency management and can simulate various emergency events under real conditions.

In order to achieve the purpose, the invention adopts the technical scheme that: a method for planning a group intelligent agent path under a virtual fire scene comprises the following steps:

step 1: establishing an environment model for the selected real scene map, extracting coordinate information in the scene and generating a three-dimensional plane map corresponding to the real scene map;

step 2: creating character model animation, regarding each character model as an intelligent agent, virtualizing the escape process of people, realizing escape path planning of group intelligent agents, and adjusting the motion path of each intelligent agent according to the planned path;

and step 3: the group intelligent agent vividly shows the escape process of the real crowd in a form under a virtual real scene, including individual escape forms, escape paths and escape group intelligence of the crowd, so that a new form of real disaster exercises (such as fire exercises) is provided. A swarm agent is a multi-agent with social attributes. The group intelligent agent is a multi-intelligent agent with social attributes, and comprises multi-intelligent agents with three social relations of cooperation, competition, zero and zero.

Step 1, modeling a three-dimensional real scene, and creating character model skeleton animation, wherein the virtual scene is a map containing static map information and related crowd information; the static map information describes physical attributes of the environment; the crowd information includes a description of location information and crowd density information.

In step 2, the escape path planning of the group agents comprises the following steps: the escape path planning is to select the most suitable path for the group intelligent agents and control the intelligent agents to operate according to the path so as to carry out the virtual and real crowd escape process. In order to realize group intelligence, a deep reinforcement learning algorithm is adopted, in the training process, based on the exploration-trial-and-error large-scale attempt of an intelligent body state-behavior space in a scene, the optimal network structure and the optimal hyper-parameters are trained by combining a deep learning network as a reliable cost function evaluation and optimization device according to the feedback reward of the environment in the intelligent body behavior-state process, the trained deep learning network takes any position in an input scene as parameter input, and the motion path of each intelligent body as an output parameter to control the motion of each intelligent body.

In the escape path planning of the group intelligent agents, according to real fire exercise videos and data, the crowd often selects routes in a multi-group mode, in order to effectively simulate real scene activities, the group intelligent agents are also realized in a group mode, each group comprises a plurality of intelligent agents and plans to be the same path, and the intelligent agents move according to the planned path.

The grouping of groups includes: setting trust and aggregation for each individual according to the motion trajectory data of the real crowd, clustering and grouping large-scale crowds based on trust values based on a leader-follower framework, and selecting a leader from the group; after the crowd clustering grouping, the crowd is divided into two roles of a leader and a follower, the follower moves according to the direction selected by the leader in the evacuation process, and when the path is planned, the path of the leader is planned, and the followers in the same group move along with the leader.

The crowd escape track is extracted from the video of the large-scale disaster scene in real life, and the deep learning network is trained by taking the track as a learning sample.

And controlling the movement of the agents in the group, setting the speed direction and the speed of each agent on the planned path, and controlling the movement of each agent according to the speed direction and the speed.

In step 3, the group agents realistically display the escape process of the real population in a virtual real scene, including individual escape forms, escape paths and escape group intelligence of the population, so as to provide a new form of real disaster exercise (such as fire exercise).

The invention has the advantages that: the method has the advantages that the immersive fire scene is vividly virtualized, the intelligent path planning is carried out on the escape behaviors of the group agents in the scene, and a new development form is provided for fire emergency exercises. Meanwhile, various other emergencies including the terrorist attack of people and the like can be expanded, the exercise cost is greatly reduced, unnecessary casualties of people are reduced to zero, and meanwhile, due to the high immersion, the defects of entertainment, falling into forms and the like of exercise participation personnel can be overcome to a certain extent.

Drawings

The following is a brief description of the contents of the drawings and the symbols in the drawings, which are expressed in the specification of the present invention:

fig. 1 is a framework diagram of the evacuation process of group agents according to the present invention;

FIG. 2 is a flow chart of the escape path planning of the group agents according to the present invention;

FIG. 3 is a deep reinforcement learning network of the present invention: DQN network training path procedure

Detailed Description

The following description of preferred embodiments of the invention will be made in further detail with reference to the accompanying drawings.

In order to reduce the high cost and high risk faced in the actual disaster drilling (practicing), a series of problems such as lack of reality and flowing in form of drilling scene are solved. Therefore, the virtual reality technology is adopted to overcome the defects to a certain extent, namely, the participants pass through virtual helmet equipment (such as AR/VR equipment), the avatar is an intelligent body, the immersive virtualization mode is put into the exercise of the disaster scene, the problem that the participants cannot be really put into the exercise is overcome to a certain extent, meanwhile, the exercise cost is greatly reduced, potential safety hazards and the like which may appear in the exercise are avoided, and in order to more vividly virtualize the behaviors of the group intelligent bodies, it is very necessary to research the group wisdom of the group intelligent bodies in the disaster scene. Therefore, the invention provides a method for path planning in a large-scale virtual fire scene, and participants are immersed in fire exercises.

As shown in fig. 1 and 2, the path planning algorithm in the large-scale virtual fire scene specifically performs the following operations.

(1) Environment modeling: and carrying out scene modeling according to the real environment and creating skeleton animation of the character model. The environment model is a map containing static map information and related crowd information; the static map information describes physical attributes of the environment; the crowd information comprises position information and description of crowd density information; generally, the considered crowd density information is static, while the location information changes with the behavior of the individual and is dynamic.

(2) Learning of escape track data of real people: due to the wide use of the monitoring equipment, the crowd escape track is extracted from the video of the large-scale disaster scene in real life, and the track is used as a learning sample to establish a state-behavior reward mechanism in the reinforcement learning process, so that the behavior of the intelligent agent in the virtual scene is more vivid and reasonable. In order to obtain the crowd escape trajectory data, the video of the emergency situation should be learned, including analyzing the segmentation of video objects (crowd instances), multi-frame segmentation trajectory extraction, and the like.

(3) Group generation algorithm: in the process of real evacuation, people can gather in the subconscious and show following and leading behaviors to form a group in a certain sense so as to carry out group escape. After the crowd clustering grouping, the crowd is divided into two roles of a leader and a follower, and the follower moves according to the direction selected by the leader in the evacuation process, so that different groups are formed. In a virtual scene, for the group agents, planning a path for each agent is obviously unrealistic and unnecessary, and by constructing a plurality of agent groups, the calculated amount of the system can be reduced and the escape crowd in the virtual reality can be vividly simulated.

(4) Analyzing the group intelligent agent behaviors: in a real scene, when a disaster happens, people can show various behaviors, such as observation and negligence due to panic; following running or walking with bamboo on the chest, etc.; in a virtual scene, the agent is an approximate simulation of evacuating people, so the agent should also exhibit various escape behaviors, such as walking, looking, running; and analyzing the action which the intelligent agent may take according to the surrounding environment of the intelligent agent in the virtual scene and the like. And according to the behaviors, the intelligent agent is controlled to move through an algorithm. The moving mode of the intelligent body mainly comprises a speed direction and a speed magnitude.

(5) And (3) planning the escape paths of the group agents: the escape path planning is the most important step, the most appropriate path is selected for the intelligent agent, a deep reinforcement learning method is adopted, and in the training process, based on the exploration-trial-and-error type large-scale attempt of the intelligent agent state-behavior space in the scene, the optimal network structure and the optimal hyper-parameters are trained by combining a deep learning network as a reliable cost function evaluator according to the feedback reward of the environment in the intelligent agent behavior-state process. In the testing process (application), by inputting any position in a scene, the deep learning network can output the optimal behavior (such as forward or backward or leftward or rightward) based on the trained parameters, plan a path which can effectively avoid obstacles and reach an escape exit for an intelligent agent, and achieve the goal of an escape scheme in a virtual scene.

Extracting real crowd escape track data, and extracting crowd behavior tracks from videos of large-scale disaster scenes in real life; setting trust and aggregation for each individual according to the motion trajectory data of the real crowd, clustering and grouping large-scale crowds based on trust values based on a leader-follower framework, and selecting a leader from the group; a group may be formed by the interaction between individuals who may change their behavior after the group is formed. When all individuals are divided into groups, the behavior of the individual will be determined by both the group behavior model and the individual behavior model. The individual behavior model is influenced by the upper group behavior model, and the individual state based on the environmental condition is updated;

the final choice of agent behavior is decided by examining the agent's current state and the actions previously taken.

The specific steps of the group intelligent agent path planning algorithm under the virtual fire scene are as follows:

step 1: according to a real environment, 3DS MAX software is used for establishing a scene model for a specific disaster scene (such as a fire); creating character model skeleton animation, adjusting the animation according to possible actions of an individual in a disaster scene, and providing support for a subsequent algorithm;

step 2: and (3) segmenting the video example by utilizing a deep learning network from the crowd video of the actual disaster exercise, and extracting the motion trail of the example. The method comprises the steps of segmenting objects in a video based on networks such as Mask-RCNN and the like, and extracting behavior track routes of the objects, so that data support is provided for a reward function of subsequent reinforcement learning.

And step 3: in consideration of trust and aggregation among individuals, the intelligent agents with close relations can show collective behaviors, and the crowd can be aggregated into a group in the moving process. Clustering groups of group agents in a leader-follower mode based on trust values, and generating a leader from a group according to a relevant strategy, wherein the mutual distances among the individuals in the group are neither too close nor too far. As trust values between individuals may change, the group selected by the individual may also change.

A plurality of agents are selected from the virtual scene to serve as leader agents, and other agents select the most appropriate following objects according to the positions, the trust values and the like of the agents, so that an escape structure of a leader-follower model is formed.

And 4, step 4: a DQN algorithm is one of deep reinforcement learning methods applied to a discrete environment. The DQN algorithm is used for path planning of the intelligent agent, a twin deep learning network is constructed based on the continuous state updating behavior of the Bellman formula, one network takes the s state of the intelligent agent in a scene as input, Q _ predict is output as Q (s, a; theta) through the network, the other network takes the s 'state calculated by the Bellman formula as input, Q _ target is output through the network, in the training process, the cost function of the deep learning network is used for evaluating the difference of Q values of the intelligent agent at the t-th time point and the t +1 time point in the states s and s', the minimum value L (theta) is finally reached through continuous iteration, the network parameter theta at the time is optimal, and the strategy track of the intelligent agent in the scene can be predicted through the neural network. The formula is as follows:

Q(s，a)←Q(s，a)+α[r+γmax_a′(s′，a′)-Q(s，a)]

L(θ)＝E{[Q_target-Q(s，a；θ)]²}

Q_target＝r+γmax_a′Q(S′，a′；θ)

the parameter theta represents a parameter of a deep learning network, two networks (a main network and a target network) with the same nerves are constructed by the DQN, the main network is used for learning the Q value of the state s at the time point t, the target network is used for fitting the Q value of the next time point s't +1 through averaging error MSE, and meanwhile, in each training period, the main network continuously copies the parameter of the main network to the target network. In order to increase randomness and training efficiency, massive samples are stored in an experience buffer pool, and in the training process, samples are randomly extracted from the buffer pool to train a network. As shown in fig. 3, the environment continuously generates a huge amount of samples, i.e. the sequence of state behaviors (s, a, r, s').

(1) The invention combines deep reinforcement learning and swarm intelligent agent path planning for the first time, and provides a path planning method under a large-scale virtual fire scene based on a deep Q learning network.

(2) The invention provides a group intelligent agent grouping method, which combines a leader-follower model with intelligent agent escape, simulates an intelligent agent to escape along with a leader in a virtual environment, and greatly reduces the calculated amount of large-scale multi-intelligent agent escape, thereby ensuring certain authenticity and reducing the time overhead of multi-intelligent agent path planning.

(3) According to the method, crowd movement track data are extracted from the real disaster exercise video, relevant track samples are established, and an excitation mechanism of reinforcement learning is established in a simulation learning mode, so that reliable data support is provided for the reinforcement learning.

It is clear that the specific implementation of the invention is not restricted to the above-described embodiments, but that various insubstantial modifications of the inventive process concept and technical solutions are within the scope of protection of the invention.

Claims

1. A method for planning a group intelligent agent scene under a virtual fire scene is characterized by comprising the following steps: the method comprises the following steps:

step 1: establishing a three-dimensional environment model for the selected real scene map, extracting coordinate information in the three-dimensional scene and generating a three-dimensional map corresponding to the real scene map;

step 2: creating character model skeleton animation, regarding each character model as an intelligent body, thereby forming a group intelligent body of a virtual real disaster scene, planning escape paths in the virtual real disaster scene for each intelligent body, and adjusting motion paths of the group intelligent bodies according to the planned paths;

and step 3: the crowd intelligent agent shows the escape process of real crowd in a form under a virtual real scene, including individual escape form, escape path and escape crowd intelligence, wherein the escape crowd intelligence refers to crowd artificial intelligence shown by a plurality of intelligent agents (crowd intelligent agents) with social attributes in the escape scene, namely twin crowd artificial intelligence for realizing crowd escape.

2. The multi-agent scenario planning method of claim 1, wherein: in the step 1, a virtual three-dimensional technology is utilized to model a scene and create a character skeleton animation, and an environment model is a map containing static map information and related crowd information; the static map information describes physical attributes of the environment; the crowd information includes a description of location information and crowd density information.

3. The multi-agent scenario planning method of claim 1, wherein: the virtual escape path planning of the group of intelligent agents in the step 2 comprises the following steps: the escape path planning is that the group intelligent bodies select the most appropriate path according to the self related information (such as positions, group structures and the like) and control the group intelligent bodies to move according to the path, thereby realizing the virtualization of the group behaviors in the real scene; the escape path planning adopts a deep Q learning DQN algorithm, in the training process, the intelligent body conducts large-scale trial of state-behavior space exploration-trial-and-error based on a Bellman formula, the optimal network structure and the optimal hyper-parameters are trained by combining a deep neural network as a reliable cost function evaluator according to feedback rewards of the environment in the intelligent body behavior-state process, and the trained deep neural network takes an image of the intelligent body at a certain position in a scene as input and outputs a related intelligent body motion strategy, namely a state-behavior track.

4. The multi-agent scenario planning method of claim 3, wherein: in the escape path planning of the virtual real crowd, the group agents are grouped in a group mode, the group comprises a plurality of agents, the agents in each group are planned to be the same path, and move according to the planned path.

5. The method of claim 4, wherein the method comprises: the grouping of groups includes: setting trust and aggregation for each individual according to the motion trajectory data of the real crowd, clustering and grouping large-scale crowds based on trust values based on a leader-follower framework, and selecting a leader from the group; after the crowd clustering grouping, the crowd is divided into two roles of a leader and a follower, the follower moves according to the direction selected by the leader in the evacuation process, and when the path is planned, the path of the leader is planned, and the followers in the same group move along with the leader.

6. The method of claim 3, wherein the method comprises: the crowd escape track is extracted from the video of the large-scale disaster scene in real life, and the deep learning network is trained by taking the track as a learning sample.

7. The method of claim 5, wherein the method comprises: and controlling the movement of the agents in the group, setting the speed direction and the speed of each agent on the planned path, and controlling the movement of each agent according to the speed direction and the speed.

8. The method for multi-agent scenario planning under virtual fire scenario as claimed in any of claims 1-7, wherein: in step 3, the movement of the group of agents is controlled according to the planned path, the moving direction and the moving speed of each agent, and the agents are displayed in a visual form such as a webpage, or virtual experience with immersion is performed through equipment such as a head-mounted virtual helmet and virtual glasses.