CN114020041A - Multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method and system - Google Patents
Multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method and system Download PDFInfo
- Publication number
- CN114020041A CN114020041A CN202111524975.0A CN202111524975A CN114020041A CN 114020041 A CN114020041 A CN 114020041A CN 202111524975 A CN202111524975 A CN 202111524975A CN 114020041 A CN114020041 A CN 114020041A
- Authority
- CN
- China
- Prior art keywords
- exploration
- unmanned aerial
- map
- aerial vehicle
- creating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 58
- 230000006870 function Effects 0.000 claims description 21
- 239000011159 matrix material Substances 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 12
- 238000012549 training Methods 0.000 description 7
- 230000002787 reinforcement Effects 0.000 description 6
- 239000003795 chemical substances by application Substances 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical group C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
- G05D1/104—Simultaneous control of position or course in three dimensions specially adapted for aircraft involving a plurality of aircrafts, e.g. formation flying
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method and system, and relates to the technical field of unmanned aerial vehicles. A multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method comprises the steps of establishing an initial exploration map; then, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map; then respectively creating corresponding threads for each simulation unmanned aerial vehicle; and finally, controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model respectively, generating a two-dimensional exploration map, realizing multi-machine collaborative exploration, thereby being capable of saving exploration time and improving exploration efficiency.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicles, in particular to a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method and system.
Background
Along with the continuous improvement of the application level of the unmanned aerial vehicle in the fields of resource exploration, safety prevention and control, emergency rescue and disaster relief and the like, the unmanned aerial vehicle replaces the human beings to finish the work of badness, danger and boring, greatly expands the dimensionality, hierarchy and field of the human beings exploring nature and society, and generates considerable social and economic values.
In the aspects of natural resource exploration, disaster monitoring and assessment, border safety inspection and monitoring and the like, the tasks to be completed usually aim at space exploration without clear path planning from a starting point to an end point, and the existing scheme adopts a single unmanned aerial vehicle for exploration, but the single unmanned aerial vehicle is low in exploration efficiency and poor in robustness.
Disclosure of Invention
The invention aims to provide a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method and system, which are used for solving the problems of low exploration efficiency and poor robustness of a single unmanned aerial vehicle in the prior art.
In a first aspect, an embodiment of the present application provides a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method, including the following steps:
creating an initial exploration map;
creating a plurality of simulated drones on the initial exploration map;
respectively creating corresponding threads for each simulation unmanned aerial vehicle;
and controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and respectively inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model, and generating a two-dimensional exploration map.
In the implementation process, an initial exploration map is created; then, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map; then respectively creating corresponding threads for each simulation unmanned aerial vehicle; and finally, controlling each simulated unmanned aerial vehicle to fly according to the corresponding thread, acquiring and inputting the current exploration data of each simulated unmanned aerial vehicle into a preset mappo algorithm model respectively, generating a two-dimensional exploration map, realizing multi-thread exploration of a plurality of simulated unmanned aerial vehicles in the same initial exploration map, and realizing multi-machine collaborative exploration by sharing a public area of exploration, thereby saving exploration time and improving exploration efficiency. The mappo algorithm model is used as a multi-agent deep reinforcement learning algorithm, continuous actions are output, the possibility of being applied to actual scenes is achieved, training is more stable, and boundary values cannot be obtained continuously due to actions caused by the structure of the critic network.
Based on the first aspect, in some embodiments of the present invention, the step of creating the initial exploration map includes the steps of:
creating a random number matrix;
assigning values to elements in the random number matrix according to a preset rule to obtain a plurality of random points;
inputting a plurality of random points into a preset filter to generate a rough map;
and creating a search layer on the rough map, and generating an initial search map.
Based on the first aspect, in some embodiments of the present invention, the step of creating a corresponding thread for each simulated drone includes the following steps:
aiming at each unmanned aerial vehicle, respectively adopting a threading module of python to designate a navigation function as a thread bearing function;
and executing the thread bearing function to obtain the thread of each unmanned aerial vehicle.
Based on the first aspect, in some embodiments of the present invention, the step of controlling each simulated unmanned aerial vehicle to fly according to a corresponding thread, acquiring and respectively inputting the current exploration data of each simulated unmanned aerial vehicle into a preset mappo algorithm model, and generating the two-dimensional exploration map includes the following steps:
controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread;
acquiring current exploration data of each simulated unmanned aerial vehicle;
respectively judging whether the current exploration area exceeds a preset exploration area according to the current exploration data of each simulation unmanned aerial vehicle, if so, outputting a current exploration map, and updating an actor network and a critic network in a preset mappo algorithm model to generate a new mappo algorithm model; if not, updating an actor network and a critic network in the preset map algorithm model to generate a new map algorithm model;
acquiring the current cycle number, and adding 1 to the current cycle number to obtain a new cycle number;
judging whether the new cycle number is equal to a preset screen number or not, and if so, taking the current exploration map as an exploration map; and if not, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map.
Based on the first aspect, in some embodiments of the present invention, the method further comprises the following steps:
when the current exploration graph is output, the operator network and the critic network in the current mappo algorithm model are output.
Based on the first aspect, in some embodiments of the present invention, the step of respectively determining whether the current exploration area exceeds the preset exploration area according to the current exploration data of each simulated unmanned aerial vehicle includes the following steps:
respectively extracting rewards in the current exploration data of each simulated unmanned aerial vehicle;
comparing the reward with the existing reward sum, and if the reward is greater than the existing reward sum, determining that the current exploration area exceeds the preset exploration area; and if the reward is not greater than the sum of the existing rewards, the current exploration area is considered not to exceed the preset exploration area.
In a second aspect, an embodiment of the present application provides a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation system, including:
the map creating module is used for creating an initial exploration map;
the unmanned aerial vehicle creating module is used for creating a plurality of simulated unmanned aerial vehicles on the initial exploration map;
the thread creating module is used for creating corresponding threads for each simulation unmanned aerial vehicle;
and the exploration module is used for controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and respectively inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model, and generating an exploration map.
In the implementation process, an initial exploration map is created through a map creation module; the unmanned aerial vehicle creation module creates a plurality of simulated unmanned aerial vehicles on the initial exploration map; the thread creating module creates corresponding threads for each simulation unmanned aerial vehicle; the exploration module controls each simulation unmanned aerial vehicle to fly according to the corresponding thread, current exploration data of each simulation unmanned aerial vehicle is acquired and respectively input into a preset mappo algorithm model, a two-dimensional exploration map is generated, multithread exploration of a plurality of simulation unmanned aerial vehicles in the same initial exploration map can be realized, multi-machine collaborative exploration is realized by sharing a public area of exploration, exploration time can be saved, exploration efficiency is improved, meanwhile, due to the fact that a plurality of simulation unmanned aerial vehicles exist, when a certain simulation unmanned aerial vehicle breaks down in the exploration process, other simulation unmanned aerial vehicles can continue to explore, and robustness of an exploration scheme is improved. The mappo algorithm model is used as a multi-agent deep reinforcement learning algorithm, continuous actions are output, the possibility of being applied to actual scenes is achieved, training is more stable, and boundary values cannot be obtained continuously due to actions caused by the structure of the critic network.
Based on the second aspect, in some embodiments of the invention, the map creation module includes:
a random number matrix unit for creating a random number matrix;
the random point unit is used for assigning values to elements in the random number matrix according to a preset rule to obtain a plurality of random points;
the filter unit is used for inputting a plurality of random points into a preset filter to generate a coarse map;
and a search map creation unit for creating a search layer on the rough map and generating an initial search map.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory for storing one or more programs; a processor. The one or more programs, when executed by the processor, implement the method as described in any of the first aspects above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the method as described in any one of the above first aspects.
The embodiment of the invention at least has the following advantages or beneficial effects:
the embodiment of the invention provides a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method and system, wherein an initial exploration map is created; then, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map; then respectively creating corresponding threads for each simulation unmanned aerial vehicle; and finally, controlling each simulated unmanned aerial vehicle to fly according to the corresponding thread, acquiring and inputting the current exploration data of each simulated unmanned aerial vehicle into a preset mappo algorithm model respectively, generating a two-dimensional exploration map, realizing multi-thread exploration of a plurality of simulated unmanned aerial vehicles in the same initial exploration map, and realizing multi-machine collaborative exploration by sharing a public area of exploration, thereby saving exploration time and improving exploration efficiency. The mappo algorithm model is used as a multi-agent deep reinforcement learning algorithm, continuous actions are output, the possibility of being applied to actual scenes is achieved, training is more stable, and boundary values cannot be obtained continuously due to actions caused by the structure of the critic network. Because the mappo algorithm model runs one training round, and has the capability of outputting a complete and large-area neural network for exploring the area, the operator network and the critic network in the current mappo algorithm model can be output and used in other places, and the use by a user is facilitated.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a flowchart of a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method provided by an embodiment of the present invention;
FIG. 2 is a schematic diagram of a rough map according to an embodiment of the present invention;
fig. 3 is a detailed flowchart of generating a two-dimensional exploration map by using a mappo algorithm model according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of a two-dimensional exploration map provided in accordance with an embodiment of the present invention;
fig. 5 is a structural block diagram of a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation system provided by the embodiment of the invention;
fig. 6 is a block diagram of an electronic device according to an embodiment of the present invention.
Icon: 1100-map creation module; 1110 — random number matrix unit; 1120-random Point Unit; 1130-a filter unit; 1140-an exploration map creation unit; 1200-a drone creation module; 1300-thread creation module; 1310-a function selection unit; 1320-a function execution unit; 1400-exploration module; 1410-a flight control unit; 1420-an exploration data acquisition unit; 1430-a judgment unit; 1431-reward extraction subunit; 1432-comparison subunit; 1440-cycle number update unit; 1450-cycle number judging unit; 1500-neural network output module; 101-a memory; 102-a processor; 103-communication interface.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.
Referring to fig. 1, fig. 1 is a flowchart of a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method according to an embodiment of the present invention. The multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method comprises the following steps:
step S110: creating an initial exploration map; the initial search map is a map created by simulating an area that the unmanned aerial vehicle needs to search, and may be generated by simulating an actual map or created by a random seed. The method comprises the steps that observation is needed to be used as input in depth reinforcement learning, the observed visual field needs to be capable of displaying a wall, the visual field of reward calculation does not display the wall, and therefore a multi-layer map building mechanism is needed for building an initial exploration map, the initial exploration map comprises a coarse map layer and a gray layer, the coarse map layer comprises a white area and a black area, the white area represents an unmanned aerial vehicle explorable area, the black area represents an obstacle and an unexplosive area, the gray layer covers the coarse map layer, and the unmanned aerial vehicle conducts exploration on the gray layer during exploration. The process of creating the initial exploration map by using the random seeds comprises the following steps:
firstly, creating a random number matrix; the creation may be a random generation of a series of points in a range of squares, which may be constructed as a random number matrix.
Then, assigning values to elements in the random number matrix according to a preset rule to obtain a plurality of random points; the preset rule may be that the element value greater than 0.5 is changed to 1, and the element value less than or equal to 0.5 is changed to 0, thereby obtaining a plurality of random points consisting of 0 and 1. Where 1 represents an obstacle and 0 represents a blank area, which can be searched.
Then, inputting a plurality of random points into a preset filter to generate a rough map; referring to fig. 2, fig. 2 is a schematic diagram of a rough map according to an embodiment of the present invention. The preset filter is a smoothing filter created according to a convolutional neural network, and a coarse map is generated by inputting a plurality of random points into the preset filter to perform smoothing convolution processing. The rough map includes a white area and a black area.
And finally, creating an exploration layer on the rough map and generating an initial exploration map. The search layer is a gray area, the whole rough map is covered, the unmanned aerial vehicle searches in the search layer during search, and when a white area is searched, the corresponding area on the search layer is changed into white. And carrying out map construction through the generated random matrix, and generating a simulated non-structural environment map in a softening way.
Step S120: creating a plurality of simulated drones on the initial exploration map; the creating process may be to set a plurality of simulated drones in a blank area of the initial search map, and the plurality of simulated drones may be in the same location or different locations. Above-mentioned emulation unmanned aerial vehicle can acquire exploration data, and above-mentioned emulation unmanned aerial vehicle just can establish through prior art, consequently, just no longer gives unnecessary details here.
Step S130: respectively creating corresponding threads for each simulation unmanned aerial vehicle; each simulation unmanned aerial vehicle corresponds to different threads and can be randomly generated. Multiple threads may be implemented with a threading library and controlled by while loops. The creating of the corresponding thread comprises the following steps:
firstly, aiming at each unmanned aerial vehicle, respectively adopting a threading module of python to designate a navigation function as a thread bearing function; and then, executing a thread bearing function to obtain the thread of each unmanned aerial vehicle. The thread is generated by adopting the threading module until the function is finished, and the creating of the multiple threads by adopting the threading module belongs to the prior art and is not described herein any more.
Step S140: and controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and respectively inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model, and generating a two-dimensional exploration map. After corresponding threads are created for each simulation unmanned aerial vehicle respectively, the simulation unmanned aerial vehicles fly according to the corresponding threads, circular exploration is started, currently obtained exploration data are input into a preset mappo algorithm model, continuous actions are generated, the simulation unmanned aerial vehicles further calculate the positions and the orientations of the next step according to the continuous actions, and the simulation unmanned aerial vehicles provide an action input interface which can input the continuous actions (a number from 0 to 1) as a rotation angle range and can also input discrete actions (an integer from 0 to 21) as a rotation direction. After the continuous action is converted into the corner, the unmanned aerial vehicle moves in the next step according to the corner, and the discrete action is to uniformly divide the continuous action into 21 parts. And (3) the simulated unmanned aerial vehicle carries out the next exploration according to the position and the direction of the next step, the steps are circulated until the circulation is stopped, and a two-dimensional exploration map is output, wherein the two-dimensional exploration map is used for marking the currently explored white area at the corresponding position of the dust layer in the exploration process and changing the marked area into white. The above-mentioned mappo algorithm model belongs to the prior art, and is not described herein again. The current exploration data comprises position data, rewards, punishments and the like of the current unmanned aerial vehicle. Referring to fig. 3 and 4, fig. 3 is a detailed flowchart of generating a two-dimensional exploration map by using a mappo algorithm model according to an embodiment of the present invention, and fig. 4 is a schematic diagram of the two-dimensional exploration map according to the embodiment of the present invention. The process of generating the two-dimensional search map includes the steps of:
firstly, controlling each simulation unmanned aerial vehicle to fly according to a corresponding thread; each simulation unmanned aerial vehicle flies according to respective thread and explores in the flying process.
Then, acquiring current exploration data of each simulated unmanned aerial vehicle; the current exploration data comprise position data, punishment, explored sector areas, observation of each step, reward, action, observation of the next step and the like of the simulated unmanned aerial vehicle at the current position. The rewards and penalties are stored in a pool of experience. The reward means that a part of area is searched and is rewarded; the penalty is given as-1000 for a wall strike.
Then, respectively judging whether the current exploration area exceeds a preset exploration area according to the current exploration data of each simulated unmanned aerial vehicle, if so, outputting a current exploration map, and updating an operator network and a critic network in a preset mappo algorithm model to generate a new mappo algorithm model; if not, updating an actor network and a critic network in the preset map algorithm model to generate a new map algorithm model; the above-mentioned judging whether the current search area exceeds the preset search area is to judge whether a larger search area is obtained, and may be by judging whether the current gray-whitened area exceeds the maximum value of the existing search white area, or by judging whether the current reward exceeds the existing reward sum. Updating the operator network and the critic network in the preset mappo algorithm model means that the operator network and the critic network are trained according to current exploration data, so that the operator network and the critic network can be stronger, the operator network and the critic network can be trained and an exploration map is output every cycle until the operator network and the critic network exceed a preset exploration area, and the current exploration map is output. The operator network is used for selecting actions, so that the simulated unmanned aerial vehicle can acquire the flight direction. The process of determining whether to obtain a larger search area through the reward comprises the following steps:
firstly, respectively extracting rewards in current exploration data of each simulated unmanned aerial vehicle; the bonus is that if a white area is searched in the search process, the area of the area is rewarded as a bonus value, or a value corresponding to the bonus area may be calculated.
Then comparing the reward with the existing reward sum, and if the reward is greater than the existing reward sum, determining that the current exploration area exceeds the preset exploration area; and if the reward is not greater than the sum of the existing rewards, the current exploration area is considered not to exceed the preset exploration area. The sum of the existing rewards is the sum of the rewards already in the experience pool, and the sum of the rewards may be calculated before the comparison and then compared.
Then, acquiring the current cycle number, and adding 1 to the current cycle number to obtain a new cycle number; the cycle number refers to the number of times calculated by using a mappo algorithm model, and is also called a curtain, and after one-time mappo algorithm model calculation, the cycle number is added with 1 to obtain a new cycle number. After the initial search map is built, a loop number parameter may be set, and the loop number parameter is assigned to 0, for example, the loop number parameter may be set to i, when the mappo algorithm model is first calculated, i is 0, after the mappo algorithm model is calculated once, i +1 is 1, and the calculation is performed once each subsequent mappo algorithm model is performed.
Finally, judging whether the new cycle number is equal to a preset screen number or not, and if so, taking the current exploration map as an exploration map; and if not, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map. The preset number of curtains may be set according to an actual situation, and is used to determine whether to end the cycle, for example, the preset number of curtains is M, and when the new cycle number is equal to M, the cycle is ended, and the current exploration map is output; and when the new cycle number is not equal to M, destroying the simulated unmanned aerial vehicle, regenerating a new simulated unmanned aerial vehicle, and further continuing exploring the new simulated unmanned aerial vehicle.
In the implementation process, an initial exploration map is created; then, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map; then respectively creating corresponding threads for each simulation unmanned aerial vehicle; and finally, controlling each simulated unmanned aerial vehicle to fly according to the corresponding thread, acquiring and inputting the current exploration data of each simulated unmanned aerial vehicle into a preset mappo algorithm model respectively, generating a two-dimensional exploration map, realizing multi-thread exploration of a plurality of simulated unmanned aerial vehicles in the same initial exploration map, and realizing multi-machine collaborative exploration by sharing a public area of exploration, thereby saving exploration time and improving exploration efficiency. The mappo algorithm model is used as a multi-agent deep reinforcement learning algorithm, continuous actions are output, the possibility of being applied to actual scenes is achieved, training is more stable, and boundary values cannot be obtained continuously due to actions caused by the structure of the critic network.
Wherein, still include the following step:
when the current exploration graph is output, the operator network and the critic network in the current mappo algorithm model are output. Because the mappo algorithm model runs one training round, and has the capability of outputting a complete and large-area neural network for exploring the area, the operator network and the critic network in the current mappo algorithm model can be output and used in other places, and the use by a user is facilitated.
Based on the same inventive concept, the invention further provides a multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation system, please refer to fig. 5, and fig. 5 is a structural block diagram of the multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation system provided by the embodiment of the invention. This many unmanned aerial vehicle multithread two-dimensional exploration simulation system includes:
a map creation module 1100 for creating an initial exploration map;
a drone creating module 1200 for creating a plurality of simulated drones on the initial exploration map;
the thread creating module 1300 is configured to create corresponding threads for each simulated unmanned aerial vehicle;
and the exploration module 1400 is configured to control each simulated unmanned aerial vehicle to fly according to the corresponding thread, acquire and input the current exploration data of each simulated unmanned aerial vehicle into a preset mappo algorithm model, and generate an exploration map.
In the implementation process, an initial exploration map is created through the map creation module 1100; the drone creating module 1200 creates a plurality of simulated drones on the initial exploration map; the thread creating module 1300 creates corresponding threads for each simulated unmanned aerial vehicle respectively; the exploration module 1400 controls each simulated unmanned aerial vehicle to fly according to a corresponding thread, acquires and inputs current exploration data of each simulated unmanned aerial vehicle into a preset mappo algorithm model respectively, generates a two-dimensional exploration map, can realize that a plurality of simulated unmanned aerial vehicles perform multi-thread exploration in the same initial exploration map, and realizes multi-machine collaborative exploration by sharing a public area of exploration, thereby saving exploration time, improving exploration efficiency. The mappo algorithm model is used as a multi-agent deep reinforcement learning algorithm, continuous actions are output, the possibility of being applied to actual scenes is achieved, training is more stable, and boundary values cannot be obtained continuously due to actions caused by the structure of the critic network.
The map creation module 1100 includes:
a random number matrix unit 1110 for creating a random number matrix;
a random point unit 1120, configured to assign values to elements in the random number matrix according to a preset rule, so as to obtain a plurality of random points;
a filter unit 1130 for inputting a plurality of random points into a preset filter to generate a rough map;
and an exploration map creation unit 1140 for creating an exploration layer on the rough map and generating an initial exploration map.
The thread creating module 1300 includes:
a function selecting unit 1310, configured to designate a navigation function as a thread carrying function by using a threading module of python for each unmanned aerial vehicle;
the function execution unit 1320 is configured to execute a thread bearing function to obtain a thread of each drone.
The search module 1400 includes:
the flight control unit 1410 is used for controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread;
an exploration data acquisition unit 1420, configured to acquire current exploration data of each simulated unmanned aerial vehicle;
the judging unit 1430 is configured to respectively judge whether the current exploration area exceeds a preset exploration area according to the current exploration data of each simulated unmanned aerial vehicle, and if so, output the current exploration map, update an actor network and a critic network in a preset mappo algorithm model, and generate a new mappo algorithm model; if not, updating an actor network and a critic network in the preset map algorithm model to generate a new map algorithm model;
a cycle number updating unit 1440 for obtaining the current cycle number, and adding 1 to the current cycle number to obtain a new cycle number;
a cycle number judging unit 1450, configured to judge whether the new cycle number is equal to a preset number, and if so, use the current exploration map as an exploration map; and if not, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map.
Wherein, still include:
and a neural network output module 1500, configured to output an actor network and a critic network in the current mappo algorithm model when the current exploration map is output.
The determining unit 1430 includes:
an incentive extracting subunit 1431, configured to extract incentives from the current exploration data of each simulated unmanned aerial vehicle, respectively;
a comparison subunit 1432, configured to compare the reward with the existing reward sum, and if the reward is greater than the existing reward sum, consider that the current exploration area exceeds the preset exploration area; and if the reward is not greater than the sum of the existing rewards, the current exploration area is considered not to exceed the preset exploration area.
Referring to fig. 6, fig. 6 is a schematic structural block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device comprises a memory 101, a processor 102 and a communication interface 103, wherein the memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The memory 101 may be configured to store software programs and modules, such as program instructions/modules corresponding to the multi-drone multithreading two-dimensional exploration simulation system provided in the embodiment of the present application, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 may be used for communicating signaling or data with other node devices.
The Memory 101 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 102 may be an integrated circuit chip having signal processing capabilities. The Processor 102 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
It will be appreciated that the configuration shown in fig. 6 is merely illustrative and that the electronic device may include more or fewer components than shown in fig. 6 or have a different configuration than shown in fig. 6. The components shown in fig. 6 may be implemented in hardware, software, or a combination thereof.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The above-described functions, if implemented in the form of software functional modules and sold or used as a separate product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above-described method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. A multi-unmanned aerial vehicle multithreading two-dimensional exploration simulation method is characterized by comprising the following steps:
creating an initial exploration map;
creating a plurality of simulated drones on the initial exploration map;
respectively creating corresponding threads for each simulation unmanned aerial vehicle;
and controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and respectively inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model, and generating a two-dimensional exploration map.
2. The multi-drone multithreading two-dimensional exploration simulation method according to claim 1, wherein said step of creating an initial exploration map comprises the steps of:
creating a random number matrix;
assigning values to elements in the random number matrix according to a preset rule to obtain a plurality of random points;
inputting a plurality of random points into a preset filter to generate a rough map;
and creating a search layer on the rough map, and generating an initial search map.
3. The multi-drone multithreading two-dimensional exploration simulation method according to claim 1, wherein said step of creating a corresponding thread for each simulated drone respectively comprises the steps of:
aiming at each unmanned aerial vehicle, respectively adopting a threading module of python to designate a navigation function as a thread bearing function;
and executing the thread bearing function to obtain the thread of each unmanned aerial vehicle.
4. The multi-unmanned-aerial-vehicle multithreading two-dimensional exploration simulation method according to claim 1, wherein the step of controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and respectively inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model, and generating the two-dimensional exploration map comprises the following steps:
controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread;
acquiring current exploration data of each simulated unmanned aerial vehicle;
respectively judging whether the current exploration area exceeds a preset exploration area according to the current exploration data of each simulation unmanned aerial vehicle, if so, outputting a current exploration map, and updating an actor network and a critic network in a preset mappo algorithm model to generate a new mappo algorithm model; if not, updating an actor network and a critic network in the preset map algorithm model to generate a new map algorithm model;
acquiring the current cycle number, and adding 1 to the current cycle number to obtain a new cycle number;
judging whether the new cycle number is equal to a preset screen number or not, and if so, taking the current exploration map as an exploration map; and if not, creating a plurality of simulated unmanned aerial vehicles on the initial exploration map.
5. The multi-UAV multithreading two-dimensional exploration simulation method according to claim 4, further comprising the steps of:
when the current exploration graph is output, the operator network and the critic network in the current mappo algorithm model are output.
6. The multi-unmanned-aerial-vehicle multithreading two-dimensional exploration simulation method according to claim 4, wherein the step of respectively judging whether the current exploration area exceeds the preset exploration area according to the current exploration data of each simulation unmanned aerial vehicle comprises the following steps:
respectively extracting rewards in the current exploration data of each simulated unmanned aerial vehicle;
comparing the reward with the existing reward sum, and if the reward is greater than the existing reward sum, determining that the current exploration area exceeds the preset exploration area; and if the reward is not greater than the sum of the existing rewards, the current exploration area is considered not to exceed the preset exploration area.
7. The utility model provides a many unmanned aerial vehicle multithread two-dimentional exploration simulation system which characterized in that includes:
the map creating module is used for creating an initial exploration map;
the unmanned aerial vehicle creating module is used for creating a plurality of simulated unmanned aerial vehicles on the initial exploration map;
the thread creating module is used for creating corresponding threads for each simulation unmanned aerial vehicle;
and the exploration module is used for controlling each simulation unmanned aerial vehicle to fly according to the corresponding thread, acquiring and respectively inputting the current exploration data of each simulation unmanned aerial vehicle into a preset mappo algorithm model, and generating an exploration map.
8. The multi-drone multithreading two-dimensional exploration simulation system according to claim 7, wherein said map creation module comprises:
a random number matrix unit for creating a random number matrix;
the random point unit is used for assigning values to elements in the random number matrix according to a preset rule to obtain a plurality of random points;
the filter unit is used for inputting a plurality of random points into a preset filter to generate a coarse map;
and a search map creation unit for creating a search layer on the rough map and generating an initial search map.
9. An electronic device, comprising:
a memory for storing one or more programs;
a processor;
the one or more programs, when executed by the processor, implement the method of any of claims 1-6.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111524975.0A CN114020041B (en) | 2021-12-14 | 2021-12-14 | Multi-unmanned aerial vehicle multi-thread two-dimensional exploration simulation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111524975.0A CN114020041B (en) | 2021-12-14 | 2021-12-14 | Multi-unmanned aerial vehicle multi-thread two-dimensional exploration simulation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114020041A true CN114020041A (en) | 2022-02-08 |
| CN114020041B CN114020041B (en) | 2024-02-20 |
Family
ID=80068523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111524975.0A Active CN114020041B (en) | 2021-12-14 | 2021-12-14 | Multi-unmanned aerial vehicle multi-thread two-dimensional exploration simulation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114020041B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011051501A2 (en) * | 2009-11-02 | 2011-05-05 | Eurosimtec Gmbh | Training simulation system for drone systems |
| CN103093047A (en) * | 2013-01-12 | 2013-05-08 | 天津大学 | Typical aircraft visual simulation system |
| CN106444423A (en) * | 2016-09-30 | 2017-02-22 | 天津大学 | Indoor multi unmanned aerial vehicle formation flight simulation verification platform and achieving method thereof |
| CN108845802A (en) * | 2018-05-15 | 2018-11-20 | 天津大学 | Unmanned plane cluster formation interactive simulation verifies system and implementation method |
| CN109739254A (en) * | 2018-11-20 | 2019-05-10 | 国网浙江省电力有限公司信息通信分公司 | Using the unmanned plane and its localization method of visual pattern positioning in a kind of electric inspection process |
| CN110262545A (en) * | 2019-05-30 | 2019-09-20 | 中国南方电网有限责任公司超高压输电公司天生桥局 | A kind of unmanned plane during flying Three-Dimensional Path Planning Method |
| CN110609570A (en) * | 2019-07-23 | 2019-12-24 | 中国南方电网有限责任公司超高压输电公司天生桥局 | Autonomous obstacle avoidance inspection method based on unmanned aerial vehicle |
| CN112991487A (en) * | 2021-03-11 | 2021-06-18 | 中国兵器装备集团自动化研究所有限公司 | System for multithreading real-time construction of orthoimage semantic map |
| CN113608554A (en) * | 2021-09-29 | 2021-11-05 | 中国人民解放军空军预警学院 | Multi-core and multi-thread unmanned aerial vehicle target searching method and device |
-
2021
- 2021-12-14 CN CN202111524975.0A patent/CN114020041B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011051501A2 (en) * | 2009-11-02 | 2011-05-05 | Eurosimtec Gmbh | Training simulation system for drone systems |
| CN103093047A (en) * | 2013-01-12 | 2013-05-08 | 天津大学 | Typical aircraft visual simulation system |
| CN106444423A (en) * | 2016-09-30 | 2017-02-22 | 天津大学 | Indoor multi unmanned aerial vehicle formation flight simulation verification platform and achieving method thereof |
| CN108845802A (en) * | 2018-05-15 | 2018-11-20 | 天津大学 | Unmanned plane cluster formation interactive simulation verifies system and implementation method |
| CN109739254A (en) * | 2018-11-20 | 2019-05-10 | 国网浙江省电力有限公司信息通信分公司 | Using the unmanned plane and its localization method of visual pattern positioning in a kind of electric inspection process |
| CN110262545A (en) * | 2019-05-30 | 2019-09-20 | 中国南方电网有限责任公司超高压输电公司天生桥局 | A kind of unmanned plane during flying Three-Dimensional Path Planning Method |
| CN110609570A (en) * | 2019-07-23 | 2019-12-24 | 中国南方电网有限责任公司超高压输电公司天生桥局 | Autonomous obstacle avoidance inspection method based on unmanned aerial vehicle |
| CN112991487A (en) * | 2021-03-11 | 2021-06-18 | 中国兵器装备集团自动化研究所有限公司 | System for multithreading real-time construction of orthoimage semantic map |
| CN113608554A (en) * | 2021-09-29 | 2021-11-05 | 中国人民解放军空军预警学院 | Multi-core and multi-thread unmanned aerial vehicle target searching method and device |
Non-Patent Citations (1)
| Title |
|---|
| 刘栋;童敏明;路红蕊;: "无人机多机协作探索煤矿灾变环境算法", 计算机应用, no. 08 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114020041B (en) | 2024-02-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112001585B (en) | Multi-agent decision method, device, electronic equipment and storage medium | |
| CN114247141B (en) | Method, device, equipment, medium and program product for guiding tasks in virtual scene | |
| US11235241B2 (en) | Route navigation system within a game application environment | |
| CN111298439B (en) | Data processing method, device, medium and electronic equipment | |
| KR102216749B1 (en) | Method, apparatus and computer program for coloring of a target image | |
| CN109529338A (en) | Object control method, apparatus, Electronic Design and computer-readable medium | |
| CN114241100A (en) | Skinning method, skinning device, skinning apparatus, skinning medium, and program product for virtual object | |
| CN109191590B (en) | Processing system and processing method for manufacturing virtual reality application | |
| Grübel et al. | Fused twins: A cognitive approach to augmented reality media architecture | |
| CN111701246A (en) | Game AI decision configuration method and device | |
| CN114162144A (en) | Automatic driving decision method and device and electronic equipment | |
| CN113975812A (en) | Game image processing method, device, equipment and storage medium | |
| CN114020041B (en) | Multi-unmanned aerial vehicle multi-thread two-dimensional exploration simulation method and system | |
| CN114613159B (en) | Traffic signal lamp control method, device and equipment based on deep reinforcement learning | |
| Cullen et al. | A caching approach to real-time procedural generation of cities from GIS data | |
| Bosse et al. | A cognitive model for visual attention and its application | |
| CN110427241B (en) | Target elimination method and device | |
| Krylov et al. | Automatic hierarchical task network planning system for the Unity game engine | |
| CN111249733A (en) | Method and device for creating virtual object | |
| US20240160888A1 (en) | Realistic, controllable agent simulation using guided trajectories and diffusion models | |
| KR20200068914A (en) | Method and device for optimizing hierarchy of 3d cad model | |
| McQuillan | A survey of behaviour trees and their applications for game AI | |
| CN117786147B (en) | Method and device for displaying data in digital twin model visual field range | |
| CN112861377B (en) | Planning identification method and device under condition of observable environmental part | |
| Michaelsen et al. | Interaction of control and knowledge in a structural recognition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |