CN115951598A - Virtual-real combined simulation method, device and system for multiple unmanned aerial vehicles - Google Patents
Virtual-real combined simulation method, device and system for multiple unmanned aerial vehicles Download PDFInfo
- Publication number
- CN115951598A CN115951598A CN202310068031.XA CN202310068031A CN115951598A CN 115951598 A CN115951598 A CN 115951598A CN 202310068031 A CN202310068031 A CN 202310068031A CN 115951598 A CN115951598 A CN 115951598A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- virtual
- physical
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 81
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 70
- 238000004891 communication Methods 0.000 claims abstract description 53
- 230000003993 interaction Effects 0.000 claims abstract description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000011161 development Methods 0.000 description 8
- 238000012986 modification Methods 0.000 description 7
- 230000004048 modification Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000005094 computer simulation Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 241001515997 Eristalis tenax Species 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000002068 genetic effect Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000002922 simulated annealing Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000002195 synergetic effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention provides a virtual-real combined simulation method, a virtual-real combined simulation device and a virtual-real combined simulation system for multiple unmanned aerial vehicles, wherein the method comprises the following steps: building a virtual scene based on the real scene characteristics; determining the number of physical unmanned aerial vehicles and the number of virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster; at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of all unmanned aerial vehicles based on the number of the virtual scene, the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles; constructing a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle, wherein the communication control system is used for carrying out data interaction between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle; determining state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on a communication control system; based on the communication control system, the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle are respectively controlled by combining state information, formation and flight channels of all unmanned aerial vehicles. The virtual-real combined simulation method of the multiple unmanned aerial vehicles can combine a physical unmanned aerial vehicle and a virtual unmanned aerial vehicle to realize real virtual-real combined simulation.
Description
Technical Field
The embodiment of the invention relates to the technical field of virtual-real simulation of computers, in particular to a virtual-real combined simulation method, device and system of multiple unmanned aerial vehicles.
Background
The unmanned aerial vehicle simulation is a loop which is necessary to pass actions such as designing an unmanned aerial vehicle, verifying a relevant algorithm of the unmanned aerial vehicle, and conceiving a combat style of the unmanned aerial vehicle. The current unmanned aerial vehicle simulation has played irreplaceable effect in fields such as unmanned aerial vehicle dynamics, unmanned aerial vehicle obstacle avoidance, unmanned aerial vehicle cluster mission planning and the like. The virtual-real combined simulation is a simulation method which is developed rapidly in recent years, and has good effects on weapon efficiency evaluation, combat equipment test, industrial production assembly and the like. The virtual-real combined simulation method combines elements in a virtual space of a computer with elements in the real world, and plays a role in cross-space resource allocation and cross-time environment calling. The virtual-real combined simulation can supplement the deficiency of real objects by elements in the virtual environment under the limited real object condition, for example, the real object unmanned aerial vehicle is combined with the virtual sensor, so that the real object unmanned aerial vehicle simulates in the virtual environment, and the same simulation effect as in the actual environment is achieved.
Simulation is a forerunner in equipment development and tactical technical training. And the unmanned aerial vehicle simulation is a powerful tool for unmanned aerial vehicle algorithm verification and tactical research. With the development of various intelligent algorithms such as genetic algorithm, simulated annealing algorithm, particle swarm algorithm, ant colony algorithm and the like, the problem related to the unmanned aerial vehicle is solved, and an improved intelligent algorithm is gradually adopted. In order to verify the effectiveness of the algorithms, a pure physical system has many limitations, and firstly, the number of the unmanned aerial vehicles is limited, and a common laboratory cannot use dozens of hundreds of unmanned aerial vehicles for algorithm verification. Second, the energy is restricted, and unmanned aerial vehicle's energy often can only support its time of staying empty for several hours, can't carry out long-time or simulation many times, need recharge and deploy the system, and the process is very loaded down with trivial details. Thirdly, as the verified algorithm often fails to succeed at one time, damage to the unmanned aerial vehicle can occur in the process, resulting in failure of subsequent verification.
At present, the feasibility of the algorithm is usually verified in a computer simulation mode, but the computer simulation is limited by an ideal environment, so that the verified algorithm cannot be directly deployed on a physical unmanned aerial vehicle, adjustment needs to be carried out according to actual test conditions, and the algorithm development period is prolonged. The virtual-real combination simulation is a simulation technology which is generated by the fact that full physical simulation is difficult to achieve, and the reliability of pure virtual simulation cannot meet the requirement, the virtual-real combination is a combination mode of finishing the virtual simulation of a system and the exchange and interconnection of actual equipment to be tested, the mode can play the characteristics of interface abstraction, functional simulation and test function iterative development of the virtual system simulation, and meanwhile, the authenticity of the actual equipment is combined, so that the test meets the requirements of comprehensiveness, high efficiency and reliability. The simulation mode combining virtuality and reality fully transfers digital resources and physical resources in the simulation system, so that the original development process such as design, modeling, simulation and verification is linearly carried out, the development process can be carried out in parallel in the simulation and verification stages, and the development efficiency is effectively improved.
The key point of constructing the virtual-real combined simulation system is to realize interconnection and intercommunication of virtual space and real space information, and on the basis, unified allocation of two space resources can be realized. But the present scheme all can't accomplish to fuse virtual unmanned aerial vehicle and unmanned aerial vehicle in kind, realizes real virtuality and reality and combines the emulation.
Disclosure of Invention
The invention provides a virtual-real combined simulation method, device and system for multiple unmanned aerial vehicles, which can combine a physical unmanned aerial vehicle with a virtual unmanned aerial vehicle and realize real virtual-real combined simulation.
In order to solve the technical problem, an embodiment of the present invention provides a virtual-real combined simulation method for multiple unmanned aerial vehicles, including:
determining the scene characteristics of a real object;
building a virtual scene based on the physical scene characteristics;
determining the number of physical unmanned aerial vehicles and the number of virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster;
at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the virtual scene, the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles;
constructing a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle, wherein the communication control system is used for carrying out data interaction between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle;
determining state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the communication control system;
and based on the communication control system, the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle are respectively controlled by combining the state information, the formation and the flight channels of the unmanned aerial vehicles.
As an optional embodiment, the determining the number of physical drones and the number of virtual drones in the drone cluster includes:
determining the number of physical unmanned aerial vehicles in the unmanned aerial vehicle cluster;
respectively constructing twin bodies corresponding to the real unmanned aerial vehicles in the virtual scene;
determining a number of the virtual drones and the twins.
As an optional embodiment, the determining, based on the virtual scene, the number of physical unmanned aerial vehicles, and the number of virtual unmanned aerial vehicles, at least a formation of a cluster of unmanned aerial vehicles in the virtual scene and a flight channel of each of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles includes:
determining a target formation form and a target flight channel of the unmanned aerial vehicle cluster;
in the virtual scene, determining a waypoint at each moment for each of the real unmanned aerial vehicles and the virtual unmanned aerial vehicles respectively according to a time sequence based on the number of the real unmanned aerial vehicles, the number of the virtual unmanned aerial vehicles and a target flight channel;
and determining flight channels of the formation, the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the waypoints of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles.
As an optional embodiment, the constructing a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle includes:
setting a positioning system, wherein the positioning system is used for identifying and positioning the identity of the physical unmanned aerial vehicle;
setting a service computer, wherein the service computer is in communication connection with a positioning system, is used for processing feedback information of the positioning system to form three-dimensional information corresponding to the virtual scene, and is at least used for calculating and determining the expected flight position of the physical unmanned aerial vehicle by combining an intelligent algorithm;
and a control computer is arranged, is in communication connection with the service computer, the positioning system and the physical unmanned aerial vehicle, and is at least used for receiving the expected flight position of the service computer, generating a corresponding control instruction, sending the control instruction to the physical unmanned aerial vehicle, obtaining related information of the physical unmanned aerial vehicle and forwarding the information to the service computer based on a UDP protocol.
As an optional embodiment, the positioning system includes a GPS module disposed on the physical unmanned aerial vehicle, a plurality of cameras disposed in the physical scene, and a plurality of reflective balls disposed on the physical unmanned aerial vehicle, the control computer is connected to the GPS module, the service computer is connected to the cameras to obtain the feedback information, and can send the feedback information to the control computer.
As an optional embodiment, the number and/or positions of the plurality of light-reflecting balls on the different physical unmanned aerial vehicles are different;
the service unmanned aerial vehicle or the control unmanned aerial vehicle can determine the ID of each real unmanned aerial vehicle and the position state of the real unmanned aerial vehicle in a real scene based on the set number and/or the set position of the plurality of reflective balls in the feedback information of the camera.
As an optional embodiment, the controlling the flight of the virtual drone and the real drone based on the communication control system by combining the state information, the formation and the flight channels of the drones respectively includes:
respectively determining formation positions and flight channels of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles based on the service computer;
determining a first coordinate threshold value of whether the real object unmanned aerial vehicle enters the flight channel for the first time based on the service computer and the current state information of the real object unmanned aerial vehicle, wherein the first coordinate threshold value is a coordinate threshold value corresponding to any one target position point in the flight channel, and the target position point corresponds to the expected flight position;
if so, the service computer sends the state information of the physical unmanned aerial vehicle to a target virtual unmanned aerial vehicle or a target physical unmanned aerial vehicle behind the physical unmanned aerial vehicle in a formation, so that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to a matched target position point based on respective flight channels, and feeds back the state information to the service unmanned aerial vehicle;
and when the service computer determines that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to the matched target position point based on the state information of the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle, informing the control computer to enable the control computer to respectively generate and send instructions for flying to the next target position point for the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the current state information, the formation and the respective flight channels of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle.
Another embodiment of the present invention provides a virtual-real combined simulation apparatus for multiple drones, including:
the first determining module is used for determining the real object scene characteristics;
the building module is used for building a virtual scene based on the real object scene characteristics;
the second determining module is used for determining the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster;
the third determining module is used for at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the virtual scene, the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles;
the building module is used for building a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle, and the communication control system is used for carrying out data interaction between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle;
the fourth determination module is used for determining the state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the communication control system;
and the control module is used for controlling the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle respectively by combining the state information, the formation and the flight channels of the unmanned aerial vehicles based on the communication control system.
As an optional embodiment, the determining the number of physical drones and the number of virtual drones in the drone cluster includes:
determining the number of physical unmanned aerial vehicles in the unmanned aerial vehicle cluster;
twin bodies corresponding to the real unmanned aerial vehicles are respectively constructed in the virtual scene;
determining the number of the virtual drones and the twins.
Another embodiment of the present invention further provides a virtual-real combined simulation system for multiple drones, including:
one or more processors;
a memory configured to store one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a virtual-real join simulation method for multiple drones as described in any of the embodiments above.
Another embodiment of the present invention further provides a storage medium, on which a computer program is stored, where the program, when executed by a processor, implements the virtual-real combining simulation method for multiple drones as described in any one of the above embodiments.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present application is further described in detail by the accompanying drawings and examples.
Drawings
Fig. 1 is a flowchart of a virtual-real combined simulation method for multiple drones in the embodiment of the present invention.
Fig. 2 is a flowchart of a formation method of multiple drones in the embodiment of the present invention.
Fig. 3 is a process diagram of multiple drones performing tasks in the embodiment of the present invention.
Fig. 4 is a structural relationship diagram of a communication control system in the embodiment of the present invention.
Fig. 5 is a communication flow chart between multiple drones in the embodiment of the present invention.
Fig. 6 is a block diagram of a virtual-real combined simulation apparatus of multiple drones in the embodiment of the present invention.
Detailed Description
Specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention is not limited thereto.
It will be understood that various modifications may be made to the embodiments disclosed herein. The following description is, therefore, not to be taken in a limiting sense, and is made merely as an exemplification of embodiments. Other modifications will occur to those skilled in the art within the scope and spirit of the disclosure.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the disclosure and, together with a general description of the disclosure given above, and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.
These and other characteristics of the invention will become apparent from the following description of a preferred form of embodiment, given as a non-limiting example, with reference to the accompanying drawings.
It should also be understood that, although the invention has been described with reference to some specific examples, a person of skill in the art shall certainly be able to achieve many other equivalent forms of the invention, having the characteristics as set forth in the claims and hence all coming within the field of protection defined thereby.
The above and other aspects, features and advantages of the present disclosure will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
Specific embodiments of the present disclosure are described hereinafter with reference to the accompanying drawings; however, it is to be understood that the disclosed embodiments are merely examples of the disclosure that may be embodied in various forms. Well-known and/or repeated functions and structures have not been described in detail so as not to obscure the present disclosure with unnecessary or unnecessary detail. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present disclosure in virtually any appropriately detailed structure.
The specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may each refer to one or more of the same or different embodiments in accordance with the disclosure.
The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
As shown in fig. 1, an embodiment of the present invention provides a virtual-real combined simulation method for multiple drones, including:
s101: determining the scene characteristics of the real object;
s102: building a virtual scene based on the real scene characteristics;
s103: determining the number of physical unmanned aerial vehicles and the number of virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster;
s104: at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the virtual scene, the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles;
s105: constructing a communication control system between the real unmanned aerial vehicle and the virtual unmanned aerial vehicle, wherein the communication control system is used for carrying out data interaction between the real unmanned aerial vehicle and the virtual unmanned aerial vehicle;
s106: determining state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on a communication control system;
s107: based on the communication control system, the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle are respectively controlled by combining state information, formation and flight channels of all unmanned aerial vehicles.
Based on the method of the embodiment, the fusion of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle can be effectively realized, the virtual-real combined simulation of the unmanned aerial vehicle is really realized, and the method can be applied to the military field, particularly in urban combat operations, and the unmanned aerial vehicle can pass through the interior of a building by virtue of the characteristics of small size and flexibility to complete reconnaissance tasks. The method of combining virtuality and reality can simulate an actual action scene based on the cooperation of a small number of real unmanned aerial vehicles and virtual unmanned aerial vehicles, lays a foundation for actual application, and provides effective reference. Especially in practical application, unmanned aerial vehicle can carry out the reconnaissance in the building, and the inside most of building couples together each room through door, window etc.. Therefore, it is the requisite that unmanned aerial vehicle carries out the reconnaissance mission to pass through door and window smoothly, but door and window has the shape characteristics that the space is narrow, if fail to carry out unmanned aerial vehicle's flight training in advance, the processing of flight data leads to reconnaissance mission failure very easily, especially when there are many unmanned aerial vehicles to carry out the reconnaissance mission, often need pass through door and window through the formation smoothly, so train unmanned aerial vehicle based on the method in this embodiment in advance, utilize virtual reality to combine the method simulation actual action scene can effectively assist going on smoothly of actual reconnaissance action.
Furthermore, when the virtual scene is built based on the real object scene, the condition that the boundary of the virtual scene obeys the real object scene is met. The layout of the virtual scene and the physical scene may be determined by combining the content of the simulation service, for example, the physical entity synchronizes the motion of the virtual entity in the virtual space, and when verifying the performance in the physical scene, the physical scene needs to be consistent with the virtual scene, including the size and the position of the terrain and the obstacle. In this embodiment, the virtual scene may provide a corresponding perception capability, such as a collision perception capability, in addition to the environment simulating the motion of the object. When the deviation occurs when the unmanned aerial vehicle executes the movement, or the algorithm design is wrong, so that the virtual unmanned aerial vehicle collides with the environment, the virtual scene can feed the collision information back to a control computer (described below) in real time, and meanwhile, the collision effect is rendered in the virtual scene in real time, so that the state which accords with the cognition of the user shows the collision state for the user.
Further, confirm the unmanned aerial vehicle quantity in kind and the virtual unmanned aerial vehicle quantity in the unmanned aerial vehicle cluster, include:
s108: determining the number of physical unmanned aerial vehicles in the unmanned aerial vehicle cluster;
s109: respectively constructing twin bodies corresponding to each real unmanned aerial vehicle in a virtual scene;
s110: determining the number of virtual drones and twins.
That is, the unmanned aerial vehicle cluster is formed by the cooperation of part real unmanned aerial vehicle and part virtual unmanned aerial vehicle, and wherein, virtual unmanned aerial vehicle is based on virtual scene modeling formation, and real unmanned aerial vehicle need establish twin in the virtual scene in order to realize forming a team with virtual unmanned aerial vehicle to realize synchronous with real unmanned aerial vehicle's state based on this twin. When determining the number of drone clusters, the number of virtual drones and twins is determined. Specifically, when the virtual unmanned aerial vehicle is applied, a virtual scene, a twin and simulation software can be built by using the phantom engine, and similarly, the virtual unmanned aerial vehicle can also be created by using a model in the phantom engine. And the entity in kind can use four rotor unmanned aerial vehicle, and it has small in size, long, the sensitive advantage of reaction of dead time. When a twin body is constructed, the physical unmanned aerial vehicle can be guided into the illusion engine after being modeled by using three-dimensional modeling software, and then the virtual unmanned aerial vehicle is formed.
Further, at least, formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of each virtual unmanned aerial vehicle and each physical unmanned aerial vehicle are determined based on the virtual scene, the number of physical unmanned aerial vehicles and the number of virtual unmanned aerial vehicles, including:
s111: determining a target formation form and a target flight channel of the unmanned aerial vehicle cluster;
s112: in a virtual scene, determining a waypoint at each moment for each real unmanned aerial vehicle and each virtual unmanned aerial vehicle respectively according to a time sequence based on the number of the real unmanned aerial vehicles, the number of the virtual unmanned aerial vehicles and a target flight channel;
s113: and determining formation and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the waypoints of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles.
For example, the drone cluster performing formation to drone cluster formation requires that a single drone fly at a relatively fixed position with other drones within the formation to form a designated formation, i.e., a target formation, in accordance with the formation coordination algorithm. The unmanned aerial vehicle formation method based on the waypoints is suitable for unmanned aerial vehicle clusters with small scales, the formation control of the unmanned aerial vehicles is realized by continuously updating the waypoints of all the unmanned aerial vehicles in the formation at the next moment through the control computer, namely the formation is determined based on the waypoints of the physical unmanned vehicles and the virtual unmanned vehicles, and meanwhile, the flight channels of all the virtual unmanned vehicles and the physical unmanned vehicles can be determined based on the waypoints at each moment. The algorithm flow for determining formation according to the waypoint at each time can be referred to fig. 2, wherein the = sign in fig. 2 represents the assignment.
Furthermore, the path tracking algorithm of the unmanned aerial vehicle cluster is different from the algorithm for the unmanned aerial vehicle cluster to execute formation and formation transformation, and the unmanned aerial vehicle needs to go to a target position point under the condition of formation constraint, so that the target formation is obtained, namely a target flight path (namely a target flight channel), and the flight path can be given by the path planning algorithm according to the formation of the unmanned aerial vehicle cluster and the position of the target position point. For the formation mode, a pilot-follower model can be adopted, namely the position of a pilot is given, and the position of a follower can be obtained by calculating the formation position. And the destination of the path tracking is also the destination of the pilot, and the positions of the rest unmanned aerial vehicles in the formation are calculated according to the relative position relationship in the formation.
For example, as shown in fig. 3, it shows the whole task flow and the formation switching manner of the unmanned aerial vehicle cluster, and specifically includes that after receiving the instruction, the physical unmanned aerial vehicle and the two virtual unmanned aerial vehicles take off in the order of the physical unmanned aerial vehicle, the first virtual unmanned aerial vehicle, and the second virtual unmanned aerial vehicle, the hovering position of the physical unmanned aerial vehicle is (0.0, 1.5, 1.0), the hovering position of the first virtual unmanned aerial vehicle is (1.0, 5.0, 1.5), and the hovering position of the second virtual unmanned aerial vehicle is (-1.0, 5.0, 1.5). After the second virtual unmanned aerial vehicle reaches the designated position, the three unmanned aerial vehicles form a formation to fly forwards. And the instructions of formation transformation are executed after the physical unmanned aerial vehicle arrives (0.0, 1.0 and 1.0), the first virtual unmanned aerial vehicle arrives (1.0, 3.0 and 1.5) and the second virtual unmanned aerial vehicle arrives (1.0, 3.0 and 1.5). The lower ends of the door to be passed through are located at (0.5, 0.0) (-0.5, 0.0) and have a height of 1.8 m. The physical unmanned aerial vehicle firstly goes to (0.0, -1.5, 1.0), the first virtual unmanned aerial vehicle reaches a target point along the paths of (1.0, 3.0, 1.5), (0.0, -1.0, 1.5), the second virtual unmanned aerial vehicle reaches the target point along the paths of (-1.0, 3.0, 1.5), (0.0, -0.5, 1.5), and then the three unmanned aerial vehicles execute a landing instruction.
Furthermore, the most basic requirement of the virtual-real combined simulation system on performance indexes is simulation instantaneity, the instantaneity requires that the operation of the simulation system meets a certain time constraint condition, if the instantaneity of the simulation system is not guaranteed, the operation of the simulation system enters an uncertain state, the behavior of the system is uncontrollable, and the operation result of the system is unpredictable, so that the credibility of the simulation result is difficult to guarantee.
Therefore, the real-time requirements of the virtual-real combined simulation system are mainly as follows:
(1) Time t of response of simulation system to external instruction o ；
(2) Time t for mutual synchronization of virtual entity state and physical entity state r ；
The virtual-real combined simulation is a human-in-loop simulation system, and in the running process of the simulation system, a human gives external instructions to the simulation system, such as instructions for sending queue form change, task change, firepower striking and the like, and the simulation system needs to respond to the instructions in time. In response to an external command, a preset waiting time t is usually set w If t is o Exceeds the preset waiting time t w If no response is received, the sender of the instruction in the emulation system is assumed to enter an indeterminate state, which may result in the emulation results being untrustworthy. Therefore, to improve the reliability of the system for responding to the external instruction, the real-time performance of the simulation system for responding to the external instruction should be ensured. Namely, the following conditions are satisfied: t is t o <t w
In the virtual-real combined simulation, the virtual entity and the physical entity need to acquire the state of the other party in real time to realize the virtual-real interaction. For example, when the virtual unmanned aerial vehicle and the real unmanned aerial vehicle are formed in a pilot-follower formation mode, the virtual unmanned aerial vehicle and the real unmanned aerial vehicle need to acquire the position of the opposite side in real time, and the position of the virtual unmanned aerial vehicle and the real unmanned aerial vehicle can be acquired according to the relative position relation of the preset formation. Otherwise, the virtual unmanned aerial vehicle and the physical unmanned aerial vehicle enter a state with uncertain positions, and further the simulation result is influenced. The time when the virtual entity state and the physical entity state are synchronized with each other refers to the time when one of the virtual entity and the physical entity sends the state information of the other entity, and the other entity receives the information and completes parameter binding. This time should be as small as possible to ensure that the drone has sufficient time to respond to changes in flight parameters to avoid disjointing the state of the virtual entity from the physical entity.
Further, when constructing the communication control system between real object unmanned aerial vehicle and virtual unmanned aerial vehicle in this embodiment, include:
s114: a positioning system is arranged and used for identifying and positioning the identity of the real object unmanned aerial vehicle;
s115: setting a service computer, wherein the service computer is in communication connection with the positioning system, is used for processing feedback information of the positioning system to form three-dimensional information corresponding to the virtual scene, and is at least used for calculating and determining the expected flight position of the physical unmanned aerial vehicle by combining with an intelligent algorithm;
s116: and a control computer is arranged, is in communication connection with the service computer, the positioning system and the physical unmanned aerial vehicle, and is at least used for receiving the expected flight position of the service computer, generating a corresponding control command, sending the control command to the physical unmanned aerial vehicle, acquiring related information of the physical unmanned aerial vehicle and forwarding the information to the service computer based on a UDP (user Datagram protocol) protocol.
The positioning system in this embodiment is including setting up the GPS module on unmanned aerial vehicle in kind to and lay a plurality of cameras in the scene in kind and set up a plurality of anti-light ball on unmanned aerial vehicle in kind, and control computer links to each other with the GPS module, and the service computer links to each other in order to obtain feedback information with the camera, and can send feedback information to control computer.
The number and/or the positions of the plurality of light reflecting balls on the unmanned aerial vehicles in different real objects are different;
the service unmanned aerial vehicle or the control unmanned aerial vehicle can determine the ID of each real unmanned aerial vehicle and the position state of each real unmanned aerial vehicle in the real scene based on the set number and/or the set position of a plurality of reflective balls in the feedback information of the camera.
Specifically, the boundary has been drawn for unmanned aerial vehicle movable real object scene to the laying of positioning system, and real object's positioning system in this embodiment uses indoor action capture system, through the anti-light ball that lays on the unmanned aerial vehicle of real object to and lay and carry out real object's position and attitude information acquisition in 12 cameras around the place, and its indoor positioning accuracy can reach millimeter level. The number and/or positions of the reflective balls arranged on different physical unmanned aerial vehicles are not fixed, and the ID of the unmanned aerial vehicle can be determined based on the reflective balls with different numbers and/or positions. Further, the simulation system in this embodiment is constructed by using an ROS and a DDS for communication, and simultaneously, a Mavlink (small unmanned vehicle communication protocol) is used as a communication protocol between the control computer and the unmanned aerial vehicle, so as to connect the service computer, the control computer, the indoor positioning system, and the unmanned aerial vehicle. The specific structure is shown in fig. 4, in which the arrows represent the information flow direction.
Preferably, compare and connect in WIFI, communication speed, bandwidth and the stability of wired connection are all better, therefore, in this embodiment, preferably adopt wired connection's mode to access to local area network between unmovable control computer and the service computer, unmanned aerial vehicle uses WIFI and router to access to local area network.
In this embodiment, the control computer mainly implements the following functions:
(1) And acquiring an operation instruction of the physical unmanned aerial vehicle from the service computer, converting the operation instruction into a Mallink (small unmanned vehicle communication protocol) message and sending the Mallink message to the physical unmanned aerial vehicle.
(2) The unmanned aerial vehicle in kind acquires self position according to the GPS module of self, sends for control computer through WIFI, and control computer compares the evidencing from the position message of the information source among the positioning system, confirms unmanned aerial vehicle in kind's state finally.
(3) And the control computer operates the python script, unifies the information related to the information of the physical unmanned aerial vehicle, and sends the information to the service computer through a UDP (user Datagram protocol) protocol, so that the state data acquisition of the physical unmanned aerial vehicle integrating the positioning system and the GPS (global positioning system) data of the physical unmanned aerial vehicle is realized.
The service computer comprises positioning system software, a phantom engine and an intelligent algorithm, and mainly realizes the following functions:
(1) And running positioning system software, acquiring and displaying the motion state data and the three-dimensional space position of the real object unmanned aerial vehicle on an interface in real time, acquiring the running track of each unmanned aerial vehicle within a certain time, and forwarding the position information from the positioning system to the control computer by using the VRPN.
(2) The phantom engine communicates with the indoor positioning system using DDS in the ROS2 environment. And the illusion engine sends the calculation result of the algorithm to the control computer through a UDP protocol, and the control computer generates a Mallink instruction and then sends the Mallink instruction to the corresponding unmanned aerial vehicle so as to control the unmanned aerial vehicle to move.
(3) The simulation platform in the embodiment uses Python as a development language of the intelligent algorithm, and calls related resources in the virtual engine and controls the unmanned aerial vehicle to complete simulation service by running a Python script.
The positioning system determines the ID and the state information of each real unmanned aerial vehicle through the cameras arranged on the periphery of the real scene and the reflective balls distributed on the real unmanned aerial vehicle, wherein the state information comprises position information, information about whether a target position point flight task is completed or not, and the like.
Further, in the virtual-real combined simulation, when the real entity and the virtual entity participate simultaneously, the state synchronization of the real entity and the virtual entity is an important guarantee for achieving the synergistic effect. After the physical unmanned aerial vehicle reaches the target position point, the virtual unmanned aerial vehicle needs to be informed of the information that the physical unmanned aerial vehicle reaches the designated position, and after the virtual unmanned aerial vehicle acquires the information, the virtual unmanned aerial vehicle executes a next step of instruction corresponding to the physical unmanned aerial vehicle.
When simulation is carried out in a pure virtual environment, various simulation conditions are very ideal, and the advantages of the ideal conditions are particularly remarkable in the aspect of communication of a positioning system, a controller and information of an unmanned aerial vehicle. The advantages of the sensor and the controller enable the state synchronization method of the virtual simulation to be simple, and information can be transmitted to other objects only through a simple calling mode after the objects execute specified commands. Meanwhile, the computer simulation related application or software development of the unmanned aerial vehicle is mature, command execution and result confirmation of the virtual unmanned aerial vehicle are packaged completely, and the user only needs to call the function to complete the steps. After the real-object unmanned aerial vehicle is introduced, the real-time position of the unmanned aerial vehicle needs to be read or acquired through data in the DDS, and the links of command execution and result confirmation of the unmanned aerial vehicle are designed by self so as to realize state synchronization between the real-object entity and the virtual entity.
Physical drones have sensor and drone's controller performance limitations in the positioning system that can cause physical drones to shake near the target location point. Therefore, a threshold interval (i.e. the first coordinate threshold of the present application) needs to be designed near the target point, and when the drone enters the threshold interval, that is, it is determined that the drone has reached the target point, the threshold needs to be determined according to the geographic scale of the task executed by the drone and the performance of the sensor, the controller, and the positioning system of the drone itself.
Due to the existence of the threshold interval, the unmanned aerial vehicle always sends out the information of reaching the target point when being in the threshold interval. This may cause the unmanned aerial vehicle to send information that it reaches the previous target point to other entities even if the unmanned aerial vehicle has executed the instruction and moves to the next target point. Therefore, the states of the unmanned aerial vehicle can be misjudged by other entities, and the interlocking of each simulation entity is further caused. In order to solve the above problem, in this embodiment, according to the characteristics of the real unmanned aerial vehicle and the virtual unmanned aerial vehicle and the communication structure of the simulation system, after the real unmanned aerial vehicle reaches a predetermined coordinate point, the real unmanned aerial vehicle simultaneously sends an instruction to the two virtual unmanned aerial vehicles and receives feedback results of the two virtual unmanned aerial vehicles.
For example, as shown in fig. 5, based on the communication control system, the method for controlling the flight of the virtual unmanned aerial vehicle and the physical unmanned aerial vehicle respectively in combination with the state information, the formation and the flight channels of the unmanned aerial vehicles includes:
s101: respectively determining formation positions and flight channels of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles based on a service computer;
s117: determining whether the real object unmanned aerial vehicle firstly enters a first coordinate threshold value of the flight channel based on the service computer and the current state information of the real object unmanned aerial vehicle, wherein the first coordinate threshold value is a coordinate threshold value corresponding to any one target position point in the flight channel, and the target position point corresponds to an expected flight position;
s118: if so, the service computer sends the state information of the physical unmanned aerial vehicle to a target virtual unmanned aerial vehicle or a target physical unmanned aerial vehicle behind the physical unmanned aerial vehicle in a formation, so that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to a matched target position point based on respective flight channels, and feeds back the state information to the service unmanned aerial vehicle;
s119: and when the service computer determines that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to the matched target position point based on the state information of the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle, the service computer informs the control computer to enable the control computer to respectively generate and send instructions for the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle to fly to the next target position point based on the current state information, formation and respective flight channels of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle.
As shown in fig. 6, an embodiment of the present invention provides a virtual-real combined simulation apparatus 100 for multiple drones at the same time, including:
the first determining module is used for determining the real object scene characteristics;
the building module is used for building a virtual scene based on the physical scene characteristics;
the second determining module is used for determining the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster;
the third determining module is used for at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the virtual scene, the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles;
the building module is used for building a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle, and the communication control system is used for carrying out data interaction between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle;
the fourth determination module is used for determining the state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the communication control system;
and the control module is used for controlling the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle respectively by combining the state information, the formation and the flight channels of the unmanned aerial vehicles based on the communication control system.
As an optional embodiment, the determining the number of physical drones and the number of virtual drones in the drone cluster includes:
determining the number of physical unmanned aerial vehicles in the unmanned aerial vehicle cluster;
respectively constructing twin bodies corresponding to the real unmanned aerial vehicles in the virtual scene;
determining a number of the virtual drones and the twins.
As an optional embodiment, the determining, based on the virtual scene, the number of physical unmanned aerial vehicles, and the number of virtual unmanned aerial vehicles, at least a formation of a cluster of unmanned aerial vehicles in the virtual scene and a flight channel of each of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles includes:
determining a target formation form and a target flight channel of the unmanned aerial vehicle cluster;
in the virtual scene, respectively determining waypoints at all times for each physical unmanned aerial vehicle and each virtual unmanned aerial vehicle according to a time sequence based on the number of physical unmanned aerial vehicles, the number of virtual unmanned aerial vehicles and a target flight channel;
and determining flight channels of the formation, the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the waypoints of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles.
As an optional embodiment, the constructing a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle includes:
setting a positioning system, wherein the positioning system is used for carrying out identity recognition and positioning on the physical unmanned aerial vehicle;
setting a service computer, wherein the service computer is in communication connection with a positioning system, is used for processing feedback information of the positioning system to form three-dimensional information corresponding to the virtual scene, and is at least used for calculating and determining the expected flight position of the physical unmanned aerial vehicle by combining an intelligent algorithm;
and a control computer is arranged, is in communication connection with the service computer, the positioning system and the physical unmanned aerial vehicle, and is at least used for receiving the expected flight position of the service computer, generating a corresponding control command, sending the control command to the physical unmanned aerial vehicle, obtaining related information of the physical unmanned aerial vehicle and forwarding the related information to the service computer based on a UDP protocol.
As an optional embodiment, the positioning system includes a GPS module disposed on the physical unmanned aerial vehicle, a plurality of cameras disposed in the physical scene, and a plurality of reflective balls disposed on the physical unmanned aerial vehicle, the control computer is connected to the GPS module, the service computer is connected to the cameras to obtain the feedback information, and can send the feedback information to the control computer.
As an optional embodiment, the number and/or positions of the plurality of light-reflecting balls on the real object unmanned aerial vehicle are different;
the service unmanned aerial vehicle or the control unmanned aerial vehicle can determine the ID of each real unmanned aerial vehicle and the position state of the real unmanned aerial vehicle in a real scene based on the set number and/or the set position of the plurality of reflective balls in the feedback information of the camera.
As an optional embodiment, the controlling, based on the communication control system, the virtual unmanned aerial vehicle and the physical unmanned aerial vehicle to fly in combination with the state information, the formation, and the flight channels of the unmanned aerial vehicles respectively includes:
respectively determining formation positions and flight channels of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles based on the service computer;
determining a first coordinate threshold value of whether the real object unmanned aerial vehicle enters the flight channel for the first time based on the service computer and the current state information of the real object unmanned aerial vehicle, wherein the first coordinate threshold value is a coordinate threshold value corresponding to any one target position point in the flight channel, and the target position point corresponds to the expected flight position;
if so, the service computer sends the state information of the physical unmanned aerial vehicle to a target virtual unmanned aerial vehicle or a target physical unmanned aerial vehicle behind the physical unmanned aerial vehicle in a formation, so that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to a matched target position point based on respective flight channels, and feeds back the state information to the service unmanned aerial vehicle;
and when the service computer determines that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to the matched target position point based on the state information of the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle, informing the control computer to enable the control computer to respectively generate and send instructions for flying to the next target position point for the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the current state information, the formation and the respective flight channels of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle.
That is, the real unmanned aerial vehicle determines that the corresponding target position point is reached and then notifies the virtual unmanned aerial vehicle, so that the virtual unmanned aerial vehicle executes the moving instruction, reaches the corresponding target position point, and then feeds back the moving instruction to the real unmanned aerial vehicle.
Another embodiment of the present invention further provides a virtual-real combined simulation system for multiple drones, including:
one or more processors;
a memory configured to store one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the virtual-real combined simulation method of the multiple drones.
Further, an embodiment of the present invention further provides a storage medium, on which a computer program is stored, where the program, when executed by a processor, implements the virtual-real combining simulation method for multiple drones as described above. It should be understood that each solution in this embodiment has a corresponding technical effect in the foregoing method embodiments, and details are not described here.
Further, embodiments of the present invention also provide a computer program product, tangibly stored on a computer-readable medium and comprising computer-executable instructions that, when executed, cause at least one processor to perform a virtual-real join simulation method, such as the multi-drone in the embodiments described above.
Note that the computer storage media of the present application can be either computer readable signal media or computer readable storage media or any combination of the two. The computer readable medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access storage media (RAM), a read-only storage media (ROM), an erasable programmable read-only storage media (EPROM or flash memory), an optical fiber, a portable compact disc read-only storage media (CD-ROM), an optical storage media piece, a magnetic storage media piece, or any suitable combination of the foregoing. In the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, or device. In this application, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, antenna, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
In addition, as will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
The above embodiments are only exemplary embodiments of the present invention, and are not intended to limit the present invention, and the scope of the present invention is defined by the claims. Various modifications and equivalents may be made by those skilled in the art within the spirit and scope of the present invention, and such modifications and equivalents should also be considered as falling within the scope of the present invention.
Claims (10)
1. The utility model provides a virtual reality of many unmanned aerial vehicles combines simulation method which characterized in that includes:
determining the scene characteristics of the real object;
building a virtual scene based on the physical scene features;
determining the number of physical unmanned aerial vehicles and the number of virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster;
at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the virtual scene, the number of physical unmanned aerial vehicles and the number of virtual unmanned aerial vehicles;
constructing a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle, wherein the communication control system is used for carrying out data interaction between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle;
determining state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the communication control system;
and based on the communication control system, the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle are respectively controlled by combining the state information, the formation and the flight channels of the unmanned aerial vehicles.
2. The virtual-real combined simulation method for multiple drones according to claim 1, wherein the determining the number of physical drones and the number of virtual drones in the drone cluster includes:
determining the number of physical unmanned aerial vehicles in the unmanned aerial vehicle cluster;
respectively constructing twin bodies corresponding to the real unmanned aerial vehicles in the virtual scene;
determining a number of the virtual drones and the twins.
3. The virtual-real combined simulation method of multiple drones according to claim 1, wherein the determining at least the formation of the drone cluster in the virtual scene and the flight channels of the virtual drones and the physical drones based on the virtual scene, the number of physical drones, and the number of virtual drones comprises:
determining a target formation form and a target flight channel of the unmanned aerial vehicle cluster;
in the virtual scene, respectively determining waypoints at all times for each physical unmanned aerial vehicle and each virtual unmanned aerial vehicle according to a time sequence based on the number of physical unmanned aerial vehicles, the number of virtual unmanned aerial vehicles and a target flight channel;
and determining flight channels of the formation, the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the waypoints of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles.
4. The virtual-real combined simulation method for multiple unmanned aerial vehicles according to claim 1, wherein the constructing of the communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle comprises:
setting a positioning system, wherein the positioning system is used for identifying and positioning the identity of the physical unmanned aerial vehicle;
setting a service computer, wherein the service computer is in communication connection with a positioning system, is used for processing feedback information of the positioning system to form three-dimensional information corresponding to the virtual scene, and is at least used for calculating and determining the expected flight position of the physical unmanned aerial vehicle by combining an intelligent algorithm;
and a control computer is arranged, is in communication connection with the service computer, the positioning system and the physical unmanned aerial vehicle, and is at least used for receiving the expected flight position of the service computer, generating a corresponding control instruction, sending the control instruction to the physical unmanned aerial vehicle, obtaining related information of the physical unmanned aerial vehicle and forwarding the information to the service computer based on a UDP protocol.
5. The virtual-real combined simulation method of multiple unmanned aerial vehicles according to claim 4, wherein the positioning system comprises a GPS module disposed on the physical unmanned aerial vehicle, a plurality of cameras disposed in the physical scene and a plurality of reflective balls disposed on the physical unmanned aerial vehicle, the control computer is connected to the GPS module, the service computer is connected to the cameras to obtain the feedback information, and can send the feedback information to the control computer.
6. The virtual-real combined simulation method of multiple unmanned aerial vehicles according to claim 5, wherein the number and/or positions of the multiple light-reflecting balls on different physical unmanned aerial vehicles are different;
the service unmanned aerial vehicle or the control unmanned aerial vehicle can determine the ID of each real unmanned aerial vehicle and the position state of the real unmanned aerial vehicle in a real scene based on the set number and/or the set position of the plurality of reflective balls in the feedback information of the camera.
7. The virtual-real combined simulation method of multiple drones according to claim 6, wherein the controlling the flight of the virtual drone and the physical drone, respectively, based on the communication control system in combination with the state information, the formation and the flight channels of the drones, comprises:
respectively determining formation positions and flight channels of the physical unmanned aerial vehicles and the virtual unmanned aerial vehicles based on the service computer;
determining whether the real object unmanned aerial vehicle firstly enters a first coordinate threshold value of the flight channel based on the service computer and the current state information of the real object unmanned aerial vehicle, wherein the first coordinate threshold value is a coordinate threshold value corresponding to any one target position point in the flight channel, and the target position point corresponds to the expected flight position;
if so, the service computer sends the state information of the physical unmanned aerial vehicle to a target virtual unmanned aerial vehicle or a target physical unmanned aerial vehicle behind the physical unmanned aerial vehicle in a formation, so that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to a matched target position point based on respective flight channels, and feeds back the state information to the service unmanned aerial vehicle;
and when the service computer determines that the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle flies to the matched target position point based on the state information of the target virtual unmanned aerial vehicle or the target physical unmanned aerial vehicle, informing the control computer to enable the control computer to respectively generate and send instructions for flying to the next target position point for the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the current state information, the formation and the respective flight channels of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle.
8. The utility model provides a many unmanned aerial vehicle's virtuality and reality combines simulation device which characterized in that includes:
the first determining module is used for determining the scene characteristics of the real object;
the building module is used for building a virtual scene based on the physical scene characteristics;
the second determining module is used for determining the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles in the unmanned aerial vehicle cluster;
the third determining module is used for at least determining formation of unmanned aerial vehicle clusters in the virtual scene and flight channels of the virtual unmanned aerial vehicles and the physical unmanned aerial vehicles based on the virtual scene, the number of the physical unmanned aerial vehicles and the number of the virtual unmanned aerial vehicles;
the building module is used for building a communication control system between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle, and the communication control system is used for carrying out data interaction between the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle;
the fourth determination module is used for determining the state information of the physical unmanned aerial vehicle and the virtual unmanned aerial vehicle based on the communication control system;
and the control module is used for controlling the flight of the virtual unmanned aerial vehicle and the flight of the physical unmanned aerial vehicle respectively by combining the state information, the formation and the flight channels of the unmanned aerial vehicles based on the communication control system.
9. The apparatus of claim 8, wherein the determining the number of physical drones and the number of virtual drones in the cluster of drones comprises:
determining the number of physical unmanned aerial vehicles in the unmanned aerial vehicle cluster;
respectively constructing twin bodies corresponding to the real unmanned aerial vehicles in the virtual scene;
determining the number of the virtual drones and the twins.
10. The utility model provides a many unmanned aerial vehicle's virtuality and reality combines simulation system which characterized in that includes:
one or more processors;
a memory configured to store one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the virtual-real hybrid simulation method of the multiple drones of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310068031.XA CN115951598B (en) | 2023-01-16 | 2023-01-16 | Virtual-real combination simulation method, device and system for multiple unmanned aerial vehicles |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310068031.XA CN115951598B (en) | 2023-01-16 | 2023-01-16 | Virtual-real combination simulation method, device and system for multiple unmanned aerial vehicles |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115951598A true CN115951598A (en) | 2023-04-11 |
| CN115951598B CN115951598B (en) | 2023-12-01 |
Family
ID=87297312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310068031.XA Active CN115951598B (en) | 2023-01-16 | 2023-01-16 | Virtual-real combination simulation method, device and system for multiple unmanned aerial vehicles |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115951598B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114185320A (en) * | 2020-09-15 | 2022-03-15 | 中国科学院软件研究所 | Evaluation method, device and system for unmanned system cluster and storage medium |
| CN116149376A (en) * | 2023-04-21 | 2023-05-23 | 西安迈远科技有限公司 | Unmanned aerial vehicle cruising control method based on fabricated building platform |
| CN117220762A (en) * | 2023-11-09 | 2023-12-12 | 天津云圣智能科技有限责任公司 | Method, system, storage medium and electronic equipment for multilink data transmission |
| CN117742540A (en) * | 2024-02-20 | 2024-03-22 | 成都流体动力创新中心 | Virtual-real interaction system based on virtual engine and semi-physical simulation |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108845802A (en) * | 2018-05-15 | 2018-11-20 | 天津大学 | Unmanned plane cluster formation interactive simulation verifies system and implementation method |
| CN110928200A (en) * | 2019-12-17 | 2020-03-27 | 中国人民解放军军事科学院国防科技创新研究院 | Virtual-real linkage simulation test system and method for unmanned aerial vehicle |
| CN111208842A (en) * | 2019-12-27 | 2020-05-29 | 西安羚控电子科技有限公司 | Virtual unmanned aerial vehicle and entity unmanned aerial vehicle mixed cluster task control system |
| CN113485392A (en) * | 2021-06-17 | 2021-10-08 | 广东工业大学 | Virtual reality interaction method based on digital twins |
| CN113595622A (en) * | 2021-09-29 | 2021-11-02 | 南京航空航天大学 | Digital twin-based cluster collaborative search virtual-real combined verification method |
| CN114019828A (en) * | 2021-11-29 | 2022-02-08 | 中国人民解放军国防科技大学 | Multi-mode virtual-real interaction simulation system and method for unmanned aerial vehicle cluster |
| CN114707304A (en) * | 2022-03-11 | 2022-07-05 | 中国人民解放军国防科技大学 | Virtual-real combined multi-unmanned aerial vehicle perception avoidance verification system and method |
| CN115237000A (en) * | 2022-06-23 | 2022-10-25 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle formation cooperative countermeasure simulation test platform and test method |
-
2023
- 2023-01-16 CN CN202310068031.XA patent/CN115951598B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108845802A (en) * | 2018-05-15 | 2018-11-20 | 天津大学 | Unmanned plane cluster formation interactive simulation verifies system and implementation method |
| CN110928200A (en) * | 2019-12-17 | 2020-03-27 | 中国人民解放军军事科学院国防科技创新研究院 | Virtual-real linkage simulation test system and method for unmanned aerial vehicle |
| CN111208842A (en) * | 2019-12-27 | 2020-05-29 | 西安羚控电子科技有限公司 | Virtual unmanned aerial vehicle and entity unmanned aerial vehicle mixed cluster task control system |
| CN113485392A (en) * | 2021-06-17 | 2021-10-08 | 广东工业大学 | Virtual reality interaction method based on digital twins |
| CN113595622A (en) * | 2021-09-29 | 2021-11-02 | 南京航空航天大学 | Digital twin-based cluster collaborative search virtual-real combined verification method |
| CN114019828A (en) * | 2021-11-29 | 2022-02-08 | 中国人民解放军国防科技大学 | Multi-mode virtual-real interaction simulation system and method for unmanned aerial vehicle cluster |
| CN114707304A (en) * | 2022-03-11 | 2022-07-05 | 中国人民解放军国防科技大学 | Virtual-real combined multi-unmanned aerial vehicle perception avoidance verification system and method |
| CN115237000A (en) * | 2022-06-23 | 2022-10-25 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle formation cooperative countermeasure simulation test platform and test method |
Non-Patent Citations (4)
| Title |
|---|
| 李文皓;张珩;: "用于阵形控制的无人机编队飞行仿真平台设计与实现", 系统仿真学报, no. 03 * |
| 程林;樊琼剑;杨忠;沈春林;: "基于虚拟现实的无人机飞控系统半物理仿真研究", 中国制造业信息化, no. 19 * |
| 郭立普;魏瑞轩;侯海平;张立鹏;: "无人机编队飞行视景仿真关键技术研究", 计算机工程, no. 10 * |
| 高健: "虚实结合的无人机靶场设计与实现", 《 中国优秀硕士论文电子期刊网 工程科技Ⅱ辑》, no. 11 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114185320A (en) * | 2020-09-15 | 2022-03-15 | 中国科学院软件研究所 | Evaluation method, device and system for unmanned system cluster and storage medium |
| CN114185320B (en) * | 2020-09-15 | 2023-10-24 | 中国科学院软件研究所 | Evaluation method, device and system for unmanned system cluster and storage medium |
| CN116149376A (en) * | 2023-04-21 | 2023-05-23 | 西安迈远科技有限公司 | Unmanned aerial vehicle cruising control method based on fabricated building platform |
| CN117220762A (en) * | 2023-11-09 | 2023-12-12 | 天津云圣智能科技有限责任公司 | Method, system, storage medium and electronic equipment for multilink data transmission |
| CN117220762B (en) * | 2023-11-09 | 2024-01-26 | 天津云圣智能科技有限责任公司 | Method, system, storage medium and electronic equipment for multilink data transmission |
| CN117742540A (en) * | 2024-02-20 | 2024-03-22 | 成都流体动力创新中心 | Virtual-real interaction system based on virtual engine and semi-physical simulation |
| CN117742540B (en) * | 2024-02-20 | 2024-05-10 | 成都流体动力创新中心 | Virtual-real interaction system based on virtual engine and semi-physical simulation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115951598B (en) | 2023-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115951598B (en) | Virtual-real combination simulation method, device and system for multiple unmanned aerial vehicles | |
| US11202036B2 (en) | Merged reality system and method | |
| CN109669477A (en) | A kind of cooperative control system and control method towards unmanned plane cluster | |
| Aljehani et al. | Performance evaluation of multi-UAV system in post-disaster application: Validated by HITL simulator | |
| CN108549407A (en) | A kind of control algolithm of multiple no-manned plane collaboration formation avoidance | |
| Forsmo et al. | Optimal search mission with unmanned aerial vehicles using mixed integer linear programming | |
| CN113238583B (en) | Intensive formation flying and anti-collision control method for fixed-wing unmanned aerial vehicles | |
| CN109671118A (en) | A kind of more people's exchange methods of virtual reality, apparatus and system | |
| CN110032209A (en) | A kind of multiple no-manned plane mission area assembly distributed control method | |
| Tang et al. | A joint global and local path planning optimization for UAV task scheduling towards crowd air monitoring | |
| Sun et al. | Path planning for dense drone formation based on modified artificial potential fields | |
| Wubben et al. | A novel resilient and reconfigurable swarm management scheme | |
| Chipade et al. | Aerial swarm defense by stringnet herding: Theory and experiments | |
| Kung et al. | The fast flight trajectory verification algorithm for Drone Dance System | |
| Chen et al. | A fast coordination approach for large-scale drone swarm | |
| CN114019828A (en) | Multi-mode virtual-real interaction simulation system and method for unmanned aerial vehicle cluster | |
| CN112634662B (en) | Electronic fence, control system, method, medium, unmanned aerial vehicle formation and terminal | |
| Lee et al. | A mission management system for complex aerial logistics by multiple unmanned aerial vehicles in MBZIRC 2017 | |
| Ciarletta et al. | Simulation and platform tools to develop safe flock of UAVs: a CPS application-driven research | |
| CN108983818A (en) | Unmanned plane order switching method based on virtual architecture | |
| Obdržálek | Software environment for simulation of UAV multi-agent system | |
| Pannocchi et al. | Integrated framework for fast prototyping and testing of autonomous systems | |
| Xu et al. | HNMSim: A 3D multi-purpose hybrid networked multi-agent simulator | |
| CN114047783A (en) | Unmanned aerial vehicle system and unmanned aerial vehicle simulation system | |
| Kaugerand et al. | A system of systems solution for perimeter control: Combining unmanned aerial system with unattended ground sensor network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |