CN116566792A - Distributed simulation method and system for multi-unmanned system - Google Patents
Distributed simulation method and system for multi-unmanned system Download PDFInfo
- Publication number
- CN116566792A CN116566792A CN202310416953.5A CN202310416953A CN116566792A CN 116566792 A CN116566792 A CN 116566792A CN 202310416953 A CN202310416953 A CN 202310416953A CN 116566792 A CN116566792 A CN 116566792A
- Authority
- CN
- China
- Prior art keywords
- simulation
- virtual
- network
- distributed
- unmanned system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 129
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000004891 communication Methods 0.000 claims abstract description 37
- 230000008447 perception Effects 0.000 claims abstract description 35
- 230000006870 function Effects 0.000 claims abstract description 27
- 230000003993 interaction Effects 0.000 claims abstract description 19
- 238000009877 rendering Methods 0.000 claims abstract description 18
- 206010061619 Deformity Diseases 0.000 claims abstract description 3
- 238000004422 calculation algorithm Methods 0.000 claims description 30
- 230000001360 synchronised effect Effects 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 13
- 238000005094 computer simulation Methods 0.000 claims description 6
- 238000012790 confirmation Methods 0.000 claims description 3
- 238000002360 preparation method Methods 0.000 claims description 3
- 238000012876 topography Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000005304 joining Methods 0.000 claims description 2
- 230000010354 integration Effects 0.000 abstract description 4
- 238000007726 management method Methods 0.000 description 25
- 238000004364 calculation method Methods 0.000 description 10
- 238000011161 development Methods 0.000 description 9
- 230000006872 improvement Effects 0.000 description 9
- 230000001133 acceleration Effects 0.000 description 8
- 238000012360 testing method Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000013461 design Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 238000012549 training Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 101000686031 Homo sapiens Proto-oncogene tyrosine-protein kinase ROS Proteins 0.000 description 4
- 102100023347 Proto-oncogene tyrosine-protein kinase ROS Human genes 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000005484 gravity Effects 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000001931 thermography Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000012530 fluid Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012804 iterative process Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000000699 topical effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a distributed simulation method and a distributed simulation system for a multi-unmanned system, wherein the method comprises the following steps: the distributed simulation tasks of the multi-unmanned system are decomposed into a distributed virtual simulation network, a perception control sub-network and a DDS-based collaborative communication service network according to three logic concepts of virtual simulation, control perception and task collaboration; deploying virtual simulation tasks coordinated with the tasks of the multi-unmanned system on the distributed nodes; rule arbitration judgment is provided through a comprehensive service end of the distributed virtual simulation network, map scene management, real-time state synchronization, multipoint session management, intelligent simulation and real-time situation sharing service are carried out; clients through the distributed virtual simulation network are used to provide local computing functions including dynamics, collision interactions, disfigurement simulation, graphics rendering. The system is used for implementing the method. The invention has the advantages of simple principle, wide application range, high functional integration level and the like.
Description
Technical Field
The invention mainly relates to the technical field of unmanned system simulation, in particular to a multi-unmanned system distributed simulation method and system.
Background
Unmanned systems all rely on simulation systems as platforms from structural design and module function development to integrated application, and continuous integrated test support is provided. The main task flow of the single unmanned system comprises the functional links of observation, judgment, decision and execution. The continuous integrated development process of the unmanned system requires the on-loop test of the model of the functional module level, the on-loop test of the software of the whole system level, the on-loop test of the hardware and the on-loop test of the real machine according to the stage sequence. Except for the real machine test, other links depend on simulation means, and the verification from the basic sub-functional logic module to the integrated complex functional system depends on a simulation environment capable of supporting continuous integrated development of an unmanned system.
The unmanned system simulation platform should contain object model and environment data, provide simulation and process deduction functions according with reality, also support the input/output interface of the unmanned system function module to be tested in terms of data interface, and interact corresponding simulation data with the interface. At present, the workflow of a simulation platform of an unmanned system is increasingly standardized, and the technology is gradually organized. With the development of unmanned systems represented by robotics, institutions, research institutions, and robotic suppliers have designed and developed simulation tools that provide dynamic simulation for control algorithm development, ROS/Gazebo, coppeliaSim (original V-REP), webots, adams, such as simulation tools that provide sensor-level environmental perception simulation, USARSim, flightGoggle, airsim, flightmare and Rflyim, etc., and a class of simulation environments for reinforcement learning represented by Issac gym, as offline commercial simulation tools, robot Master, robotArt, robotWorks, robcad, DELMIA, robotStudio, robomove, etc., that cover a wide range of simulation test platform software, such as automated pipeline designs. Some of these platforms simulate complex product operating environments for systems, such as RobotArt, robotWorks, robcad, others accelerate algorithm iterative processes for development processes, such as Gazebo, carla, etc., verify logic during security test, fine tune parameters during optimization, and support fast training machine learning algorithms, etc., such as OpenAI-Gym. Considering the relatively limited capabilities of a single unmanned system, more and more research has placed the center of gravity on the task collaboration of multiple unmanned systems.
In the prior art, a collaborative task simulation platform of a distributed multi-unmanned system is still in an immature stage. Most researches on collaborative planning control of heterogeneous unmanned systems are to simulate distributed collaborative control on a centralized verification platform, neglect the problem of space-time inconsistency of situation information sharing on the distributed platform, and simplify the labor division complexity in the task collaborative process under the distributed condition. Meanwhile, because of limited computing resources of a single platform, high-fidelity multi-source sensor data simulation cannot be realized, and scale-scalable distributed simulation cannot be supported.
The high-fidelity multi-source sensor data simulation can rely on various simulator technologies which are developed for years and are required by game development, and mature and complete game engine technologies. The perception simulation of the physical object of the unmanned system mainly comprises the steps of simulating an inertial measurement sensor by a simple motion state, and simulating visible light imaging by using a computer graphics technology and an image rendering engine. More common visible light image rendering engines are OSG, openGL, unreal Engine and the like, and with the improvement of the light ray tracing technology of the rendering Engine, the image rendering is more and more approaching to real details. The image rendering simulation engine can provide traditional ray tracing simulation, laser radar and depth camera simulation, and can also support static infrared sensor imaging simulation at present. For the support of an infrared thermal imaging sensor, the current online real-time simulation can only refer to the rendering mode of visible light imaging, is realized by using a UV (ultraviolet) map of temperature, and can not simulate the thermal imaging change of heat exchange between physical objects in time. Physical-level simulation modeling related to the movement of physical objects of the unmanned system is provided by physical engines in a simulation engine, and the more mature and common physical engines mainly comprise ODE, IKFast, bullet, copterSim, physX and Mujoco. The physical model resolving function applied in the simulation research of unmanned systems mainly focuses on rigid body dynamics simulation and collision simulation, and as the research goes deep, more and more simulation demands are added to the development roadmap of a physical engine, such as a flexible body, a fluid and the like. Most of the physical simulation calculation of continuous system simulation is to solve differential equation with constraint by using numerical optimization method, but physical engines adopting different modeling and calculation methods have corresponding advantages and disadvantages in specific fields.
Collaborative task simulation of a distributed multi-unmanned system relies on simulation of communication traffic. The comparison has the reference meaning of the open source projects ROS1 and ROS2. The two adopt different communication middleware respectively: ROS1 requires a Master to handle the Publish-Subscribe communication middle layer, while ROS2 uses DDS based on RTSP (Real-Time publication-subscriber) protocol as middle layer, DDS (Data-Distribution Service) Data distributed service is an industry standard for Real-Time and embedded system Publish-Subscribe communication, DDS can make the system more fault tolerant and flexible. While other supporting libraries and application software for ROS are very comprehensive, it is bulky and redundant for the algorithmic verification or simulation training of unmanned systems.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: aiming at the technical problems existing in the prior art, the invention provides the multi-unmanned system distributed simulation method and system which are simple in principle, wide in application range and high in functional integration level.
In order to solve the technical problems, the invention adopts the following technical scheme:
a multi-unmanned system distributed simulation method, comprising:
the distributed simulation tasks of the multi-unmanned system are decomposed into a distributed virtual simulation network, a perception control sub-network and a DDS-based collaborative communication service network according to three logic concepts of virtual simulation, control perception and task collaboration;
deploying virtual simulation tasks coordinated with the tasks of the multi-unmanned system on the distributed nodes;
rule arbitration judgment is provided through a comprehensive service end of the distributed virtual simulation network, map scene management, real-time state synchronization, multipoint session management, intelligent simulation and real-time situation sharing service are carried out; clients through the distributed virtual simulation network are used to provide local computing functions including dynamics, collision interactions, disfigurement simulation, graphics rendering.
As a further improvement of the process of the invention: the selection and switching of the map scene in the map scene management are uniformly controlled by a virtual environment comprehensive server; when the virtual environment comprehensive service end selects to load the map, all the clients participating in the current task load the same virtual scene map at the same time, and after all the clients complete the preloading of the map, the virtual environment comprehensive service end automatically completes the confirmation and synchronously enters the virtual scene.
As a further improvement of the process of the invention: the map scene management includes consistency management of map scene data; the map scene data comprises virtual entity objects controlled by topography, building, weather and a server side, virtual concept objects of default initial generation positions of roles, and allocation and association of dynamic non-player role virtual entity objects.
As a further improvement of the process of the invention: the real-time state synchronization process comprises the following steps:
step S1: acquiring all client-side connections connected to a virtual environment comprehensive server, firstly acquiring the maximum number of connections synchronized by each frame of an engine, and not maintaining the connections beyond the limit;
step S2: finding entity objects to be synchronized, only entity objects put into the network synchronization object list are considered;
step S3: finding the role controlled by the client;
step S4: verifying entity objects, wherein the objects to be destroyed and the objects with empty ownership cannot be synchronized;
step S5: if the synchronization time of the entity object is reached, setting the synchronization frequency of the entity object when the object is defined, calculating the next synchronization time before each synchronization, and giving up the synchronization if the next synchronization time is not reached;
step S6: if this object is set to be relevant only to the controlling client, it is put in a special list and then only synchronized to the clients belonging to him;
step S7: the entity objects in the dormant state are not synchronized, and the synchronization channel is closed for special treatment to be in the dormant state;
step S8: checking whether the current object has a channel or not, if not, checking whether the object is loaded in the scene or not, and skipping the synchronization step if not loaded; if in the scene, it is also determined whether the network is relevant, it is not synchronized for entities that are not visible or too far away;
step S9: and arranging priorities of all the objects, and calculating the priorities corresponding to the current objects according to whether a control end exists or not and whether the distance is in the visual field range, wherein the higher the priorities are, the more front the synchronization is.
As a further improvement of the process of the invention: the multipoint session management comprises the following procedures:
step S10: creating a session in a virtual simulation network by the virtual environment comprehensive service end, setting the name of the session, the number of the participating clients, whether the connection mode is local area network or Internet, and entering a preparation interface by the virtual environment comprehensive service end after clicking the creation;
step S20: after the virtual environment comprehensive service end is established, other clients find the session in a virtual environment comprehensive service end search interface, and join the session established by the previous client by clicking;
step S30: selecting a role after joining the session, wherein each or each type of unmanned system is used as a role, and the name of each accessed client is displayed in a list; the virtual environment comprehensive service end has the function of forcing the client end to kick out; and starting a simulation process by the virtual environment comprehensive server, wherein all clients synchronously enter a scene.
As a further improvement of the process of the invention: the real-time situation sharing is a service interface provided by the virtual environment comprehensive service end outwards, allows access of other situation systems, and issues the space position and state of the object to the other situation systems after the access; the function is a reserved function interface which is used for connecting a physical system to form a semi-physical simulation environment or accessing other virtual systems to integrate into a larger simulation platform.
A multi-unmanned system distributed simulation system, comprising:
the distributed virtual simulation network, the perception control sub-network and the DDS-based cooperative communication service network form a three-layer distributed simulation network;
the nodes in different sensing control sub-networks are connected by the cooperative communication service of the DDS, and are used for providing communication middleware services for situation sharing and task cooperative modules distributed in each sensing control sub-network according to task grouping of the unmanned system;
deploying virtual simulation tasks coordinated with the tasks of the multi-unmanned system on the distributed nodes; the virtual comprehensive server is used for providing rule arbitration judgment, map scene management, real-time state synchronization, multipoint session management, intelligent simulation and real-time situation service; the client is used for providing local computing functions including dynamics, collision interaction, damage simulation and graphic rendering; the client is used for providing three access modes: the operation simulation mode of the human in the loop; an intelligent unmanned system algorithm simulation mode; and simulating a deduction guiding control mode.
As a further improvement of the system of the invention: the perception control sub-network maps interaction authority to AR/VR head display equipment and man-machine equipment of a handle rocker in a loop mode or to an algorithm platform of software in a loop SITL or hardware in a loop HITL in an intelligent unmanned mode according to an access mode.
As a further improvement of the system of the invention: the DDS-based cooperative communication service network is used for providing communication middleware services for situation sharing and task cooperative modules distributed in each perception control sub-network according to task grouping of the unmanned system; the cooperative communication is forwarded through the virtual network according to the cooperative network.
As a further improvement of the system of the invention: the hardware is connected to the virtual network client in a loop mode through network connection or serial port mode, and topic type state/instruction forwarding is realized through a message network bridge; the system is directly connected with an upper computer, the dynamic simulation of the entity is realized by a virtual simulation client, the state is forwarded to a lower computer of an unmanned platform by the upper computer, the lower computer operates in a hardware loop mode, the filtered state is uploaded to the upper computer through a hardware interface, and the upper computer carries out algorithm processing; the software is in a loop mode, the distributed nodes of the collaborative algorithm are realized on python through integrating the function package of the DDS, and meanwhile, the software end acquires the corresponding perception control authority of the virtual object state/instruction from the interface service of the client through a remote procedure call mode.
Compared with the prior art, the invention has the advantages that:
1. the multi-unmanned system distributed simulation method and system provided by the invention are simple in principle, wide in application range and high in functional integration level, and can provide simulation services including dynamics, collision interaction, damage simulation, graphic rendering and communication interaction functions for the control, planning and verification of a distributed collaborative algorithm of the multi-unmanned system. The multi-unmanned system distributed simulation platform designed by the invention has six basic characteristics: a. the distributed simulation platform comprises a distributed virtual simulation network, a perception control sub-network and a three-layer distributed simulation network of a DDS-based cooperative communication service network; b. the virtual comprehensive server in the distributed virtual simulation network provides 6 main services including (1) rule arbitration judging service, (2) map scene management service, (3) real-time state synchronization service, (4) multi-point session management service, (5) intelligent blue-side simulation service and (6) real-time situation sharing service for the client; c. the client in the distributed virtual simulation network can provide three access modes of (1) an operation simulation mode of a human in a loop, (2) an algorithm simulation mode of an intelligent unmanned system and (3) a simulation deduction guide control mode; d. the client provides dynamics simulation, collision interaction calculation and graphic rendering 3-type local calculation for the virtual unmanned system object binding the authority, and provides a sensor interface of the unmanned system object and a type 2 interface of a planning control interface for the perception control sub-network; e. the perception control sub-network maps interaction authority to AR/VR head display equipment and man-machine equipment of a handle rocker in a loop mode or to an algorithm platform of software in a loop (SITL) or hardware in a loop (HITL) in an intelligent unmanned mode according to an access mode; f. the DDS-based cooperative communication service network provides communication middleware services for situation sharing and task cooperative modules distributed in each perception control sub-network according to task grouping of the unmanned system. The cooperative communication may be forwarded through the virtual network as needed by the cooperative network.
2. The multi-unmanned system distributed simulation method and system provided by the invention have the advantages that the network structure logic is clear, and the function development and service expansion integration are convenient. Aiming at the functions and service characteristics of the unmanned systems, the distributed simulation tasks of the unmanned systems are decomposed into corresponding three-layer network structures of the distributed virtual simulation network, the perception control sub-network and the DDS-based collaborative communication service network according to three logic concepts of virtual simulation, perception control and task collaboration, so that the integrated expansion of the platform system function service is facilitated. .
3. The distributed simulation method and system for the multi-unmanned system, disclosed by the invention, have the advantages of balanced simulation service load and ensured scale expansion and real-time operation support. In order to ensure real-time operation and scale expandability of simulation services, the invention disassembles the complex coupled model calculation of the multi-unmanned system into rule arbitration judgment operated at a server, map scene management, real-time state synchronization, multi-point session management, intelligent blue-side simulation and real-time situation sharing, and intelligent perception and planning control algorithms operated at a client, interaction calculation, graphic rendering and a terminal.
4. The multi-unmanned system distributed simulation method and system provided by the invention have flexible and various access modes, and support simulation training and algorithm optimization modes. The invention correspondingly designs the perception control sub-network and the hardware driving interface for realizing the operation simulation mode of the human in-loop, the algorithm simulation mode of the intelligent unmanned system and the simulation deduction guide regulation control mode according to three access requirements of the human in-loop, the intelligent algorithm and the simulation guide regulation.
5. The distributed simulation method and system of the multi-unmanned system, disclosed by the invention, have the advantages that the running modes are distributed and parallel, and the information sharing and cooperative task protocols are verified. The nodes in different sensing control sub-networks are connected by the DDS cooperative communication service, and communication middleware services can be provided for situation sharing and task cooperative modules distributed in each sensing control sub-network according to task grouping of the unmanned system, and are used for verifying protocol design of information sharing and cooperative tasks.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of the implementation principle of the present invention in a specific application example.
Fig. 3 is a schematic diagram of a control loop in a self-stabilizing mode in a specific application example of the present invention.
Fig. 4 is a schematic diagram of a control loop in a semi-self-stabilizing mode in a specific application example of the present invention.
Fig. 5 is a schematic diagram of the control loop of the present invention in a full manual mode in a specific application example.
Fig. 6 is a schematic diagram of linear position tracking in a position tracking control mode in a specific application example of the present invention.
FIG. 7 is a schematic diagram of curve position tracking in a position tracking control mode in a specific application example of the present invention.
Fig. 8 is a schematic diagram of circular arc path tracking in a position tracking control mode in a specific application example of the present invention.
Detailed Description
The invention will be described in further detail with reference to the drawings and the specific examples.
The multi-unmanned system distributed simulation method and system can provide simulation services including dynamics, collision interaction, damage simulation, graphic rendering and communication interaction functions for control, planning and verification of distributed cooperative control of the multi-unmanned system.
As shown in fig. 1 and fig. 2, the multi-unmanned system distributed simulation system of the invention comprises a distributed virtual simulation network, a perception control sub-network and a cooperative communication service network based on DDS, wherein the networks form a three-layer distributed simulation network; the distributed simulation task of the multi-unmanned system is decomposed and corresponds to three logic concepts of virtual simulation, control perception and task cooperation.
The invention further connects the nodes in different sensing control sub-networks by the DDS cooperative communication service, so that the nodes can provide communication middleware service for situation sharing and task cooperative modules distributed in each sensing control sub-network according to the task group of the unmanned system.
In a distributed virtual simulation network, the distributed simulation technology is used for deploying virtual simulation tasks coordinated with multiple unmanned system tasks on distributed nodes; the server side is used for providing rule arbitration judgment, map scene management, real-time state synchronization, multipoint session management, intelligent simulation and real-time situation sharing 6 services; the client is used to provide local computing functions including dynamics, collision interactions, damage simulation, and graphics rendering.
In a specific application example, a virtual comprehensive server in the distributed virtual simulation network provides multiple services for a client; the service includes: (1) rule arbitration decision service; (2) a map scene management service; (3) real-time state synchronization service; (4) a multipoint session management service; (5) intelligent blue-side simulation service; (6) real-time situation sharing services.
In a specific application example, the client in the distributed virtual simulation network can provide three access modes, where the access modes include: (1) the operation simulation mode of the human in the loop; (2) an intelligent unmanned system algorithm simulation mode; (3) three access modes of the control mode of the simulation deduction guide control are adopted.
In a specific application example, a client in the distributed virtual simulation network provides dynamic simulation, collision interaction calculation and graphic rendering 3-type local calculation for a virtual unmanned system object with binding authority, and provides two types of interfaces of a sensor interface and a planning control interface of the unmanned system object for a perception control sub-network.
In a specific application example, the perception control sub-network maps interaction authority to the AR/VR head display device and the man-machine device of the handle rocker in the loop mode or to the algorithm platform of Software In The Loop (SITL) or Hardware In The Loop (HITL) in the intelligent unmanned mode according to the access mode.
In a specific application example, the DDS-based collaborative communication service network provides communication middleware services for situation sharing and task collaboration modules distributed in each perception control sub-network according to task grouping of the unmanned system. The cooperative communication may be forwarded through the virtual network as needed by the cooperative network.
In a specific application example, the invention further designs a perception control sub-network and a hardware driving interface for realizing an operation simulation mode of a human-in-loop, an intelligent unmanned system algorithm simulation mode and a simulation deduction guide control mode according to three access requirements of the human-in-loop, the intelligent algorithm and the simulation guide.
In a specific application example, the distributed virtual simulation network comprises a virtual environment comprehensive service end and a virtual environment client end.
The virtual environment comprehensive service end can be deployed independently, and one of the distributed hosts can be used as a service end and a client end. The virtual environment comprehensive service end is used for providing rule arbitration judgment, map scene management, real-time state synchronization, multipoint session management, intelligent blue-side simulation and real-time situation sharing for the distributed virtual simulation network, wherein 6 services are shared. Wherein:
the rule arbitration judgment is processed by the virtual environment comprehensive server in consideration of consistency, and is mainly used for task simulation participated by the multi-unmanned system.
And the selection and switching of the map scenes in the map scene management are uniformly controlled by the virtual environment comprehensive server. When the virtual environment comprehensive service end selects to load the map, all the clients participating in the current task load the same virtual scene map at the same time, and after all the clients complete the preloading of the map, the virtual environment comprehensive service end automatically completes the confirmation and synchronously enters the virtual scene. Further, as a preferred embodiment, the map scene management includes consistency management of map scene data. The map scene data comprises virtual entity objects controlled by topography, building, weather and a server side, virtual concept objects of default initial generation positions of roles, and dynamic assignment and association of virtual entity objects of blue sides (non-player roles).
In the map scene management, the large map scenes are managed in a partition synchronization mode. The large map scene can be synchronously managed by a multi-core of the virtual environment comprehensive service end, and the map can be cut into a plurality of sub-checkpoints according to the actual application requirements in view of reducing the memory utilization rate. The local map service of the client dynamically loads the sub-checkpoints according to the correlation, and unloads the sub-checkpoints when not needed. The change of any object in the map scene (sub-checkpoint) by the client is temporarily saved through the map scene management of the virtual environment comprehensive server and is synchronized to all clients.
In the invention, the real-time state synchronization is as follows: the virtual environment comprehensive service end can execute the synchronization of the related content of the entity object in each frame. The invention adopts the following procedures in the specific application example:
step S1: acquiring all client-side connections connected to a virtual environment comprehensive service end, firstly acquiring the maximum number of connections which can be synchronized by each frame of an engine, and not maintaining the connections any more beyond the limit;
step S2: finding entity objects to be synchronized, only entity objects put into the network synchronization object list are considered;
step S3: finding a role controlled by a client (usually bound to a virtual camera), the position of which is a key factor in determining whether other physical objects are synchronized to the client;
step S4: verifying entity objects, wherein the objects to be destroyed and the objects with empty ownership cannot be synchronized;
step S5: if the synchronization time of the entity object is reached, the synchronization frequency is usually set when the object is defined, the next synchronization time is calculated before each synchronization, and if the synchronization time is not reached, the synchronization is abandoned;
step S6: if the object is set to be associated with only the controlling client, it is put in a special list and then only synchronized to the clients belonging to him;
step S7: the entity objects in the dormant state are not synchronized, and the synchronization channel is closed for special treatment to be in the dormant state;
step S8: checking whether the current object has a channel or not, if not, checking whether the object is loaded in the scene or not, and skipping the synchronization step if not loaded; if in the scene, it is also determined whether the network is relevant, it is not synchronized for entities that are not visible or too far away;
step S9: the number of syncs of an Actor may be very large, so it is necessary to prioritize all objects, usually according to whether there is a control end and whether the distance is in the field of view to calculate the priority corresponding to the current object, the higher the priority, the more the syncs are.
In the invention, the multipoint session management comprises the following procedures:
step S10: the virtual environment comprehensive service end firstly needs to create a session in the virtual simulation network, the name of the session, the number of the participating clients and the connection mode are local area network or Internet, and after clicking the creation, the virtual environment comprehensive service end enters a preparation interface.
Step S20: after the virtual environment comprehensive service end is established, other clients can find the session in the virtual environment comprehensive service end search interface, and join the session established by the previous client by clicking. For the same local area network, all virtual environment comprehensive service ends for establishing the session can be found through service discovery, and different virtual environment comprehensive service ends can be differentiated through the conditions of names, map types and the like of the virtual environment comprehensive service ends.
Step S30: and after the session is added, the roles are selected, each or each type of unmanned system is used as a role, and the name of each accessed client is displayed in a list, so that the management is convenient. The virtual environment comprehensive service end also has the function of forcing the client to kick out. And finally, starting a simulation process by the virtual environment comprehensive server, and synchronously entering the scene by all the clients.
The smart object simulation is used for providing a physical object with a certain behavior mode and can react to the environment and unmanned system targets. For example, the invention adopts a combination mode of a behavior tree, a blackboard and a behavior module.
In a specific application example, the real-time situation sharing is a service interface provided by the virtual environment comprehensive service end outwards, allows access of other situation systems, and can release the spatial position and state of the object to the other situation systems after the access. The function is a reserved function interface, can be used for connecting a physical system to form a semi-physical simulation environment, and can also be accessed into other virtual systems to integrate into a larger simulation platform. The specific data interface can be displayed through the pilot window function of the client.
In a specific application example, the client in the distributed virtual simulation network is configured to provide: the system comprises three access modes, namely an operation simulation mode of a human loop, an algorithm simulation mode of an intelligent unmanned system and a simulation deduction control mode. Furthermore, the client side provides three types of local calculation including dynamics simulation, collision interaction calculation and graphic rendering for the virtual unmanned system object binding the authority, and provides two types of interfaces including a sensor interface and a planning control interface of the unmanned system object for the perception control sub-network.
In particular applications, unmanned system models include, but are not limited to, unmanned aerial vehicles, unmanned vehicles. The input items can be input through a simulator, and the client of Python and the client of the virtual simulation deduction environment can be interacted through a perception control sub-network, and the input is mainly divided into two types: one is motion control input and the second is task execution input.
Input mapping table of unmanned system control channel
In a specific application example, the perception control sub-network maps interaction authority to the AR/VR head display device and the man-machine device of the handle rocker in the loop mode or to the algorithm platform of Software In The Loop (SITL) or Hardware In The Loop (HITL) in the intelligent unmanned mode according to the access mode.
In a specific application example, the hardware is connected to the virtual network client in a loop manner mainly through network connection or serial port manner, and the topical state/instruction forwarding is realized through a message network bridge (ROS-Integration). The system is generally directly connected with an upper computer, the dynamic simulation of the entity is realized by a virtual simulation client, the state is forwarded to a lower computer of an unmanned platform by the upper computer, the lower computer operates in a hardware loop mode, the filtered state is uploaded to the upper computer through a hardware interface, and the upper computer carries out algorithm processing.
In a specific application example, the software is in a loop mode and is mainly realized by python, a distributed node of a collaborative algorithm is realized on python through integrating a DDS function package, and meanwhile, a software end acquires a corresponding perception control authority of a virtual object state/instruction from an interface service (API-server) of a client through a Remote Procedure Call (RPC) mode.
In a specific application example, the access of the AR/VR head-display device is mainly used for simulating training of a person in a loop, and a visual data stream is generated by a client in a streaming mode and is streamed to a display device such as a display or the AR/VR head-display device. Particularly, for the AR/VR case, position and posture data of a head display camera and a camera video stream configured by AR equipment are returned, so that the simulation training body with better fusion degree can be generated.
In a specific application example, the DDS-based collaborative communication service network provides communication middleware services for situation sharing and task collaboration modules distributed in each perception control sub-network according to task grouping of the unmanned system. The cooperative communication may be forwarded through the virtual network as needed by the cooperative network.
Taking unmanned aerial vehicle control as an example, in a specific application example, the unmanned aerial vehicle dynamics simulation algorithm is further adopted in the invention:
and carrying out rigid body dynamics modeling on the unmanned aerial vehicle by adopting a Newton-Euler formula (Newton-Euler). The rigid body analysis is carried out on the unmanned aerial vehicle, and the movement of the unmanned aerial vehicle can be divided into translational movement of a base point and rotational displacement around a certain axis on the base point:
the origin of the rigid coordinate system (body coordinate system) selects the mass center of the rigid body, and the translational partIs applied by rigid body, wherein ∈>For momentum, add>The linear velocity, m is the rigid body mass; rotating the moving part->For the resultant moment to which the rigid body is subjected, +.>Is moment of momentum->The rotation angular velocity of the rigid body, and J is the moment of inertia of the rigid body.
The external force (ground coordinate system) mainly comprises the lift force generated by the rotor wingExercise induced air resistance->Gravity force. The resultant moment (body coordinate system) mainly comprises lift moment M T Moment of inertia of anti-torque M D Gyro moment (neglect).
Wherein F is i ＝ T ρn i 2 D 4 I is the lift force generated by the rotation of the corresponding rotor wing, n i Representing the rotational speed of the i rotor.
Under the ground system, the unmanned aerial vehicle receives air resistanceIn proportion to the square of the flying speed, the projection area S of the direction perpendicular to each moving axis yz ,S xy S yx Proportional to +.>For a translational drag coefficient in the corresponding direction, the drag can be expressed as:
in the geodetic coordinate system, the local gravitational acceleration is approximately g=g0 (1-2 h/R), and the body is expressed by gravity as:
and (3) synthesizing a formula to obtain resultant force born by the unmanned aerial vehicle under the ground coordinate system:
describing the rotation moment under the machine body coordinate system, and using M under the simple assumption that the rigid body and the rotor shaft are perfectly parallel Tx ,M Ty ,M Tz Indicating that the lift moment to which the body is subjected is X b ,Y b ,Z b The on-axis component, the matrix form of which can be expressed as:
wherein the method comprises the steps ofRepresenting projection vectors of x and y axes of rotor i under machine body coordinate system respectively, and inertial anti-torsion moment is M Di ＝C P ρF i 2 D 5 /(2 pi) is proportional to the square of the rotational speed.
Further, the unmanned aerial vehicle adopts four control modes:
first mode: self-stabilized mode control law;
referring to fig. 3, the control rate of the traversing machine in the self-stabilizing mode includes two-stage control of an inner loop and an outer loop, wherein the outer loop is a position control loop, and the inner loop is a gesture control loop. The input to the control algorithm is the desired speedAnd yaw rate>Output is the rotational speed (n 1 ,n 2 ,n 3 ,n 4 ). The position control loop designs three independent PID controllers, thereby achieving the desired acceleration:
wherein,,and respectively feeding back speed information for an on-board sensor (virtual inertial measurement unit). Calculating the desired roll angle θ from simultaneous solutions r Pitch angle->And throttle U:
the inner loop contains a PID control loop for attitude:
wherein,,θ s ,ψ s attitude angle data fed back by an onboard sensor (virtual inertial measurement unit) respectively. The throttle thrust force U and the expected attitude angular acceleration calculated through the position control loop are calculated by the formula inversion, and the expected motor rotating speed n is calculated i And distributing the power to the motor control module.
Second mode: semi-self-stabilized mode control law;
referring to fig. 4, a semi-self-stabilized mode (also called a trick mode) is shown in which only the control loop of the angular velocity of the gesture is shown, the difference is that there is no outer loop position control loop compared to the self-stabilized mode, and the gesture control loop uses PID feedback control of the angular velocity of the gesture as shown in the formula. The input channel definition reference table shows that the input quantity corresponding to the No. 0 channel of the remote control lever is pitch angle speed, the input quantity corresponding to the No. 1 channel is roll angle speed, the No. 2 channel is yaw angle speed, and the No. 3 channel is throttle quantity. Thus the input of the overall control algorithm isThe input to the motor control is inverted similarly to the throttle thrust U, and the rotational speeds (n 1 ,n 2 ,n 3 ,n 4 ) And then the motor control module controls the rotating speed of the rotor wing.
The attitude response of the semi-self-stabilized mode is faster and more sensitive than the self-weighted mode, and the corresponding position stability is not as good as the self-weighted mode. If the control input maintains the continuous angular velocity input, the traversing machine will continue to move around the axis and make special actions such as idle turning.
Third mode: attitude tracking mode control laws;
referring to fig. 5, in the mode, the aircraft moves according to the proportion of the rocker amount in the remote control mode, the right rocker is controlled to any direction, the attitude of the aircraft is inclined by a corresponding angle, and the original attitude can be returned after the remote control lever returns to the middle; sometimes, the correction is performed by repeatedly moving the deflector rod back and forth to enable the aircraft to return to the original attitude.
In the full manual mode, as shown, the remote control pitch, yaw and roll channel amounts do not pass through the position control loop, but only include the attitude angle control loop. Pitch angle θ and roll angleThrough a first order forward difference, and a desired yaw rate +.>Entering an attitude control loop (consistent with an inner ring attitude angle control loop of a self-stabilizing control mode), wherein the input of a control algorithm of the mode is theta and/or (I)>And throttle thrust U 1 Output is the rotational speed (n 1 ,n 2 ,n 3 ,n 4 )。
Fourth mode: a position tracking control mode;
referring to fig. 6-8, the present invention employs an L1 path-following algorithm, the basic idea of which is to select a reference point on the desired trajectory, and with this to generate a lateral acceleration, expressed as,
the flying arc of the unmanned aerial vehicle approaches the expected track under the action of the transverse acceleration, the formula is easy to deduce, and it can be seen that the acceleration is related to the current airspeed, the included angle between the airspeed and the expected point of L1, and the distance between the unmanned aerial vehicle and the expected point of L1. Airspeed can be observed, so the solution of transverse acceleration mainly comprises the steps of determining the length of L1 and solving the angle eta.
For a straight line desired path:
the calculation of the expected acceleration of the circular arc is mainly geometric derivation, and can be seen from the derivation in the paper, and mainly relates to the included angle eta between the three angular airspeeds and the tangent line of the point 2 The included angle eta between the connecting line from the point to L1 and the corresponding chord of L1 1 ，2η 3 Corresponds to the central angle corresponding to L1.
The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above examples, and all technical solutions belonging to the concept of the present invention belong to the protection scope of the present invention. It should be noted that modifications and adaptations to the invention without departing from the principles thereof are intended to be within the scope of the invention as set forth in the following claims.
Claims (10)
1. The distributed simulation method of the multi-unmanned system is characterized by comprising the following steps of:
the distributed simulation tasks of the multi-unmanned system are decomposed into a distributed virtual simulation network, a perception control sub-network and a DDS-based collaborative communication service network according to three logic concepts of virtual simulation, control perception and task collaboration;
deploying virtual simulation tasks coordinated with the tasks of the multi-unmanned system on the distributed nodes;
rule arbitration judgment is provided through a comprehensive service end of the distributed virtual simulation network, map scene management, real-time state synchronization, multipoint session management, intelligent simulation and real-time situation sharing service are carried out; clients through the distributed virtual simulation network are used to provide local computing functions including dynamics, collision interactions, disfigurement simulation, graphics rendering.
2. The multi-unmanned system distributed simulation method according to claim 1, wherein the selection and switching of map scenes in the map scene management are uniformly controlled by a virtual environment comprehensive server; when the virtual environment comprehensive service end selects to load the map, all the clients participating in the current task load the same virtual scene map at the same time, and after all the clients complete the preloading of the map, the virtual environment comprehensive service end automatically completes the confirmation and synchronously enters the virtual scene.
3. The multi-unmanned system distributed simulation method of claim 2, wherein the map scene management comprises consistency management of map scene data; the map scene data comprises virtual entity objects controlled by topography, building, weather and a server side, virtual concept objects of default initial generation positions of roles, and allocation and association of dynamic non-player role virtual entity objects.
4. A multi-unmanned system distributed simulation method according to any of claims 1 to 3, wherein the flow of real-time state synchronization comprises:
step S1: acquiring all client-side connections connected to a virtual environment comprehensive server, firstly acquiring the maximum number of connections synchronized by each frame of an engine, and not maintaining the connections beyond the limit;
step S2: finding entity objects to be synchronized, only entity objects put into the network synchronization object list are considered;
step S3: finding the role controlled by the client;
step S4: verifying entity objects, wherein the objects to be destroyed and the objects with empty ownership cannot be synchronized;
step S5: if the synchronization time of the entity object is reached, setting the synchronization frequency of the entity object when the object is defined, calculating the next synchronization time before each synchronization, and giving up the synchronization if the next synchronization time is not reached;
step S6: if this object is set to be relevant only to the controlling client, it is put in a special list and then only synchronized to the clients belonging to him;
step S7: the entity objects in the dormant state are not synchronized, and the synchronization channel is closed for special treatment to be in the dormant state;
step S8: checking whether the current object has a channel or not, if not, checking whether the object is loaded in the scene or not, and skipping the synchronization step if not loaded; if in the scene, it is also determined whether the network is relevant, it is not synchronized for entities that are not visible or too far away;
step S9: and arranging priorities of all the objects, and calculating the priorities corresponding to the current objects according to whether a control end exists or not and whether the distance is in the visual field range, wherein the higher the priorities are, the more front the synchronization is.
5. A multi-unmanned system distributed simulation method according to any of claims 1 to 3, wherein the multipoint session management comprises the following steps:
step S10: creating a session in a virtual simulation network by the virtual environment comprehensive service end, setting the name of the session, the number of the participating clients, whether the connection mode is local area network or Internet, and entering a preparation interface by the virtual environment comprehensive service end after clicking the creation;
step S20: after the virtual environment comprehensive service end is established, other clients find the session in a virtual environment comprehensive service end search interface, and join the session established by the previous client by clicking;
step S30: selecting a role after joining the session, wherein each or each type of unmanned system is used as a role, and the name of each accessed client is displayed in a list; the virtual environment comprehensive service end has the function of forcing the client end to kick out; and starting a simulation process by the virtual environment comprehensive server, wherein all clients synchronously enter a scene.
6. The multi-unmanned system distributed simulation method according to any one of claims 1 to 3, wherein the real-time situation sharing is a service interface provided by a virtual environment comprehensive service end outwards, and allows access of other situation systems, and after the access, the spatial position and state of an object are released to the other situation systems; the function is a reserved function interface which is used for connecting a physical system to form a semi-physical simulation environment or accessing other virtual systems to integrate into a larger simulation platform.
7. A multi-unmanned system distributed simulation system, comprising:
the distributed virtual simulation network, the perception control sub-network and the DDS-based cooperative communication service network form a three-layer distributed simulation network;
the nodes in different sensing control sub-networks are connected by the cooperative communication service of the DDS, and are used for providing communication middleware services for situation sharing and task cooperative modules distributed in each sensing control sub-network according to task grouping of the unmanned system;
deploying virtual simulation tasks coordinated with the tasks of the multi-unmanned system on the distributed nodes; the virtual comprehensive server is used for providing rule arbitration judgment, map scene management, real-time state synchronization, multipoint session management, intelligent simulation and real-time situation service; the client is used for providing local computing functions including dynamics, collision interaction, damage simulation and graphic rendering; the client is used for providing three access modes: the operation simulation mode of the human in the loop; an intelligent unmanned system algorithm simulation mode; and simulating a deduction guiding control mode.
8. The multi-unmanned system distributed simulation system of claim 7, wherein the perception control sub-network maps interaction rights to the AR/VR head-mounted device and the human-machine device of the handle rocker in loop mode or to the software in intelligent unmanned mode on loop SITL or the algorithm platform of hardware on loop HITL according to the access mode.
9. The multi-unmanned system distributed simulation system of claim 7, wherein the DDS-based collaborative communication services network is configured to provide communication middleware services for situation sharing and task collaboration modules distributed in each of the perception control subnetworks according to task groupings of the unmanned system; the cooperative communication is forwarded through the virtual network according to the cooperative network.
10. The multi-unmanned system distributed simulation system of claim 7, wherein the hardware is connected to the virtual network client in a loop manner through network connection or serial port manner, and the topic type state/instruction forwarding is realized through a message bridge; the system is directly connected with an upper computer, the dynamic simulation of the entity is realized by a virtual simulation client, the state is forwarded to a lower computer of an unmanned platform by the upper computer, the lower computer operates in a hardware loop mode to upload the filtered state to the upper computer through a hardware interface, the upper computer displays distributed nodes of a cooperative algorithm, and meanwhile, a software end obtains the corresponding perception control authority of the virtual object state/instruction from the interface service of the client in a remote procedure call mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310416953.5A CN116566792A (en) | 2023-04-18 | 2023-04-18 | Distributed simulation method and system for multi-unmanned system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310416953.5A CN116566792A (en) | 2023-04-18 | 2023-04-18 | Distributed simulation method and system for multi-unmanned system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116566792A true CN116566792A (en) | 2023-08-08 |
Family
ID=87500970
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310416953.5A Pending CN116566792A (en) | 2023-04-18 | 2023-04-18 | Distributed simulation method and system for multi-unmanned system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116566792A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117688679A (en) * | 2023-11-03 | 2024-03-12 | 山东建筑大学 | Machine tool transmission system parameter identification algorithm based on forward dynamics model simulation |
-
2023
- 2023-04-18 CN CN202310416953.5A patent/CN116566792A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117688679A (en) * | 2023-11-03 | 2024-03-12 | 山东建筑大学 | Machine tool transmission system parameter identification algorithm based on forward dynamics model simulation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109669477A (en) | A kind of cooperative control system and control method towards unmanned plane cluster | |
| Pettersson et al. | Probabilistic roadmap based path planning for an autonomous unmanned helicopter | |
| CN114063474A (en) | Semi-physical simulation system of unmanned aerial vehicle cluster | |
| CN113050677A (en) | Control method, system and storage medium for maintaining and changing formation of multiple unmanned aerial vehicles | |
| CN111993424B (en) | Interoperation middleware testing system and method for heterogeneous mobile robot | |
| Sislak et al. | Automated conflict resolution utilizing probability collectives optimizer | |
| CN116566792A (en) | Distributed simulation method and system for multi-unmanned system | |
| CN113887134A (en) | Large-scale lightweight unmanned cluster test simulation platform and working method | |
| Suvarna et al. | Simulation of autonomous airship on ROS-Gazebo framework | |
| Ciarletta et al. | Simulation and platform tools to develop safe flock of UAVs: a CPS application-driven research | |
| CN113138608A (en) | Four-rotor unmanned aerial vehicle visual servo control method using disturbance observer and nonlinear speed observer | |
| Riether | Agile quadrotor maneuvering using tensor-decomposition-based globally optimal control and onboard visual-inertial estimation | |
| Park et al. | Realization of distributed formation flying using a group of autonomous quadcopters and application to visual performance show | |
| US20220391558A1 (en) | Multi-agent simulation system and method | |
| Castillo-Effen et al. | Modeling and visualization of multiple autonomous heterogeneous vehicles | |
| Guerra et al. | Flightgoggles: A modular framework for photorealistic camera, exteroceptive sensor, and dynamics simulation | |
| CN114936455A (en) | Unmanned aerial vehicle digital twin model construction method | |
| Pyvovar et al. | Simulation of flight control of two uavs based on the “master-slave” model | |
| AbdElHamid et al. | A novel software simulator model based on active hybrid architecture | |
| Huang et al. | Integrated virtual simulation and test system for vision-based applications of UAVs | |
| Ji et al. | Simulation of unmanned aircraft system performing surveillance mission based on advanced distributed architecture | |
| Kang et al. | Real-time validation of formation control for fixed-wing UAVs using multi hardware-in-the-loop simulation | |
| Meier | Dynamic Robot Architecture for Robust Realtime Computer Vision | |
| CN117492381B (en) | Robot collaborative pointing simulation visualization method, system, equipment and storage medium | |
| CN116047889B (en) | Control compensation method and device in virtual-real combination simulation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |