CN112114668B - Digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and reconnaissance method thereof - Google Patents
Digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and reconnaissance method thereof Download PDFInfo
- Publication number
- CN112114668B CN112114668B CN202010911931.2A CN202010911931A CN112114668B CN 112114668 B CN112114668 B CN 112114668B CN 202010911931 A CN202010911931 A CN 202010911931A CN 112114668 B CN112114668 B CN 112114668B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- reconnaissance
- individual
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 11
- 239000004576 sand Substances 0.000 claims abstract description 51
- 230000004927 fusion Effects 0.000 claims abstract description 5
- 230000002452 interceptive effect Effects 0.000 claims abstract description 4
- 239000011521 glass Substances 0.000 claims description 51
- 238000004891 communication Methods 0.000 claims description 26
- 230000003287 optical effect Effects 0.000 claims description 21
- 230000033001 locomotion Effects 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 19
- 238000013507 mapping Methods 0.000 claims description 15
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 claims description 12
- 238000007726 management method Methods 0.000 claims description 12
- 230000005540 biological transmission Effects 0.000 claims description 10
- 230000004424 eye movement Effects 0.000 claims description 4
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 230000003993 interaction Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 238000007654 immersion Methods 0.000 claims description 2
- 241000272814 Anser sp. Species 0.000 abstract description 2
- 230000004297 night vision Effects 0.000 description 6
- 238000005286 illumination Methods 0.000 description 4
- 230000005622 photoelectricity Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000005281 excited state Effects 0.000 description 2
- 230000005283 ground state Effects 0.000 description 2
- 230000004379 myopia Effects 0.000 description 2
- 208000001491 myopia Diseases 0.000 description 2
- 238000002834 transmittance Methods 0.000 description 2
- 241000272517 Anseriformes Species 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005684 electric field Effects 0.000 description 1
- 238000005401 electroluminescence Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/08—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer
- G09B9/30—Simulation of view from aircraft
- G09B9/307—Simulation of view from aircraft by helmet-mounted projector or display
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B9/00—Simulators for teaching or training purposes
- G09B9/02—Simulators for teaching or training purposes for teaching control of vehicles or other craft
- G09B9/08—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer
- G09B9/48—Simulators for teaching or training purposes for teaching control of vehicles or other craft for teaching control of aircraft, e.g. Link trainer a model being viewed and manoeuvred from a remote point
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Educational Administration (AREA)
- Business, Economics & Management (AREA)
- Aviation & Aerospace Engineering (AREA)
- Educational Technology (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and a reconnaissance method thereof, wherein the system comprises: the single soldier reconnaissance unmanned plane is used for reconnaissance on the battlefield situation; the AR display equipment can construct an AR three-dimensional virtual space, display a real battlefield environment, personnel equipment and a virtual target in a virtual-real fusion manner, perform interactive operation on the virtual 3D reconnaissance unmanned aerial vehicle, display an AR electronic sand table and display a battlefield situation; the wearable device is communicated with the single-soldier reconnaissance unmanned aerial vehicle in real time through a data chain and is connected with the AR display device; and the AR application system is used for quickly constructing the individual reconnaissance unmanned aerial vehicle and the real battlefield environment into an AR three-dimensional virtual space, so that the 'head goose' piloting function of the 3D unmanned aerial vehicle on a single or a plurality of real individual reconnaissance unmanned aerial vehicles is realized. According to the invention, the wearable AR display equipment is used for replacing the control equipment of the handheld unmanned aerial vehicle, so that the task roles can be conveniently and rapidly switched by the control hand, and the rifle and other weapons are used for fight.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicle reconnaissance, in particular to an AR individual soldier accompanying reconnaissance unmanned aerial vehicle system based on digital twinning and a reconnaissance method thereof.
Background
The light high-mobility synthesis part (partial) team adopts a three-dimensional reconnaissance mode of ground reconnaissance staff, reconnaissance vehicles and aerial unmanned aerial vehicles, so that the commander can master and draw battlefield situations, make a battle scheme, conduct battle command and reasonably use battle force and battle guarantee.
The single soldier accompanying reconnaissance unmanned aerial vehicle generally refers to a single soldier portable ultra-short-range unmanned aerial vehicle and is used for carrying out ultra-short-range (within 15 km) reconnaissance tasks accompanying teams or teams, such as a multi-rotor unmanned aerial vehicle, a rainbow "-902 small unmanned aerial vehicle and the like. The single-soldier accompanying reconnaissance unmanned aerial vehicle is small in size, and can be stored into a single-soldier knapsack after being folded, so that the single-soldier accompanying reconnaissance unmanned aerial vehicle is convenient to carry and use; the unmanned aerial vehicle is provided with the multispectral optical camera, can be used for various purposes such as reconnaissance monitoring, target positioning, firepower guiding, damage evaluation and the like, and becomes a telescope which can fly in the hands of a basic-level commander.
The single soldier accompanies reconnaissance unmanned aerial vehicle is controlled by the control hand through portable radio remote control equipment's action bars and panel part and takes off, flight reconnaissance and landing, and unmanned aerial vehicle remote control equipment's hi-lite display screen is used for showing unmanned aerial vehicle photoelectricity nacelle's optical image. The program control device of the unmanned aerial vehicle has the function of realizing one-key take-off or landing, can carry out autonomous navigation according to route planning, and automatically or semi-automatically uploads the reconnaissance image and data of the unmanned aerial vehicle to a commander. The main defects and the shortcomings are as follows:
1) The portable control equipment has a certain volume and weight, and the carrying and holding operation of the portable control equipment occupies the two hands of the control hand, so that the control hand is not beneficial to rapidly switching task roles, such as fight by using weapons such as rifle;
2) The portable control equipment has small volume and weight limitation, and a display screen is not convenient for the control hand to observe images, identify and track targets, display battlefield situations, route planning of unmanned aerial vehicles and the like;
3) The portable control equipment display screen can only display two-dimensional images, cannot display three-dimensional images such as a three-dimensional electronic sand table and the like, cannot construct a virtual three-dimensional space, and performs unmanned aerial vehicle control skill training, reconnaissance scheme deduction, pre-flight training, reconnaissance command training and the like of a three-dimensional virtual environment.
Disclosure of Invention
The invention aims to solve the technical problem of providing a digital twin-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and a reconnaissance method thereof, which are used for solving the defects pointed out in the background art, so that AR glasses with three-dimensional virtual image display and gesture recognition control functions replace portable radio remote control equipment of the individual soldier reconnaissance unmanned aerial vehicle, take-off, flying reconnaissance, landing and other operations of the unmanned aerial vehicle are performed in a virtual three-dimensional space constructed by the AR glasses, and a remote control photoelectric pod is used for performing manual image recognition and tracking.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows.
Digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system includes:
The single-soldier reconnaissance unmanned aerial vehicle can fly against a reconnaissance area to reconnaissance the battlefield situation;
the AR display equipment can construct an AR three-dimensional virtual space, display a real battlefield environment, personnel equipment and virtual targets in a virtual-real fusion manner, perform interactive operation on the 3D unmanned aerial vehicle, display an AR electronic sand table and display battlefield situations;
The wearable device is in real-time communication with the single-soldier reconnaissance unmanned aerial vehicle through a data link, receives the operation parameters of the single-soldier reconnaissance unmanned aerial vehicle and is connected with the AR display device; the wearable equipment can navigate and position the control hand and plot on the AR electronic sand table in real time; and
The AR application system can quickly construct the single-soldier reconnaissance unmanned aerial vehicle and the real battlefield environment into an AR three-dimensional virtual space, realizes that the 3D unmanned aerial vehicle and the real single-soldier reconnaissance unmanned aerial vehicle are in the same space-time, has a virtual-real mapping relation, can carry out support and guarantee tasks, and realizes the piloting function of the 3D unmanned aerial vehicle on single or multiple real single-soldier reconnaissance unmanned aerial vehicles.
According to the technical scheme, the single-soldier reconnaissance unmanned aerial vehicle comprises a machine body, a flight control system, a navigation positioning system, a photoelectric pod, a servo control system and a data chain airborne terminal, wherein the data chain airborne terminal is connected with the flight control system, the navigation positioning system, the photoelectric pod and the servo system of the single-soldier reconnaissance unmanned aerial vehicle through standard interfaces respectively.
Further optimizing the technical proposal, in the AR three-dimensional virtual space constructed by the AR display equipment, the 3D unmanned aerial vehicle of the three-dimensional electronic sand table is controlled to fly by the control hand through gesture recognition, the head gesture of the control hand is sensitive to control the rotation of the 3D photoelectric pod, and the 3D unmanned aerial vehicle and the motion parameters of the 3D nacelle thereof drive the real individual reconnaissance unmanned aerial vehicle to take off, fly reconnaissance and land, and the image recognition and target tracking of the photoelectric nacelle to realize the virtual-real mapping relationship between the 3D unmanned aerial vehicle and the real unmanned aerial vehicle, and are used for unmanned aerial vehicle control skill training, reconnaissance scheme deduction, pre-flying exercise and reconnaissance command training.
Further optimizing the technical scheme, the wearing device can be inserted into a left/right arm pocket of the camouflage suit or a side pocket of a tactical backpack; the wearable device includes:
The individual navigation positioning system is used for performing navigation positioning on the control hand, automatically positioning data of the individual navigation positioning system on the AR electronic sand table, and displaying the position of the control hand;
the data link ground terminal realizes real-time communication of remote control, telemetering information and reconnaissance images and is respectively connected with the individual navigation positioning system and the AR glasses through standard interfaces; the data transmission is carried out between the data link ground terminal and the data link airborne terminal; and
And the power supply module is used for providing electric energy.
According to the technical scheme, the position and speed parameters of the control hand can drive the real individual reconnaissance unmanned aerial vehicle to fly along with the flight, and the inclined distance and the angle between the control hand and the real individual reconnaissance unmanned aerial vehicle are randomly generated within a certain range each time.
Further optimizing the technical scheme, wherein the AR display equipment is AR glasses; the AR glasses comprise an AR optical module, a microprocessor, an image acquisition module, a sensor, a communication module and a structural component;
the microprocessor is used for constructing and controlling the AR three-dimensional virtual space and the 3D unmanned aerial vehicle;
the image acquisition module is used for performing image processing and gesture recognition;
the sensor comprises a head tracking sensor, a hand tracking sensor, an eye movement tracking sensor, a rainbow distance measuring sensor, a voice AI chip and a video AI chip;
the communication module comprises a wifi communication module and a Bluetooth communication module;
The structural component is used for installing and positioning the AR optical module, the microprocessor, the image acquisition module, the sensor and the communication module, and is integrated into the individual goggles or directly integrated onto the helmet.
Further optimizing technical scheme, AR glasses can build the virtual space that has the sense of immersing, can carry out the space mapping that virtual reality fused, can build AR electron sand table fast, carries out manual tracking, eye and moves tracking and head and trace, can control 3D unmanned aerial vehicle and its photoelectricity nacelle with human natural interaction's mode.
Further optimizing the technical scheme, the AR application system comprises:
AR management software for system login, task management, virtual screen and space management, model management and database management;
The AR electronic sand table software adopts a two-dimensional or three-dimensional electronic sand table form, can display, plot and calculate the two-dimensional or three-dimensional battlefield situation of the electronic sand table, and performs the route planning of the individual reconnaissance unmanned aerial vehicle;
the reconnaissance and aiming software displays reconnaissance and aiming images on a virtual screen of the AR glasses, and the reconnaissance images and aiming reticles can drive a photoelectric pod of a real individual reconnaissance unmanned aerial vehicle to rotate by collecting parameters of a head tracking sensor, reconnaissance and aiming reticles are driven to identify suspicious targets marked by automatic images, and firepower striking and fixed point throwing support and guarantee tasks are executed;
3D unmanned aerial vehicle control software, in virtual space with the virtual control part of gesture recognition operation, control virtual 3D unmanned aerial vehicle virtual take off, flight reconnaissance and landing of virtual reality, and 3D virtual photoelectric pod's image recognition and target tracking, 3D unmanned aerial vehicle control software is used for unmanned aerial vehicle control skill training, reconnaissance scheme deduction, advance flight exercise and reconnaissance command training; and
The AR digital twin software is characterized in that on one hand, the position, the gesture and the speed parameters of a real individual reconnaissance unmanned aerial vehicle can drive a 3D unmanned aerial vehicle on an AR electronic sand table to synchronously move, so that real-time battlefield situation display is realized; or on the one hand, the 3D unmanned aerial vehicle of the three-dimensional electronic sand table is controlled to fly through gesture recognition, and the head gesture of a hand is controlled to control the rotation of the 3D virtual photoelectric pod in a sensitive manner, so that the 3D unmanned aerial vehicle and the motion parameters of the 3D virtual photoelectric pod drive the real individual reconnaissance unmanned aerial vehicle to fly and perform photoelectric reconnaissance, and the virtual-real mapping relation between the 3D unmanned aerial vehicle and the real individual reconnaissance unmanned aerial vehicle is realized.
The digital twin-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system-based reconnaissance method is carried out on the basis of the digital twin-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and comprises the following steps:
s1, operating a hand to wear Augmented Reality (AR) glasses, logging in and starting a task, and quickly constructing a three-dimensional electronic sand table according to data of a digital map;
s2, setting the position of the individual reconnaissance unmanned aerial vehicle through gesture recognition or voice control, and carrying out route planning of the individual reconnaissance unmanned aerial vehicle;
S3, high-speed transmission is carried out between the data link onboard terminal and the data link ground terminal: the position, the gesture and the speed parameters of the single-soldier reconnaissance unmanned aerial vehicle, visible light and infrared reconnaissance images of the photoelectric pod, aiming reticle images, control data of a 3D unmanned aerial vehicle flight control system in an AR electronic sand table and control data of a servo system of the 3D unmanned aerial vehicle;
S4, the AR glasses receive the position, the gesture and the speed parameters of the real individual reconnaissance unmanned aerial vehicle of the data chain, and the synchronous data drive plots the motion of the 3D unmanned aerial vehicle in the AR electronic sand table, so that the real-time battlefield situation display is realized;
s5, the AR glasses receive images of the actual individual reconnaissance unmanned aerial vehicle of the data chain and aim at, and the images are displayed on a virtual screen of the AR glasses for controlling the hand to reconnaissance the target and aim at shooting;
S6, when the AR glasses watch the reconnaissance image and the aiming division, the suspicious target marked by the automatic image recognition mark is reconnaissance and screened, and the aiming division aiming target is controlled;
s7, controlling the hand to operate in a gesture recognition mode in the virtual space, controlling virtual take-off, flight reconnaissance and landing of the 3D unmanned aerial vehicle in virtual reality, transmitting operation parameters of the 3D unmanned aerial vehicle to the individual reconnaissance unmanned aerial vehicle, and controlling the individual reconnaissance unmanned aerial vehicle to follow the 3D unmanned aerial vehicle to move;
and controlling the head gesture of the hand to control the rotation of the 3D virtual photoelectric pod, so that the motion parameters of the 3D unmanned aerial vehicle and the 3D virtual photoelectric pod drive the individual reconnaissance unmanned aerial vehicle to move and the photoelectric reconnaissance.
By adopting the technical scheme, the invention has the following technical progress.
According to the invention, equipment such as wearable AR glasses, microprocessors, miniature antennas and the like are used for replacing control equipment of a handheld unmanned aerial vehicle, the size is small, the weight is light, the portability is good, double hands of a control hand can be liberated, the control hand can conveniently and rapidly change task roles, and weapons such as a rifle and the like are used for fight.
The invention constructs an immersive three-dimensional virtual space by the AR glasses, can display a two-dimensional or three-dimensional electronic sand table, can carry out unmanned aerial vehicle route planning, constructs a virtual control handle, a switch button and the like in the virtual space, and adopts a gesture recognition mode for operation. The scout image of the drone is displayed on the virtual screen of the AR glasses. The position, the motion parameters and the flight parameters of the real unmanned aerial vehicle are transmitted to the AR glasses in real time through a data link in a descending mode, and are plotted in a three-dimensional electronic sand table in real time, so that the battlefield situation is displayed in real time.
According to the invention, the 3D unmanned aerial vehicle in the AR virtual space is controlled to fly through gesture recognition, the head gesture of a sensitive control hand is used for controlling the rotation of the 3D virtual photoelectric pod, and the position and the motion parameters of the 3D unmanned aerial vehicle can be plotted in the three-dimensional electronic sand table in real time. Meanwhile, the 3D unmanned aerial vehicle is driven to fly by the position and motion parameters of the 3D unmanned aerial vehicle, the position and motion display of the 3D unmanned aerial vehicle on the AR electronic sand table are corrected, and the extremely similar virtual-real mapping relation (digital twin) between the 3D unmanned aerial vehicle and the real unmanned aerial vehicle is realized.
The unmanned plane can fly autonomously according to the route planning, and even can cut off communication contact, so that radio silence is kept. The unmanned aerial vehicle has the advantages that the unmanned aerial vehicle can fly along with the position and the speed of the control hand, the inclined distance and the angle between the unmanned aerial vehicle and the control hand are randomly generated within a certain range every time, the position of the control hand can be prevented from being accurately positioned by an enemy, and the safety of the control hand is guaranteed. The unmanned aerial vehicle can be set to be in a following state, the flight parameters of the 3D unmanned aerial vehicle are transmitted to the flight control system of the real unmanned aerial vehicle in an ascending mode, the real unmanned aerial vehicle is controlled to fly along with the 3D unmanned aerial vehicle, and the pilot function of the 3D unmanned aerial vehicle on a single or a plurality of real unmanned aerial vehicles is achieved.
The single soldier accompanying unmanned aerial vehicle can also execute support and guarantee tasks such as photoelectric reconnaissance, firepower striking, fixed-point throwing and the like; the virtual screen of the AR glasses can be windowed to display a reconnaissance image and an aiming division, and the unmanned aerial vehicle can be authorized to automatically track shooting or select a manual aiming tracking and shooting mode.
The control hand, the unmanned aerial vehicle position and the navigation information can be encrypted and uploaded to the command information system, and the information such as the issued combat command and the battlefield situation is received and displayed on the AR glasses. When the hand is controlled to travel or perform other tasks, the display positions or the hiding of the infrared night vision image of the helmet and the day and night image of the unmanned aerial vehicle can be controlled by voice or dragged by gestures, so that information interference is not caused.
Drawings
FIG. 1 is a system block diagram of a digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system of the present invention;
FIG. 2 is an outline of AR glasses in the digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system of the invention;
FIG. 3 is a control connection diagram of an AR optical module in the digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system;
FIG. 4 is a holographic optical waveguide optical scheme of an AR optical module in an AR individual soldier companion reconnaissance unmanned aerial vehicle system based on digital twinning;
fig. 5 is a free-form surface reflection scheme of an AR optical module in an AR individual soldier companion reconnaissance unmanned aerial vehicle system based on digital twinning.
Detailed Description
The invention will be described in further detail with reference to the drawings and the specific embodiments.
An AR individual soldier companion reconnaissance unmanned aerial vehicle system based on digital twinning, shown in connection with fig. 1, includes: the system comprises an individual reconnaissance unmanned plane, a data chain, an AR display device, a wearable device and an AR application system.
The single-soldier reconnaissance unmanned aerial vehicle can fly to a reconnaissance area to reconnaissance the battlefield situation.
The single-soldier reconnaissance unmanned aerial vehicle comprises a machine body, a flight control system, a navigation positioning system, a photoelectric pod, a servo control system and a data chain airborne terminal, wherein the data chain airborne terminal is connected with the flight control system, the navigation positioning system, the photoelectric pod and the servo system of the single-soldier reconnaissance unmanned aerial vehicle through standard interfaces respectively.
AR display device can construct the three-dimensional virtual space of AR, and virtual reality fuses real battlefield environment, personnel's equipment and virtual target of display, carries out interactive operation to 3D unmanned aerial vehicle, can show AR electron sand table, and the battlefield situation shows to carry out virtual reality's 3D unmanned aerial vehicle and come virtual individual soldier reconnaissance unmanned aerial vehicle's function.
The three-dimensional virtual space of AR that AR display device constructed controls the 3D unmanned aerial vehicle flight of three-dimensional electronic sand table through gesture recognition to the head gesture of sensitive control hand controls the rotation of 3D photoelectricity nacelle, and with the take-off of 3D unmanned aerial vehicle and 3D nacelle's motion parameter drive real individual soldier reconnaissance unmanned aerial vehicle, flight reconnaissance and landing, and the image recognition and the target tracking of photoelectricity nacelle, realize 3D unmanned aerial vehicle and real unmanned aerial vehicle's very similar virtual reality mapping relation (digital twin), be used for unmanned aerial vehicle control skill training, reconnaissance scheme deduction, advance flight exercise and reconnaissance command training.
The AR display device is AR glasses; the AR glasses include AR optical module, microprocessor, image acquisition module, sensor, communication module and structural component.
The sensor comprises a head tracking sensor, a hand tracking sensor, an eye movement tracking sensor, a rainbow distance measuring sensor, a voice AI chip and a video AI chip, wherein the output ends of the head tracking sensor, the hand tracking sensor, the eye movement tracking sensor, the rainbow distance measuring sensor, the voice AI chip and the video AI chip are respectively connected with the input end of the microprocessor.
The microprocessor is used for constructing and controlling the AR three-dimensional virtual space and the 3D unmanned aerial vehicle.
The image acquisition module is used for carrying out image processing and gesture recognition.
The communication module comprises a wifi communication module and a Bluetooth communication module.
The structural component is used for installing and positioning the AR optical module, the microprocessor, the image acquisition module, the sensor and the communication module, and is integrated into the individual goggles or directly integrated onto the helmet.
The AR glasses are integrated into individual goggles or directly into helmets, and can see virtual images generated by a microprocessor and visible light and infrared images of a wireless received photoelectric pod or other devices on a virtual screen of the AR glasses while viewing the real world in a perspective manner. The virtual screen size of the AR glasses is far larger than that of a display screen of the handheld control equipment, and the three-dimensional electronic sand table can be displayed to construct an AR three-dimensional virtual space.
The AR glasses can customize myopia lenses to replace original myopia lenses, the lining is arranged on helmet goggles, a large-size virtual screen can be displayed, for example, the AR glasses with small and medium angles of view in a certain FOV of 40 degrees can achieve the effect of a virtual screen of 86 inches at a position of 3 meters.
The AR glasses are free-form surface AR glasses with light transmittance of more than 50% or optical waveguide AR glasses with light transmittance of more than 80%, and the observation of battlefield environments by operating hands is not affected.
The appearance of AR glasses is shown in fig. 2. The AR optical module generally employs RGB optical illumination and LCOS reflective liquid crystal on silicon projection technology or organic light emitting diode OLED devices, as shown in fig. 3. The LCOS controller processes the signals and then loads the signals into the LCOS, and simultaneously transmits RGB illumination signals to the illumination control system, and the illumination control system lights RGBLEDs again to light the LCOS. The OLED is a device which utilizes a multilayer organic film structure to generate electroluminescence, holes generated by an anode and electrons generated by a cathode are respectively injected into a hole transmission layer and an electron transmission layer under the action of an electric field and migrate to a light-emitting layer, when the electrons and the holes at the interface of the light-emitting layer are accumulated to a certain number, the electrons and the holes are combined to generate excitons, electrons at the outermost layer of an excited organic molecule are transited from a ground state to an excited state, and energy is released in a light mode in the process of transiting the electrons at the excited state to the ground state, so that the excited light-emitting molecule generates visible light, and the light-emitting intensity is in direct proportion to the injected current.
The AR optical module generally adopts an optical scheme such as an optical waveguide or a free-form surface prism, and fig. 4 is a holographic optical waveguide optical scheme and fig. 5 is a free-form surface reflection scheme.
Visible light or infrared images of the unmanned aerial vehicle photoelectric pod can be wirelessly transmitted to a virtual screen of the AR glasses, and remote reconnaissance is carried out on an enemy target; the visible light or infrared image of the helmet night vision image intensifier or the individual weapon image sighting device can be transmitted to the AR virtual screen in a wired or wireless mode, so that the helmet night vision image intensifier or the individual weapon image sighting device has a night vision function, does not need to switch a mechanical helmet night vision device, is convenient to use, can realize remote control sighting of weapons, and has a 'no outcrop' shooting function.
The AR three-dimensional virtual image can construct a virtual space with immersion sense, can perform virtual-real fusion space mapping, can quickly construct an AR electronic sand table, performs manual tracking, eye tracking and head tracking, and can control the 3D unmanned aerial vehicle and the photoelectric pod thereof in a human body natural interaction mode.
The wearable device is in real-time communication with the single-soldier reconnaissance unmanned aerial vehicle through a data chain, and is connected with the AR display device through a standard interface to receive the operation parameters of the single-soldier reconnaissance unmanned aerial vehicle. The operation parameters comprise photoelectric images, positions and motion parameters of the individual reconnaissance unmanned aerial vehicle.
The wearing equipment can navigate and position the control hand and plot on the AR electronic sand table in real time.
The wearable device adopts an integrated and miniaturized design scheme and can be inserted into a left/right arm pocket of the camouflage suit or a side pocket of a tactical backpack. The wearable device mainly comprises an individual navigation positioning system, a data link ground terminal, a power module and the like.
The individual navigation positioning system is used for performing navigation positioning on the control hand, and the data of the individual navigation positioning system are automatically positioned on the AR electronic sand table to display the position of the control hand.
The power module is used for providing electric energy.
The data link ground terminal and the airborne terminal of the individual reconnaissance unmanned aerial vehicle adopt integrated and miniaturized designs, realize real-time communication (modulation, transmission, reception and demodulation) of remote control, telemetering information and reconnaissance images, and are respectively connected with the individual navigation positioning system and the AR glasses through standard interfaces. And data transmission is carried out between the ground terminal of the data link and the airborne terminal of the data link.
The visible light or infrared reconnaissance image, the aiming mark and the like can be displayed on the AR virtual screen, and automatic calibration can be performed. The data link adopts a working mode of combining frequency hopping and spreading to improve the anti-interference capability, adopts quantum communication to realize secret information transmission, adopts TDMA or dynamic TDMA networking control, and realizes a tactical data communication system with bidirectional, high-speed, secret and anti-interference.
The data link adopts a secret and anti-interference tactical information communication system, the data link on-board terminal is connected with a flight control system, a navigation positioning system, a photoelectric pod and a servo system of the unmanned aerial vehicle through standard interfaces, and the data link ground terminal is connected with a navigation positioning system of a control hand and AR glasses through standard interfaces.
The AR application system can quickly construct the single-soldier reconnaissance unmanned aerial vehicle and the real battlefield environment into an AR three-dimensional virtual space, so that the 3D unmanned aerial vehicle and the real single-soldier reconnaissance unmanned aerial vehicle are in the same space-time, have virtual-real mapping relation, can support and guarantee tasks, and realize the piloting function of the 3D unmanned aerial vehicle on single or multiple real single-soldier reconnaissance unmanned aerial vehicles.
The AR application system comprises AR management software, AR electronic sand table software, reconnaissance and aiming software, 3D unmanned aerial vehicle control software and AR digital twin software.
The AR management software is used for system login, task management, virtual screen and virtual space management, model management and database management.
The AR electronic sand table software adopts a two-dimensional or three-dimensional electronic sand table form, can display, plot and calculate the battlefield situation of the electronic sand table in two dimensions or three dimensions, and performs the route planning of the single-soldier reconnaissance unmanned aerial vehicle. The position, the gesture, the speed and other parameters of the single-soldier reconnaissance unmanned aerial vehicle can drive the 3D unmanned aerial vehicle on the AR electronic sand table to plot synchronously, so that real-time battlefield situation display is realized.
The reconnaissance and aiming software reconnaissance and aiming images are displayed on the virtual screen of the AR glasses, the reconnaissance images and aiming reticles can drive the photoelectric pod of the real individual reconnaissance unmanned aerial vehicle to rotate by collecting parameters of the head tracking sensor, the reconnaissance and aiming reticles are driven to aim at targets by identifying suspicious targets marked by automatic images, and firepower striking and fixed point throwing support and guarantee tasks are executed.
The 3D unmanned aerial vehicle control software is used for controlling skill training, reconnaissance scheme deduction, pre-flight training and reconnaissance command training of the unmanned aerial vehicle.
The AR digital twin software is characterized in that on one hand, the position, the gesture and the speed parameters of a real individual reconnaissance unmanned aerial vehicle can drive a 3D unmanned aerial vehicle on an AR electronic sand table to synchronously move, so that real-time battlefield situation display is realized;
or on the other hand, the 3D unmanned aerial vehicle of the three-dimensional electronic sand table is controlled to fly through gesture recognition, and the head gesture of a hand is controlled to control the rotation of the 3D virtual photoelectric pod in a sensitive manner, so that the 3D unmanned aerial vehicle and the motion parameters of the 3D virtual photoelectric pod drive the real single-soldier reconnaissance unmanned aerial vehicle to fly and perform photoelectric reconnaissance, and the very similar virtual-real mapping relation (digital twinning) between the 3D unmanned aerial vehicle and the real single-soldier reconnaissance unmanned aerial vehicle is realized; the pilot function of the 3D unmanned aerial vehicle on the 'head goose' of the single or multiple real individual reconnaissance unmanned aerial vehicle can also be realized.
Digital twinning (DIGITALTWIN) has no standard definition accepted by the industry, and the digital twinning of the invention is an industry popular concept, namely, a virtual 3D object (digital twinning body) and a physical entity (physical twinning body) have very similar virtual-real mapping relation.
After the operation parameters of the single-soldier reconnaissance unmanned aerial vehicle are collected by the wearable equipment, the 3D unmanned aerial vehicle can be driven by the AR application system to synchronously plot on the AR electronic sand table, and the battlefield situation is displayed in real time, so that the virtual-real mapping relation between the 3D unmanned aerial vehicle and the single-soldier reconnaissance unmanned aerial vehicle is realized.
The digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system-based reconnaissance method is carried out on the basis of the digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system, and comprises the following steps:
s1, operating a hand to wear Augmented Reality (AR) glasses, logging in and starting a task, and quickly constructing a three-dimensional electronic sand table according to data of a digital map.
S2, the electronic sand table has the functions of displaying, plotting, calculating and the like of two three-dimensional battlefield situations, and the position of the individual reconnaissance unmanned aerial vehicle is set through gesture recognition or voice control so as to conduct route planning of the individual reconnaissance unmanned aerial vehicle.
S3, high-speed transmission is carried out between the data link onboard terminal and the data link ground terminal: the position, the gesture and the speed parameters of the single-soldier reconnaissance unmanned aerial vehicle, visible light and infrared reconnaissance images of the photoelectric pod, aiming reticle images, control data of a 3D unmanned aerial vehicle flight control system in an AR electronic sand table and control data of a servo system of the 3D unmanned aerial vehicle.
S4, the AR glasses receive the position, the gesture and the speed parameters of the real individual reconnaissance unmanned aerial vehicle of the data chain, and the synchronous data drive plots the motion of the 3D unmanned aerial vehicle in the AR electronic sand table, so that the real-time battlefield situation display is realized.
S5, the AR glasses receive images of the actual individual reconnaissance unmanned aerial vehicle of the data chain and aim, and the images are displayed on a virtual screen of the AR glasses for controlling the hand to reconnaissance the target and aim and shoot.
S6, when the AR glasses watch the reconnaissance image and the aiming division, the suspicious target marked by the automatic image recognition mark is reconnaissance and screened, and the aiming division aiming target is controlled. Parameters of the head tracking sensor can be collected and uploaded to an unmanned aerial vehicle data link onboard terminal and a photoelectric pod servo system, so that the real unmanned aerial vehicle photoelectric pod is driven to rotate, aiming division aiming targets are controlled, and support and guarantee tasks such as aiming shooting and fixed point throwing by a control hand in a remote control mode are provided.
S7, controlling the hand to operate in a gesture recognition mode in the virtual space, controlling virtual take-off, flight reconnaissance and landing of the 3D unmanned aerial vehicle in virtual reality, transmitting operation parameters of the 3D unmanned aerial vehicle to the individual reconnaissance unmanned aerial vehicle, and controlling the individual reconnaissance unmanned aerial vehicle to follow the 3D unmanned aerial vehicle to move;
and controlling the head gesture of the hand to control the rotation of the 3D virtual photoelectric pod, so that the motion parameters of the 3D unmanned aerial vehicle and the 3D virtual photoelectric pod drive the individual reconnaissance unmanned aerial vehicle to move and the photoelectric reconnaissance. The image recognition and target tracking of the 3D virtual photoelectric pod can be used for unmanned aerial vehicle control skill training, scout scheme deduction, pre-flight training, scout command training and the like.
Through gesture recognition in the AR glasses, the 3D unmanned aerial vehicle with the controllable three-dimensional electronic sand table of the control hand flies, and the head gesture of the sensitive control hand controls the rotation of the 3D virtual photoelectric pod, so that the 3D unmanned aerial vehicle and the motion parameters of the 3D virtual photoelectric pod drive the real unmanned aerial vehicle to fly and photoelectric reconnaissance, and the very similar digital twin relation of the 3D unmanned aerial vehicle and the real unmanned aerial vehicle is realized.
The pilot function of the 3D unmanned aerial vehicle on the 'head geese' of a single or a plurality of real unmanned aerial vehicles can also be realized, and the specific realization process is as follows: setting a real unmanned aerial vehicle to be in a following state, and enabling flight parameters of the 3D unmanned aerial vehicle to be transmitted to a flight control system of the real unmanned aerial vehicle in an uplink mode to control the real unmanned aerial vehicle to follow the 3D unmanned aerial vehicle.
The invention provides a concept of an individual soldier accompanying flight reconnaissance unmanned aerial vehicle, wherein the unmanned aerial vehicle can carry out autonomous flight according to route planning, and an individual soldier accompanying flight reconnaissance mode can also be selected. The control hand has satellite positioning function, and the position and the speed parameter of control hand can drive true individual soldier reconnaissance unmanned aerial vehicle to accompany the flight, and the inclined distance and the angle between control hand and the true individual soldier reconnaissance unmanned aerial vehicle all produce at random at every turn in certain within range, can prevent that the enemy from utilizing unmanned aerial vehicle to pinpoint the position of control hand, consequently, the enemy can't utilize unmanned aerial vehicle to pinpoint the position of control hand, has ensured the safety of controlling the hand.
The inclined distance and the angle between the control hand and the real individual reconnaissance unmanned aerial vehicle can be set randomly every time, so that the aim is to prevent the situation that the control hand is easily locked and hit by an enemy due to the relative position relation of regularity of the control hand and the real individual reconnaissance unmanned aerial vehicle, and the aim is to realize the accompanying marching of the control hand of the individual reconnaissance unmanned aerial vehicle. When the single-soldier reconnaissance unmanned aerial vehicle is in marching, the position and the motion parameters of the control hand navigation positioning system are overlapped by the preset inclined distance and angle, so that the required position of the single-soldier reconnaissance unmanned aerial vehicle can be calculated, and the single-soldier reconnaissance unmanned aerial vehicle is used for controlling the single-soldier reconnaissance unmanned aerial vehicle to autonomously and concomitantly control hand marching, so that falcon marching is realized; unmanned aerial vehicle can carry out autonomous flight according to the route planning.
When the hand is controlled to travel or other tasks are executed, the infrared night vision image of the AR glasses and the display position or the display position of the day and night image of the unmanned aerial vehicle are hidden through voice control or gesture recognition dragging switching, so that information interference is not caused; the real-time control of the control hand is not needed, and even the communication connection between the unmanned aerial vehicle and the ground can be cut off, so that the radio silence is kept.
The image sighting device of the individual weapon can be selected and installed, remote sighting of the weapon can be realized, and the shooting function of 'no outcrop' is realized.
The position and navigation information of the individual soldier and the unmanned aerial vehicle can be encrypted, anti-interference uploaded and connected with the combat command system, and information such as combat commands and battlefield situations issued by the individual soldier and the unmanned aerial vehicle is received.
Claims (6)
1. AR individual soldier accompanies reconnaissance unmanned aerial vehicle system based on digit twin, its characterized in that includes:
The single-soldier reconnaissance unmanned aerial vehicle can fly against a reconnaissance area to reconnaissance the battlefield situation;
the AR display equipment can construct an AR three-dimensional virtual space, display a real battlefield environment, personnel equipment and virtual targets in a virtual-real fusion manner, perform interactive operation on the 3D unmanned aerial vehicle, display an AR electronic sand table and display battlefield situations;
the wearable device is in real-time communication with the single-soldier reconnaissance unmanned aerial vehicle through a data link, receives the operation parameters of the single-soldier reconnaissance unmanned aerial vehicle and is connected with the AR display device; the wearable equipment can navigate and position the control hand and plot on the AR electronic sand table in real time;
the wearable device can be inserted into a left or right arm pocket of a camouflage suit or a side pocket of a tactical backpack; the wearable device includes:
the individual navigation positioning system is used for performing navigation positioning on the control hand, automatically positioning data of the individual navigation positioning system on the AR electronic sand table, and displaying the position of the control hand; the position and speed parameters of the control hand can drive the real individual reconnaissance unmanned aerial vehicle to fly along with the flight, and the inclined distance and the angle between the control hand and the real individual reconnaissance unmanned aerial vehicle are randomly generated within a certain range each time;
The data link ground terminal realizes real-time communication of remote control, telemetering information and reconnaissance images and is respectively connected with the individual navigation positioning system and the AR glasses through standard interfaces; the data transmission is carried out between the data link ground terminal and the data link airborne terminal; and a power module for providing electrical energy;
The AR application system can quickly construct the single-soldier reconnaissance unmanned aerial vehicle and the real battlefield environment into an AR three-dimensional virtual space, so that the 3D unmanned aerial vehicle and the real single-soldier reconnaissance unmanned aerial vehicle are in the same space-time, have virtual-real mapping relation, can carry out support and guarantee tasks, and realize the piloting function of the 3D unmanned aerial vehicle on a single or a plurality of real single-soldier reconnaissance unmanned aerial vehicles;
The digital twin-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system-based reconnaissance method is carried out on the basis of the digital twin-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and comprises the following steps:
s1, operating a hand to wear Augmented Reality (AR) glasses, logging in and starting a task, and quickly constructing a three-dimensional electronic sand table according to data of a digital map;
s2, setting the position of the individual reconnaissance unmanned aerial vehicle through gesture recognition or voice control, and carrying out route planning of the individual reconnaissance unmanned aerial vehicle;
S3, high-speed transmission is carried out between the data link onboard terminal and the data link ground terminal: the position, the gesture and the speed parameters of the single-soldier reconnaissance unmanned aerial vehicle, visible light and infrared reconnaissance images of the photoelectric pod, aiming reticle images, control data of a 3D unmanned aerial vehicle flight control system in an AR electronic sand table and control data of a servo system of the 3D unmanned aerial vehicle;
S4, the AR glasses receive the position, the gesture and the speed parameters of the real individual reconnaissance unmanned aerial vehicle of the data chain, and the synchronous data drive plots the motion of the 3D unmanned aerial vehicle in the AR electronic sand table, so that the real-time battlefield situation display is realized;
s5, the AR glasses receive images of the actual individual reconnaissance unmanned aerial vehicle of the data chain and aim at, and the images are displayed on a virtual screen of the AR glasses for controlling the hand to reconnaissance the target and aim at shooting;
S6, when the AR glasses watch the reconnaissance image and the aiming division, the suspicious target marked by the automatic image recognition mark is reconnaissance and screened, and the aiming division aiming target is controlled;
s7, controlling the hand to operate in a gesture recognition mode in the virtual space, controlling virtual take-off, flight reconnaissance and landing of the 3D unmanned aerial vehicle in virtual reality, transmitting operation parameters of the 3D unmanned aerial vehicle to the individual reconnaissance unmanned aerial vehicle, and controlling the individual reconnaissance unmanned aerial vehicle to follow the 3D unmanned aerial vehicle to move;
and controlling the head gesture of the hand to control the rotation of the 3D virtual photoelectric pod, so that the motion parameters of the 3D unmanned aerial vehicle and the 3D virtual photoelectric pod drive the individual reconnaissance unmanned aerial vehicle to move and the photoelectric reconnaissance.
2. The digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system according to claim 1, wherein the individual soldier reconnaissance unmanned aerial vehicle comprises a body, a flight control system, a navigation positioning system, a photoelectric pod, a servo control system and a data link on-board terminal, and the data link on-board terminal is respectively connected with the flight control system, the navigation positioning system, the photoelectric pod and the servo system of the individual soldier reconnaissance unmanned aerial vehicle through standard interfaces.
3. The digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system according to claim 2, wherein an AR three-dimensional virtual space constructed by the AR display equipment is characterized in that a 3D unmanned aerial vehicle of a three-dimensional electronic sand table is controlled to fly by a control hand through gesture recognition, the rotation of a 3D photoelectric pod is controlled by the head gesture of the control hand, the real individual soldier reconnaissance unmanned aerial vehicle is driven to take off, fly reconnaissance and land by the motion parameters of the 3D unmanned aerial vehicle and the 3D pod thereof, the image recognition and target tracking of the photoelectric pod are realized, and the virtual-real mapping relation between the 3D unmanned aerial vehicle and the real unmanned aerial vehicle is realized for unmanned aerial vehicle control skill training, reconnaissance scheme deduction, pre-flying and reconnaissance command training.
4. The digital twinning-based AR individual companion reconnaissance unmanned aerial vehicle system of claim 1, wherein the AR display device is AR glasses; the AR glasses comprise an AR optical module, a microprocessor, an image acquisition module, a sensor, a communication module and a structural component;
the microprocessor is used for constructing and controlling the AR three-dimensional virtual space and the 3D unmanned aerial vehicle;
The image acquisition module is used for performing depth image processing and gesture recognition;
the sensor comprises a head tracking sensor, a hand tracking sensor, an eye movement tracking sensor, a rainbow distance measuring sensor, a voice AI chip and a video AI chip;
the communication module comprises a wifi communication module and a Bluetooth communication module;
The structural component is used for installing and positioning the AR optical module, the microprocessor, the image acquisition module, the sensor and the communication module, and is integrated into the individual goggles or directly integrated onto the helmet.
5. The digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system according to claim 4, wherein the AR glasses can construct a virtual space with immersion sense, can perform virtual-real fusion space mapping, can quickly construct an AR electronic sand table, perform manual tracking, eye tracking and head tracking, and can operate and control a 3D unmanned aerial vehicle and a photoelectric pod thereof in a human body natural interaction mode.
6. The digital twinning-based AR individual soldier companion reconnaissance unmanned aerial vehicle system of claim 4, wherein the AR application system comprises:
AR management software for system login, task management, virtual screen and space management, model management and database management;
The AR electronic sand table software adopts a two-dimensional or three-dimensional electronic sand table form, can display, plot and calculate the two-dimensional or three-dimensional battlefield situation of the electronic sand table, and performs the route planning of the individual reconnaissance unmanned aerial vehicle;
the reconnaissance and aiming software displays reconnaissance and aiming images on a virtual screen of the AR glasses, and the reconnaissance images and aiming reticles can drive a photoelectric pod of a real individual reconnaissance unmanned aerial vehicle to rotate by collecting parameters of a head tracking sensor, reconnaissance and aiming reticles are driven to identify suspicious targets marked by automatic images, and firepower striking and fixed point throwing support and guarantee tasks are executed;
3D unmanned aerial vehicle control software, in virtual space with the virtual control part of gesture recognition operation, control virtual 3D unmanned aerial vehicle virtual take off, flight reconnaissance and landing of virtual reality, and 3D virtual photoelectric pod's image recognition and target tracking, 3D unmanned aerial vehicle control software is used for unmanned aerial vehicle control skill training, reconnaissance scheme deduction, advance flight exercise and reconnaissance command training; and
The AR digital twin software is characterized in that on one hand, the position, the gesture and the speed parameters of a real individual reconnaissance unmanned aerial vehicle can drive a 3D unmanned aerial vehicle on an AR electronic sand table to synchronously move, so that real-time battlefield situation display is realized; or on the one hand, the 3D unmanned aerial vehicle of the three-dimensional electronic sand table is controlled to fly through gesture recognition, and the head gesture of a hand is controlled to control the rotation of the 3D virtual photoelectric pod in a sensitive manner, so that the 3D unmanned aerial vehicle and the motion parameters of the 3D virtual photoelectric pod drive the real individual reconnaissance unmanned aerial vehicle to fly and perform photoelectric reconnaissance, and the virtual-real mapping relation between the 3D unmanned aerial vehicle and the real individual reconnaissance unmanned aerial vehicle is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010911931.2A CN112114668B (en) | 2020-09-02 | 2020-09-02 | Digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and reconnaissance method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010911931.2A CN112114668B (en) | 2020-09-02 | 2020-09-02 | Digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and reconnaissance method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112114668A CN112114668A (en) | 2020-12-22 |
| CN112114668B true CN112114668B (en) | 2024-04-19 |
Family
ID=73805161
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010911931.2A Active CN112114668B (en) | 2020-09-02 | 2020-09-02 | Digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and reconnaissance method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112114668B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113055543B (en) * | 2021-03-31 | 2022-08-19 | 上海市东方医院（同济大学附属东方医院） | Construction method of digital twin command sand table of mobile hospital |
| CN113516903A (en) * | 2021-05-11 | 2021-10-19 | 中钢集团马鞍山矿山研究总院股份有限公司 | Intelligent mine scene oriented digital twin evolution mechanism and method |
| FI20215591A1 (en) * | 2021-05-18 | 2022-11-19 | Code Planet Saver Oy | Target acquisition system for an unmanned air vehicle |
| CN113468724B (en) * | 2021-06-09 | 2023-01-17 | 中国民航大学 | Digital twin system simulation method and device for airport aircraft landing guidance |
| CN113485392B (en) * | 2021-06-17 | 2022-04-08 | 广东工业大学 | Virtual reality interaction method based on digital twins |
| CN113595622B (en) * | 2021-09-29 | 2022-01-18 | 南京航空航天大学 | Digital twin-based cluster collaborative search virtual-real combined verification method |
| CN114565268B (en) * | 2022-02-25 | 2023-01-06 | 军事科学院系统工程研究院网络信息研究所 | Situation awareness-based scene self-adaptive cooperative command control system and method |
| CN114973835B (en) * | 2022-06-22 | 2024-03-22 | 常州萃思信息科技有限公司 | Digital deep immersion type interactive learning model training system and implementation method |
| CN116385685B (en) * | 2023-04-17 | 2023-10-31 | 北京誉飞科技发展有限公司 | Satellite ground reconnaissance twin system |
| CN116740293B (en) * | 2023-06-13 | 2024-07-26 | 西安宇速防务集团有限公司 | Digital twinning-based three-dimensional terrain model acquisition method, device and storage medium |
| CN116483210B (en) * | 2023-06-25 | 2023-09-08 | 安徽大学 | Brain-controlled unmanned aerial vehicle method and system based on deep learning and sliding mode control |
| CN116578035A (en) * | 2023-07-14 | 2023-08-11 | 南京理工大学 | Rotor unmanned aerial vehicle autonomous landing control system based on digital twin technology |
| CN117793754B (en) * | 2023-11-28 | 2024-06-25 | 山东大学 | Intelligent reflecting surface communication system deduction optimization method and system based on digital twinning |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR19990001299A (en) * | 1997-06-13 | 1999-01-15 | 양승택 | Operation method of UAV system |
| CN206202691U (en) * | 2016-10-19 | 2017-05-31 | 南京庞托斯智能科技有限公司 | A kind of unmanned aerial vehicle onboard head chases after individual-soldier system |
| CN107193371A (en) * | 2017-04-28 | 2017-09-22 | 上海交通大学 | A kind of real time human-machine interaction system and method based on virtual reality |
| CN108257145A (en) * | 2017-12-13 | 2018-07-06 | 北京华航无线电测量研究所 | A kind of UAV Intelligent based on AR technologies scouts processing system and method |
| CN110329515A (en) * | 2019-07-29 | 2019-10-15 | 陶文英 | It is a kind of to examine the design method and system for making integrated aircraft |
| CN111077909A (en) * | 2019-12-31 | 2020-04-28 | 北京理工大学 | Novel unmanned aerial vehicle self-group self-consistent optimization control method based on visual information |
-
2020
- 2020-09-02 CN CN202010911931.2A patent/CN112114668B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR19990001299A (en) * | 1997-06-13 | 1999-01-15 | 양승택 | Operation method of UAV system |
| CN206202691U (en) * | 2016-10-19 | 2017-05-31 | 南京庞托斯智能科技有限公司 | A kind of unmanned aerial vehicle onboard head chases after individual-soldier system |
| CN107193371A (en) * | 2017-04-28 | 2017-09-22 | 上海交通大学 | A kind of real time human-machine interaction system and method based on virtual reality |
| CN108257145A (en) * | 2017-12-13 | 2018-07-06 | 北京华航无线电测量研究所 | A kind of UAV Intelligent based on AR technologies scouts processing system and method |
| CN110329515A (en) * | 2019-07-29 | 2019-10-15 | 陶文英 | It is a kind of to examine the design method and system for making integrated aircraft |
| CN111077909A (en) * | 2019-12-31 | 2020-04-28 | 北京理工大学 | Novel unmanned aerial vehicle self-group self-consistent optimization control method based on visual information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112114668A (en) | 2020-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112114668B (en) | Digital twinning-based AR individual soldier accompanying reconnaissance unmanned aerial vehicle system and reconnaissance method thereof | |
| CN112150635B (en) | AR individual soldier and robot mixed-braiding combat system and method based on digital twin | |
| CN112823268B (en) | Display system for viewing optic | |
| US10972668B2 (en) | Display device and control method for display device | |
| US10599142B2 (en) | Display device and control method for display device | |
| CN107438804B (en) | It is a kind of for controlling the wearable device and UAV system of unmanned plane | |
| JP2021522464A (en) | Observation optics with direct active reticle targeting | |
| US10310502B2 (en) | Head-mounted display device, control method therefor, and computer program | |
| CN110402406A (en) | Observation optical frames with integrated display system | |
| US10030931B1 (en) | Head mounted display-based training tool | |
| US20090229160A1 (en) | Weapon Ball Stock With Integrated Weapon Orientation | |
| US11025826B2 (en) | Display system, display device, and control method for display device | |
| CN106125747A (en) | Based on the servo-actuated Towed bird system in unmanned aerial vehicle onboard the first visual angle mutual for VR | |
| CN108762291A (en) | A kind of method and system finding and track black winged unmanned aerial vehicle remote controller | |
| CN108759837A (en) | Unmanned plane multi computer communication combat system and method | |
| CN107140209A (en) | A kind of unmanned plane targeted system | |
| CN204426845U (en) | There is the remote control helmet seen and take aim at and guide function | |
| CN112099620A (en) | Combat collaboration system and method for soldier and team combat | |
| JP2018165066A (en) | Head mounted display and method for controlling the same | |
| CN208389335U (en) | A kind of system suitable for unmanned plane laser countermeasure (s) shooting game | |
| US20210335145A1 (en) | Augmented Reality Training Systems and Methods | |
| CN106444044A (en) | Headset display for civil aircraft | |
| CN206149414U (en) | Unmanned aerial vehicle reconnoiters device based on first visual angle of virtual reality is alert to be used | |
| CN209167680U (en) | Universal multidimensional becomes rail Ambient Intelligence graduation system | |
| CN114485268B (en) | Fire control system of unmanned helicopter integrating observation and striking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: Room 301, 3rd Floor, Building 8, No. 49 Badachu Road, Shijingshan District, Beijing, 100144 Applicant after: Beijing J&T Simulation Technology Co.,Ltd. Address before: Room 504, 505, 5 / F, building 8, yard 49, Badachu Road, Shijingshan District, Beijing 100144 Applicant before: BEIJING J&T SIMULATION TECHNOLOGY CO.,LTD. Country or region before: China |
|
| CB02 | Change of applicant information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Bai Xiaoliang Inventor after: Li Zhijun Inventor after: Sun Jianzhuang Inventor after: Gao Yushui Inventor before: Bai Xiaoliang Inventor before: Li Zhijun Inventor before: Sun Jianzhuang Inventor before: Gao Yushui |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |