CN113778222A - AR-based special case handling method and device - Google Patents
AR-based special case handling method and device Download PDFInfo
- Publication number
- CN113778222A CN113778222A CN202110833683.9A CN202110833683A CN113778222A CN 113778222 A CN113778222 A CN 113778222A CN 202110833683 A CN202110833683 A CN 202110833683A CN 113778222 A CN113778222 A CN 113778222A
- Authority
- CN
- China
- Prior art keywords
- data
- software
- module
- aircraft
- special
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000002452 interceptive effect Effects 0.000 claims description 35
- 238000004891 communication Methods 0.000 claims description 6
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 7
- 230000003190 augmentative effect Effects 0.000 abstract description 6
- 230000015572 biosynthetic process Effects 0.000 abstract 1
- 238000003786 synthesis reaction Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 9
- 238000004088 simulation Methods 0.000 description 6
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000011017 operating method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Geometry (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Computer Hardware Design (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention belongs to the field of AR (augmented reality) technology application, in particular to an AR-based special case handling method and an AR-based special case handling device, which solve the problems that in the prior art, a special case handling mode depends on the coolness and the accuracy of operation of an operator, operation errors are easy to occur, or the operation processing speed is not timely, so that the special case handling fails, and therefore greater danger is caused, wherein the AR-based special case handling method comprises the following processes: the method comprises the steps of aircraft state judgment, aircraft special situation judgment, aircraft cabin operation element identification, special situation handling flow judgment, AR image synthesis and AR special situation handling guidance. The special condition handling method based on AR provided by the invention is quick, accurate and timely, ensures that the aircraft is separated from danger as soon as possible and enters a normal operation flight state, and ensures the safety of personnel and the aircraft.
Description
Technical Field
The invention belongs to the field of AR (augmented reality) technology application, particularly relates to a special situation disposal method and device based on AR, and particularly relates to a method for carrying out special situation disposal by using AR technology for a manned aircraft.
Background
The Augmented Reality (Augmented Reality) technology is a technology for skillfully fusing virtual information and a real world, and after virtual information such as characters, images, three-dimensional models, music, videos and the like generated by a computer is simulated, the virtual information is applied to the real world and seamlessly combined with a real scene, and the two kinds of information are mutually supplemented, so that the real world is 'Augmented', and the Augmented Reality technology has strong Reality sense.
The AR has three major technical points: three-dimensional registration (i.e., three-dimensional registration technology), virtual reality fusion, and human-computer interaction. The method comprises the steps that a camera is used for collecting a real scene, a sensor is used for collecting coordinates and position data of the real scene, an AR computing system is responsible for generating a virtual object, computing coordinates of the real scene, and carrying out three-dimensional registration fusion of the virtual object and the real scene, so that an output scene formed by fusing the real scene and the virtual object into a whole is formed and displayed in an AR helmet or other display equipment.
At present, a manned aircraft, such as a flying device of an airplane, a spacecraft and the like, is a complex system, and the special situations of a power component, a control component, an electrical system, a mechanical system and the like can occur in the operation process. When the systems are in special conditions, the flight and control safety is often endangered, and the systems need to be timely treated and corrected to be in a normal state so as to ensure the safety of flight operation. In the special handling process, under the state that the aircraft flies at a high speed, operators need to be cool and rational, and fast and accurate handling is carried out according to the field condition so as to correct the aircraft to be in a normal state as soon as possible and to be out of danger.
Currently, the aircraft special treatment mode is that an operator completes operation according to special treatment steps and operation modes according to memory, ground command guidance or field POP manual (flight operation manual) so as to correct the aircraft. The special handling mode depends on the coolness of an operator and the accuracy of operation, operation errors are easy to occur, or the operation processing speed is not timely, so that the special handling is failed, and further more danger is caused. How to maintain fast special treatment guidance and accurately and fast treatment is a key issue concerned by those skilled in the art. Based on the above statements, the invention provides a special case handling method and device based on AR.
Disclosure of Invention
The invention aims to solve the problems that in the prior art, a special case handling mode depends on the coolness of an operator and the accuracy of operation, operation errors are easy to occur, or the special case handling is failed due to the fact that the operation processing speed is not timely, and therefore greater danger is caused.
An AR-based idiom handling method, comprising the following processes: judging the state of the aircraft, judging the special situation of the aircraft, identifying an operation element of an aircraft cabin, judging a special situation handling flow, synthesizing an AR image and guiding the AR special situation handling; specifically, the state judgment based on the big data can be performed through data of an airborne sensor of the aircraft, when the state judgment is in the special situation state, the flow and the method can be extracted according to the special situation processing flow and method of the expert database, the AR image and the guide information of the special situation processing flow are fused by being matched with an aircraft cabin and an element image, and an operator is guided to perform special situation processing.
A special situation handling device based on AR comprises airborne software, simulation software, support software, intermediate interface software, a data interface and a soft protocol interface;
the airborne software and the simulation software both adopt the same architecture and comprise a software function module, a data module, an interactive data service module and an adapter module;
the airborne software is used for acquiring data of the aircraft sensor and processing the acquired data by an algorithm;
the soft protocol interface directly delivers the protocol communication packet to the software bus for communication and simulates the receiving and sending conditions of the airborne software and the simulation software.
Preferably, the software function module is a function module of onboard software or a function module of simulation software; for example, avionics software, radar software, flight control software and the like in airborne software, flight/flight control simulation, visual simulation, computer force simulation and the like in simulation software.
Preferably, the data module is a data pool for publishing and subscribing data, and is mainly connected with the software function module and the interactive data service module, so that on one hand, the publishing data of the software function module is stored, and the subscribed data is received; and on the other hand, the interactive data service module is provided with local published data and receives local subscription data from a remote place.
Preferably, the interactive data service module is a data interactive module manufactured according to a subscription and publishing service mode, and is used for realizing publishing of local data and subscribing of remote data.
Preferably, the software function module, the data module, the interactive data service module and the like can be reused.
Preferably, one end of the adapter module is connected with the interactive data service module, and can receive published data from the interactive data service module, or provide subscription data for the interactive data service module and provide interactive data for the adapter module; the other end is connected with a bus of a specific physical protocol, mainly translates data to be transmitted by the interactive data service module into a data packet required by the specific bus protocol, then transmits the data packet to the other end through the bus, receives the bus data at the other end, translates the data packet into a subscription data structure required by the interactive data service module, and triggers subscription processing.
Preferably, the sensor data acquisition and algorithmic processing includes, but is not limited to, radar, airspeed, barometric pressure, engine.
The special situation handling method based on AR provided by the invention has the following beneficial effects:
1. the special condition handling method based on AR provided by the invention is quick, accurate and timely, ensures that the aircraft is separated from danger as soon as possible and enters a normal operation flight state, and ensures the safety of personnel and the aircraft.
2. The invention adopts a method of combining virtuality and reality, can identify cockpit operating equipment and operating elements thereof, judges special conditions according to the sensor states of an aircraft and an engine, uses a special condition disposal expert library to carry out data retrieval, superposes the guiding information of each step of operating method and operating direction on the image of a field camera, and displays the image and the direction guiding information of the cockpit and the operating flow and method on an AR helmet display, guides an operator to carry out accurate and quick operation, can realize quick and accurate processing of special condition disposal, and pulls the aircraft state back to a normal state by the fastest optimal flow, thereby ensuring the safety of flight motion of the aircraft.
Drawings
Fig. 1 is a software architecture integrating airborne software and simulation software in an AR-based special case handling device according to the present invention;
fig. 2 is a flowchart of a special case handling method based on AR according to the present invention.
Detailed Description
The present invention will be further illustrated with reference to the following specific examples.
Example one
The invention provides an AR-based special case handling method, which comprises the following steps: judging the state of the aircraft, judging the special situation of the aircraft, identifying an operation element of an aircraft cabin, judging a special situation handling flow, synthesizing an AR image and guiding the AR special situation handling; specifically, the state judgment based on the big data can be performed through data of an airborne sensor of the aircraft, when the state judgment is in the special situation state, the flow and the method can be extracted according to the special situation processing flow and method of the expert database, the AR image and the guide information of the special situation processing flow are fused by being matched with an aircraft cabin and an element image, and an operator is guided to perform special situation processing.
A special situation handling device based on AR comprises airborne software, simulation software, support software, intermediate interface software, a data interface and a soft protocol interface;
the airborne software and the simulation software adopt the same architecture and comprise a software function module, a data module, an interactive data service module and an adapter module;
the software functional module is a functional module of airborne software or a functional module of simulation software; for example, avionics software, radar software, flight control software and the like in airborne software, flight/flight control simulation, visual simulation, computer force simulation and the like in simulation software.
The data module is a data pool for publishing and subscribing data, is mainly connected with the software functional module and the interactive data service module, stores the publishing data of the software functional module on one hand, and receives the subscribed data; and on the other hand, the interactive data service module is provided with local published data and receives local subscription data from a remote place.
The interactive data service module is a data interactive module manufactured according to a subscription and publishing service mode and is used for realizing publishing of local data and subscribing of remote data.
The software function module, the data module, the interactive data service module and the like can be reused.
One end of the adapter module is connected with the interactive data service module, and can receive published data from the interactive data service module, or provide subscription data for the interactive data service module and provide interactive data for the adapter module; the other end is connected with a bus of a specific physical protocol, mainly translates data to be transmitted by the interactive data service module into a data packet required by the specific bus protocol, then transmits the data packet to the other end through the bus, receives the bus data at the other end, translates the data packet into a subscription data structure required by the interactive data service module, and triggers subscription processing.
The airborne software is used for acquiring data of aircraft sensors, acquiring data information including radar, airspeed, air pressure, an engine and the like, and performing algorithm processing on acquired data of various sensors;
the soft protocol interface directly delivers the protocol communication packet to the software bus for communication and simulates the receiving and sending conditions of the airborne software and the simulation software.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (6)
1. An AR-based special case handling method is characterized by comprising the following processes: judging the state of the aircraft, judging the special situation of the aircraft, identifying an operation element of an aircraft cabin, judging a special situation handling flow, synthesizing an AR image and guiding the AR special situation handling; specifically, the state judgment based on the big data can be performed through data of an airborne sensor of the aircraft, when the state judgment is in the special situation state, the flow and the method can be extracted according to the special situation processing flow and method of the expert database, the AR image and the guide information of the special situation processing flow are fused by being matched with an aircraft cabin and an element image, and an operator is guided to perform special situation processing.
2. An AR-based idiom handling device according to claim 1, comprising onboard software, emulation software, support software, intermediate interface software, data interface, and soft protocol interface;
the airborne software and the simulation software both adopt the same architecture and comprise a software function module, a data module, an interactive data service module and an adapter module;
the airborne software is used for acquiring data of the aircraft sensor and processing the acquired data by an algorithm;
the soft protocol interface directly delivers the protocol communication packet to the software bus for communication and simulates the receiving and sending conditions of the airborne software and the simulation software.
3. The AR-based special case handling device according to claim 2, wherein the data module is a data pool for publishing and subscribing data, which is mainly connected to the software function module and the interactive data service module, on one hand, storing the publishing data of the software function module and receiving the subscribed data; and on the other hand, the interactive data service module is provided with local published data and receives local subscription data from a remote place.
4. The AR-based special case handling device according to claim 2, wherein the interactive data service module is a data interactive module manufactured according to a subscription and publication service mode, and is used for publishing local data and subscribing remote data.
5. The AR-based special case handling device according to claim 2, wherein one end of the adapter module is connected to the interactive data service module, and can receive published data from the interactive data service module, or provide subscription data for the interactive data service module, and provide interactive data for the adapter module; the other end is connected with a bus of a specific physical protocol, mainly translates data to be transmitted by the interactive data service module into a data packet required by the specific bus protocol, then transmits the data packet to the other end through the bus, receives the bus data at the other end, translates the data packet into a subscription data structure required by the interactive data service module, and triggers subscription processing.
6. The AR-based idiom treatment device of claim 2, wherein said sensor data acquisition and algorithmic processing includes but is not limited to radar, airspeed, barometric pressure, motor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110833683.9A CN113778222A (en) | 2021-07-23 | 2021-07-23 | AR-based special case handling method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110833683.9A CN113778222A (en) | 2021-07-23 | 2021-07-23 | AR-based special case handling method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113778222A true CN113778222A (en) | 2021-12-10 |
Family
ID=78835949
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110833683.9A Pending CN113778222A (en) | 2021-07-23 | 2021-07-23 | AR-based special case handling method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113778222A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102483863A (en) * | 2010-04-28 | 2012-05-30 | 新日铁系统集成株式会社 | Information processing system, information processing method, and program |
| CN106802576A (en) * | 2015-11-26 | 2017-06-06 | 中国飞行试验研究院 | A kind of flight failure decision method based on emulation |
| CN108628654A (en) * | 2018-03-30 | 2018-10-09 | 周瑞佳 | Electronic documentation system based on augmented reality and Internet of Things |
-
2021
- 2021-07-23 CN CN202110833683.9A patent/CN113778222A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102483863A (en) * | 2010-04-28 | 2012-05-30 | 新日铁系统集成株式会社 | Information processing system, information processing method, and program |
| CN106802576A (en) * | 2015-11-26 | 2017-06-06 | 中国飞行试验研究院 | A kind of flight failure decision method based on emulation |
| CN108628654A (en) * | 2018-03-30 | 2018-10-09 | 周瑞佳 | Electronic documentation system based on augmented reality and Internet of Things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11378988B2 (en) | Aircrew automation system and method with integrated imaging and force sensing modalities | |
| US11693407B2 (en) | Aircrew automation system and method | |
| CN108983756B (en) | Avionics flight control system ground comprehensive debugging and verification platform | |
| KR101083128B1 (en) | Image based uav flight test system and method | |
| EP2280359A1 (en) | Training method and system using augmented reality | |
| CN112817295B (en) | Test system and method of airplane management system | |
| CN110471313B (en) | Flight simulation subsystem of simulation aircraft | |
| CN113496635A (en) | Flight simulator and flight training simulation method | |
| CN105489083A (en) | Two-degree-of-freedom 360-degree flight simulation cockpit simulation motion platform | |
| CN105632271A (en) | Ground simulation training system for low-speed wind tunnel model flight experiment | |
| WO2024087764A1 (en) | Evtol navigation synthetic visual method and system | |
| CN114707304A (en) | Virtual-real combined multi-unmanned aerial vehicle perception avoidance verification system and method | |
| CN113778222A (en) | AR-based special case handling method and device | |
| CN110853410B (en) | Monitoring unit and monitoring method for airborne synthetic view guidance system | |
| Blake | The NASA Advanced Concepts Flight Simulator-A unique transport aircraft research environment | |
| EP4044179A1 (en) | On-board information assisting system and method | |
| CN112084630B (en) | Identification test method and system for D-level flight simulation training equipment | |
| Wisely | Digital twinning within a novel human-in-the-loop verification method for HUD safety-critical approach and landing | |
| CN112884807B (en) | Flight control action monitoring method and system based on infrared thermal imaging | |
| CN115933607A (en) | Automatic test system and method for airborne unmanned aerial vehicle anti-collision technology | |
| Ye et al. | Research on Augmented Reality Technology in the Training of Pre-flight Safety Inspect Process | |
| CN214846395U (en) | Four rotor unmanned aerial vehicle flight control systems | |
| CN217008440U (en) | Simulation training platform for function test of integrated avionics system | |
| Rave | Edge Processing of Image for UAS Sense and Avoidance | |
| Behringer et al. | System for synthetic vision and augmented reality in future flight decks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |