CN114913445A - Intelligent inspection operation system for airplane route based on image recognition - Google Patents
Intelligent inspection operation system for airplane route based on image recognition Download PDFInfo
- Publication number
- CN114913445A CN114913445A CN202111184177.8A CN202111184177A CN114913445A CN 114913445 A CN114913445 A CN 114913445A CN 202111184177 A CN202111184177 A CN 202111184177A CN 114913445 A CN114913445 A CN 114913445A
- Authority
- CN
- China
- Prior art keywords
- image recognition
- video
- inspection
- image
- inspection operation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Library & Information Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides an intelligent inspection operation system for an airplane route based on image recognition. It is characterized by comprising the following steps: the data acquisition module acquires a video image of an airplane route inspection operation process, transmits the video image to the video processing module in the processor in real time, and transmits each frame of video image subjected to coding and decoding processing to the image identification module; the image recognition module automatically recognizes the defect faults, missed detection and illegal behaviors of the aircraft equipment components in the inspection operation area through a neural network image recognition analysis algorithm, and transmits inspection video images and image recognition analysis result data to the interactive terminal and the supervision background in real time; the interactive terminal realizes voice broadcast reminding and detail checking of fault defects, missed detection and illegal behaviors; the monitoring background realizes remote real-time monitoring of the patrol operation behavior and guidance of troubleshooting operation, and an index database of the patrol operation information of the airplane route is constructed so as to backtrack, analyze and query historical data of the patrol process.
Description
Technical Field
The invention relates to the operation and maintenance industry of airplane routes, in particular to an intelligent inspection operation system of an airplane route based on image recognition.
Background
At present, the inspection operation of an airplane route comprises the inspection operation of short stop, before flight, after flight and departure, the traditional visual inspection operation mode is adopted, the whole process is manually detected, a paper pen records a work sheet, and equipment components in different areas of the airplane, such as a nose, an engine, an undercarriage, a wheel cabin, a wing, a tail and the like, are inspected one by one according to the operation flow of the inspection operation of the winding machine.
Because the number of equipment components needing to be checked in the process of the coiling polling operation is very large, and the operation time window period is short, the polling operation quality of the traditional visual inspection completely depends on the experience and responsibility of polling personnel, so that the problems that the condition of the completion quality of the polling operation cannot be traced, whether the missed inspection exists or not cannot be confirmed, and a work single card cannot be ensured to be used as a tag exist, the economic losses such as the return of an airplane, the delay of a flight and the like are often caused, and even the flight safety accidents can be caused.
The intelligent inspection operation system for the airplane route based on image recognition, disclosed by the invention, provides real-time operation guidance for inspection personnel at an external station to avoid missing inspection, and improves the efficiency and quality of inspection operation; an effective supervision tool is provided for quality safety management personnel, and safety accident tracing and risk avoidance are realized; the basic database, the behavior tag recording database, the defect fault characteristic database and the like of the airline plane can be unified and integrated, statistical analysis of big data is realized, and the comprehensive management capability of airline operation and maintenance data is effectively improved.
Disclosure of Invention
In order to overcome the problems or at least partially solve the problems, the embodiment of the invention provides an intelligent inspection operation system for an airplane route based on image recognition.
The embodiment of the invention is realized by the following steps:
an aircraft route intelligent inspection operation system based on image recognition comprises: the data acquisition module is used for acquiring video images of airplane equipment components and inspection operation behaviors in the course of route inspection operation and transmitting the video images to the processor in real time; the processor is used for processing, storing and analyzing the video images transmitted by the data acquisition module, automatically identifying the defect, fault, missing inspection and illegal operation behaviors of the airplane equipment components in the video images in real time, and transmitting the image identification results to the interactive terminal and the supervision background in real time; the interactive terminal is used for controlling the start and stop of the processor, is connected with the processor to check the video image recognition result of the inspection operation, converts the text prompt information in the image recognition result into corresponding voice content and carries out real-time voice broadcast reminding; and the supervision background is used for being connected with the processor to remotely check the real-time video image of the inspection operation, constructing an aircraft route inspection operation information index database and realizing backtracking check of the historical data record of the inspection operation.
In some embodiments of the invention, a processor comprises a video processing module, wherein the video processing module is used for receiving video image information of an airplane equipment component and a patrol operation behavior, performing H.265 compression coding storage, video stream MJPEG continuous frame screenshot, video stream real-time transcoding and transmission protocol format conversion and distribution processing, and transmitting the video image information subjected to data processing to an image identification module in real time; and the image recognition module is used for performing feature matching recognition judgment on the input continuous frame screenshots of the video stream MJPEG by using a pre-trained feature model library of airplane equipment components, defect faults and operation behaviors according to the video image information transmitted by the image processing module based on an image recognition analysis algorithm of a neural network, and outputting an image recognition result.
In some embodiments of the invention, the video processing module at least comprises an aircraft equipment component, a defect fault and a characteristic model library of operation behaviors, wherein the aircraft equipment component at least comprises a characteristic model of a nose, a radome, an engine, a landing gear, a tire, a wheel cabin, a wing, an empennage, a fuselage and a hatch door; the fault at least comprises a characteristic model of pits, lightning stroke, dirt, defect and bird strike; the operation behaviors at least comprise characteristic models of target omission detection, insufficient or overtime inspection duration and incomplete inspection coverage areas.
In some embodiments of the invention, an intelligent inspection operation system for an aircraft route based on image recognition is provided, wherein the image recognition result in the image recognition module at least comprises the name and the area of an aircraft device component, the name of a fault defect classification, the name of a violation classification, a video screenshot image with a positioning label, a timestamp and character labeling prompt information.
In some embodiments of the invention, the aircraft route inspection operation information index database constructed by the supervision background at least comprises unified integrated inspection data, a public basic database, a route aircraft basic database, an inspection process behavior database, a defect fault database and the like, and the information index data at least comprises an aircraft number, a work order number, a work card number, operation starting and stopping time, an inspection video file access path and an image identification result.
Some embodiments of the invention have at least the following advantages or benefits:
1. the system of the invention realizes automatic real-time recognition of the defect fault of the airplane equipment component in the course of route inspection operation by using the neural network image recognition technology, and realizes real-time voice prompt in the whole course, thereby improving the recognition efficiency of the defect of the airplane component and greatly reducing the error rate and the omission factor of manual recognition.
2. The system realizes remote real-time whole-course monitoring of the inspection operation behavior, automatically identifies the illegal operation behavior, can effectively improve the working quality of the inspection operation, and greatly reduces the flight safety risk and hidden danger caused by the illegal operation.
3. The system provided by the invention can be used for retrieving and inquiring the flight route inspection operation behavior and the video image data of the fault defects of the airplane equipment, and rapidly checking the history records of the fault defects, thereby greatly improving the working efficiency of the inspection operation and solving the problem that the safety accidents cannot be traced.
4. The system of the invention improves the data management capability, integrates the operation and maintenance data of the airplane route uniformly, establishes a route airplane basic database, a routing inspection operation behavior database, a defect fault database and the like, and realizes the statistical analysis of the big operation and maintenance data of the route.
5. The system has flexible expandability, has rich interfaces and expansion capability, can flexibly expand the defect fault identification types, and provides a general identification service for a three-party system.
Drawings
To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the embodiments will be briefly described below, it should be understood that the following drawings only identify certain implementations of the present invention and therefore should not be considered as limiting the scope, and those skilled in the art may also obtain other drawings based on the drawings without inventive labor.
FIG. 1 is a block diagram of an aircraft route intelligent inspection operating system based on image recognition;
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that: like reference numerals and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments and features of the embodiments described below can be combined with each other without conflict.
Examples
Referring to fig. 1, the present embodiment provides an intelligent inspection operation system for an aircraft route based on image recognition, including:
the data acquisition module is used for acquiring video image information of aircraft equipment parts such as a nose, a radome, wings, an engine, a tail wing, a landing gear, an engine room, a cabin cover plate and an antenna in the course of route inspection operation in real time, and video image information of inspection operation behaviors such as equipment inspection areas, time length and operation steps; the invention adopts the mechanical anti-shake ear-hung camera, is worn by inspection personnel in the operation process, and can transmit video stream data with 4K high-definition resolution and MJPEG format in real time.
And the processor is connected with the data acquisition module and used for processing, storing and analyzing the video images transmitted by the data acquisition module, automatically identifying the defect faults, missed detection and illegal operation behaviors of the airplane equipment components in the video images in real time, and transmitting the image identification results to the interactive terminal and the supervision background in real time. The processor comprises a video processing module and an image recognition module; the video processing module is used for receiving video stream data with 4K high-definition resolution and MJPEG format transmitted by the data acquisition module, performing H.265 compression coding to obtain high-definition video with resolution of 1920 x 1080 and 30 frames, storing a local disk, and transmitting the video stream data to the interactive terminal and the supervision background in real time according to RTSP transmission protocol format; capturing each frame of a video stream, and transmitting the 4k high-definition MJEPG format continuous frame captured images to an image identification module in real time; and the image recognition module is used for performing feature matching recognition judgment on the input video stream MJPEG continuous frame screenshots according to the video stream MJPEG continuous frame screenshots transmitted by the image processing module and based on an image recognition analysis algorithm of a neural network by using a pre-trained feature model library of airplane equipment components, defect faults and operation behaviors, and outputting an image recognition result. The image recognition result comprises the name and the area of the airplane equipment component, the fault defect classification name, a video screenshot image with a positioning label, a timestamp and character labeling prompt information; the invention adopts a portable edge computing host to realize 4K high-definition video stream input and real-time coding and decoding conversion output, realizes an image recognition analysis algorithm and GPU real-time processing and calculation based on a neural network, and outputs an image recognition result.
The interactive terminal is connected with the processor and controls the start and stop of the processor by sending a signaling message; the image recognition module is communicated with the image recognition module according to an HTTP transmission protocol to acquire image-text information in an image recognition result, decode and check a video screenshot in the image-text information in real time, and convert the text prompt information into corresponding voice content for instant voice broadcasting; according to the invention, a waterproof, dustproof and anti-falling three-prevention smart phone terminal is adopted, and the WiFi network of the smart phone terminal is connected with the processor, so that the start-stop operation of the processor is realized, and the graphic information and the voice output in the image recognition result are checked.
The monitoring background is connected with the processor, is communicated with the video processing module according to an RTSP transmission protocol, decodes and displays the polling operation video stream image with 1920 x 1080 resolution, 30 frames and H.265 format in real time, and the time delay of the video image is less than 600 ms; the system is communicated with an image identification module according to an HTTP transmission protocol to acquire and store an image identification result, and on the basis, an aircraft route inspection operation information index database is constructed to realize backtracking check of historical data records of inspection operation, wherein the information index database at least comprises an aircraft number, a work order number, a work card number, operation starting and stopping time, an inspection video file access path and an image identification result; the cloud server is remotely connected with the processor through the network, so that the operation behavior of the airplane inspection can be remotely monitored in real time, the defect fault level can be judged in an auxiliary mode, the troubleshooting operation can be guided, and the historical information data of the defect fault and the operation behavior in the inspection process can be searched through backtracking analysis.
Furthermore, the intelligent acquisition and intelligent analysis of the video images of the inspection operation and the key capabilities of automatic output of image data analysis results and the like can be realized through the image recognition technology based on the neural network.
Furthermore, the system realizes intelligent real-time identification of the defect faults of the aircraft equipment components in the route inspection operation process, and the whole process is prompted in real time through voice, so that the identification efficiency of the defects of the aircraft components is improved, and the error rate and the omission factor of manual identification are greatly reduced.
Furthermore, the system realizes remote real-time monitoring of the violation behaviors of the inspection operation and multi-party communication, can effectively improve the working quality of the inspection operation, and greatly reduces the flight safety risk and hidden danger caused by the violation operations.
Furthermore, the system realizes the traceability analysis and retrieval query of the flight line inspection operation behavior and the video image data of the fault defect of the airplane equipment, and can quickly check the history record of the fault defect and guide the troubleshooting operation, thereby greatly improving the working efficiency of the polling operation and solving the problem that the safety accident cannot be traced.
Furthermore, the system improves the data management capability, integrates airplane route operation and maintenance data uniformly, establishes a route airplane basic database, a routing inspection operation behavior database, a defect and troubleshooting guide database and the like, and realizes statistical analysis of route operation and maintenance big data.
Furthermore, the system has flexible expandability, has rich interfaces and expansion capability, can flexibly expand defect identification types, and provides general identification service for a three-party system.
Furthermore, the system realizes the tracking and monitoring of the real-time state and the routing inspection behavior of the airplane components; and the inspection plan and the operation process are subjected to unified and standardized management. The defect hidden danger characteristic map library is automatically learned and accumulated, the defect hidden danger is accurately identified and verified, and application development of defect hidden danger intelligent identification technology in the airline operation and maintenance field is promoted.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made without departing from the spirit and principle of the present invention shall fall within the protection scope of the present invention.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (5)
1. An intelligent inspection operation system for airplane route based on image recognition is characterized by comprising
The data acquisition module is used for acquiring video images of aircraft equipment components and inspection operation behaviors in the course of route inspection operation and transmitting the video images to the processor in real time;
the processor is used for processing, storing and analyzing the video images transmitted by the data acquisition module, automatically identifying the defect, fault, missing inspection and illegal operation behaviors of the airplane equipment components in the video images in real time, and transmitting the image identification results to the interactive terminal and the supervision background in real time;
the interactive terminal is used for controlling the start and stop of the processor, is connected with the processor to check the video image recognition result of the inspection operation, converts the text prompt information in the image recognition result into corresponding voice content and carries out real-time voice broadcast reminding;
and the supervision background is used for being connected with the processor to remotely check the real-time video image of the inspection operation, constructing an aircraft route inspection operation information index database and realizing backtracking check of the historical data record of the inspection operation.
2. The intelligent patrol inspection operation system for the aircraft route based on image recognition is characterized in that the processor comprises a video processing module and an image recognition module, wherein:
the video processing module is used for receiving video image information of airplane equipment components and inspection operation behaviors, performing H.265 compression coding storage, video stream MJPEG continuous frame screenshot, video stream real-time transcoding and transmission protocol format conversion and distribution processing, and transmitting the video image information subjected to data processing to the image identification module in real time;
the image recognition module is used for performing feature matching recognition judgment on input video stream MJPEG continuous frame screenshots according to video image information transmitted by the image processing module and based on an image recognition analysis algorithm of a neural network by using a pre-trained feature model library of airplane equipment components, defect faults and operation behaviors, and outputting an image recognition result.
3. The intelligent inspection operation system for the aircraft route based on the image recognition is characterized in that the video processing module at least comprises a characteristic model library of aircraft equipment components, defect faults and operation behaviors; the aircraft equipment components at least comprise characteristic models of a nose, a radome, an engine, an undercarriage, tires, an engine room, wings, an empennage, a fuselage and a hatch cover door; the fault defect at least comprises a characteristic model of a pit, a lightning stroke, dirt, defect and bird strike; the operation behaviors at least comprise characteristic models of target omission detection, insufficient or overtime inspection duration and incomplete inspection coverage areas.
4. The intelligent inspection operation system for the aircraft airline based on the image recognition of claim 2, wherein the image recognition result in the image recognition module at least comprises the names and areas of aircraft equipment components, the names of fault and defect classifications, the names of violation classifications, video screenshot images with positioning labels, timestamps and text labeling prompt messages.
5. The intelligent airplane route inspection operation system based on image identification according to claim 1, wherein the airplane route inspection operation information index database constructed by the supervision background at least comprises unified integrated inspection data, a public basic database, a route airplane basic database, an inspection process behavior database, a defect fault database and the like are established, and the information index data at least comprises an airplane number, a work order number, a work card number, operation starting and stopping time, an inspection video file access path and an image identification result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111184177.8A CN114913445A (en) | 2021-10-11 | 2021-10-11 | Intelligent inspection operation system for airplane route based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111184177.8A CN114913445A (en) | 2021-10-11 | 2021-10-11 | Intelligent inspection operation system for airplane route based on image recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114913445A true CN114913445A (en) | 2022-08-16 |
Family
ID=82761958
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111184177.8A Pending CN114913445A (en) | 2021-10-11 | 2021-10-11 | Intelligent inspection operation system for airplane route based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114913445A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557570A (en) * | 2024-01-12 | 2024-02-13 | 中数智科(杭州)科技有限公司 | Rail vehicle abnormality detection method and system |
-
2021
- 2021-10-11 CN CN202111184177.8A patent/CN114913445A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117557570A (en) * | 2024-01-12 | 2024-02-13 | 中数智科(杭州)科技有限公司 | Rail vehicle abnormality detection method and system |
| CN117557570B (en) * | 2024-01-12 | 2024-05-24 | 中数智科(杭州)科技有限公司 | Rail vehicle abnormality detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110826538B (en) | Abnormal off-duty identification system for electric power business hall | |
| CN111310947A (en) | Building facility operation and maintenance method, equipment, storage medium and system based on 5G | |
| CN108515991B (en) | Receiving and dispatching train standardized operation detecting system | |
| CN106162089A (en) | A kind of power-line patrolling image automatic identification analyzes system and the method for analysis thereof | |
| CN110807460B (en) | Transformer substation intelligent patrol system based on image recognition and application method thereof | |
| CN112113978A (en) | Vehicle-mounted tunnel defect online detection system and method based on deep learning | |
| CN111294561A (en) | Video-based online patrol method, electronic device, storage medium and system | |
| CN111259748A (en) | Edge calculation and communication system for video monitoring | |
| CN104268525A (en) | Vehicle monitoring method and system based on face recognition | |
| CN112478483B (en) | Automatic monitoring and early warning method and system for hazardous chemical substance normal-pressure storage tank | |
| CN211184122U (en) | Intelligent video analysis system for linkage of railway operation safety prevention and control and large passenger flow early warning | |
| CN114913445A (en) | Intelligent inspection operation system for airplane route based on image recognition | |
| CN115695541A (en) | Method, device and equipment for monitoring dot polling based on edge calculation and storage medium | |
| CN110992512A (en) | Power transmission line inspection arrival rate online monitoring system and method based on electronic tag | |
| CN107564272B (en) | Terminal reporting alarm automatic alarm handling device and method based on recorder management platform | |
| CN112686130B (en) | Wisdom fishing boat supervision decision-making system | |
| CN104574729A (en) | Alarming method, device and system | |
| CN114445782A (en) | Power transmission line image acquisition system based on edge AI and Beidou short messages | |
| CN112260402B (en) | Monitoring method for state of intelligent substation inspection robot based on video monitoring | |
| CN116580362B (en) | Transmission operation cross-system fusion data acquisition method and digital asset processing system | |
| CN117037289A (en) | Personnel operation behavior recognition method and system based on machine vision | |
| CN112528791A (en) | Anti-intrusion system and method for railway station yard | |
| CN113392706A (en) | Device and method for detecting smoking and using mobile phone behaviors | |
| CN113095160A (en) | Power system personnel safety behavior identification method and system based on artificial intelligence and 5G | |
| CN117636225A (en) | Image recognition-based aircraft inspection safety pin recognition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |