CN111325735A - Aero-engine insurance state detection method based on deep learning - Google Patents
Aero-engine insurance state detection method based on deep learning Download PDFInfo
- Publication number
- CN111325735A CN111325735A CN202010117406.3A CN202010117406A CN111325735A CN 111325735 A CN111325735 A CN 111325735A CN 202010117406 A CN202010117406 A CN 202010117406A CN 111325735 A CN111325735 A CN 111325735A
- Authority
- CN
- China
- Prior art keywords
- deep learning
- insurance state
- engine
- insurance
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Abstract
The invention discloses a method for detecting the safety state of an aero-engine based on deep learning, which is used for identifying fuses, seals and nuts in pictures, judging the attributes of the nuts such as the traction force direction of the fuses and the like, and judging whether the safety state is qualified. After the picture is obtained, feature extraction, target positioning and classification, target contour cutting, attribute feature extraction and attribute judgment are carried out through a deep learning model, and the result is output in a customized mode. The invention changes the traditional method of manually detecting and recording, can greatly improve the accuracy and efficiency of aviation insurance state detection, and can intensively store the identification photos for later verification.
Description
Technical Field
The invention belongs to the field of computer vision, and particularly relates to an aero-engine insurance state detection method based on deep learning.
Background
The detection and recording process of the safety state of the aero-engine is an essential part in the assembly process of an aero-plane, and the aero-manufacturing field has extremely high requirements on safety and reliability. The aircraft engine is a key component of aircraft flight equipment, and detection and recording work must be done. In the existing detection process, a manual detection mode of workers is generally adopted. The traditional identification process has the following disadvantages:
(1) the mode that relies on the workman to detect, the operation is very loaded down with trivial details, have wrong judgement's hidden danger, and can't guarantee the reliability of record information.
(2) The manual record only has the text information about the insurance state of the aero-engine, and the parameters such as specific angle, direction, position and the like cannot be obtained under the condition of no measurement, so that accurate data are difficult to provide for the detection.
(3) And the traditional detection process has low efficiency.
Disclosure of Invention
The invention aims to solve the defects of the prior art, and provides a method for detecting the safety state of an aero-engine based on deep learning, which comprises mechanical arm equipment for clamping a camera, the camera and electronic equipment running a deep learning algorithm model, and specifically comprises the following steps:
s1 model pretraining
Training a deep learning model by utilizing a large number of aero-engine insurance state pictures and marking contents aiming at the pictures;
s2 obtaining the insurance status picture of the aeroengine
Shooting an engine insurance state picture through camera equipment clamped by the mechanical arm;
s3 feature extraction
Performing feature extraction on the whole picture by utilizing a deep learning network;
s4 object location and classification
According to the extracted features, determining the approximate position of a target object (including a fuse, a seal, a nut and the like) by using a rectangular frame, and judging the category;
s5 target contour cutting
Extracting the outline information of the target object according to the extracted features, the position and the category information;
s6 attribute feature extraction
Extracting attribute features by combining the original picture according to the category and contour information of the target object;
s7 attribute judgment
Judging the specific attributes of the target object, such as the stress direction of the nut pulled by the fuse, according to the attribute characteristics, the position of the complex, the winding direction, the arrangement space sequence of the object and other factors;
s8 customizing the output result
And synthesizing all the information, summarizing an insurance state result, and outputting the result to the terminal equipment after processing the result.
Further, photographing is carried out on the safety state of the aerospace engine by using photographing equipment clamped by the mechanical arm.
Still further, a plurality of different scales of feature map data are returned using the detection model.
Further, coordinates, classes and contours of the target object on the aircraft engine are detected using the trained deep learning target detection model.
Further, in using the detection model, an attribute feature is extracted, and attribute feature determination is performed.
Furthermore, in using the detection model, the insurance status is determined according to the multiple attributes of the target object on the picture, and the output result is customized.
Has the advantages that: 1. according to the invention, pictures are acquired by photographic equipment, the identification and judgment are carried out through a deep learning algorithm, and the image data and the identification data are stored in a centralized manner, so that the traditional method of manually identifying and recording is changed, the accuracy and efficiency of aviation insurance state detection can be greatly improved, the aviation insurance state detection can be conveniently checked in the later period, and the accuracy of the data is further ensured; 2. compared with the traditional network, the invention combines the local characteristic and the integral characteristic to judge the attribute, is extremely specific to the attribute judgment of the insurance state class, integrates multiple factors such as the position of the object, the winding direction, the arrangement space sequence of the object and the like to judge and identify, and has higher judgment reliability. 3. According to the method, a large amount of parameters do not need to be adjusted, only engineering personnel need to maintain a model file trained through a deep learning technology, when new data are added, only the added data need to be added into a model to retrain the model, and then the model file is updated
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a general block diagram of Resnet with FPN architecture;
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, a method for detecting the insurance state of an aircraft engine based on deep learning includes the following steps:
in the present example, model pre-training is required. And in the model pre-training period, shooting engine insurance state pictures through camera equipment, and collecting a large amount of aeroengine insurance state data. And training the deep learning model by utilizing a large number of aero-engine insurance state pictures and sample description files made aiming at the pictures to construct an insurance state detection and identification model. Before the data is sent to the neural network training, data preprocessing operations, namely necessary operations such as data auditing, screening, data enhancement, normalization and the like are carried out.
In the embodiment of the invention, in the application process, an aviation engine insurance state picture needs to be acquired first, and the engine insurance state picture is shot through the camera equipment.
In the embodiment of the invention, Mask-RCNN is selected as a basic detection framework for identifying the position, classification and contour of the target object. In the Mask-RCNN detection framework, a convolution layer of a feature extraction part adopts a Resnet-101 network with an FPN structure. There are a convolutional layer 112 layer and a Batch-Normalization layer 104 layer. The Resnet-101 network with the FPN structure performs convolution operation on the text picture to generate a five-layer characteristic diagram, and the flow chart is shown in figure 2.
Resnet-101 with FPN structure is selected because it has excellent residual network structure, and can obtain better accuracy with small calculation amount. Meanwhile, due to the FPN structure, the target object with an overlarge or undersize area can be detected, and omission is avoided.
In the embodiment of the invention, the Region Probable Networks (RPN) generates a detection frame anchor, and traverses the feature map in the detection frame to generate the approximate position information of the target object.
In the embodiment of the invention, the box network and the mask network are used for generating the specific position and contour information of the target object and judging.
In the embodiment of the invention, the model for detecting the insurance state of the aircraft engine and the attribute network are used for combining the obtained information such as the category, the position, the mask code and the like of the detected object with the characteristic information of the whole image to form two data streams of local information and the whole image information, and the two data streams enter the deep learning attribute judgment network to finally obtain the attribute judgment information. Wherein the attribute network is a network that is custom built for insurance status detection.
In the embodiment of the invention, the safety state detection model of the aircraft engine uses smooth _ l1_ loss as a regression loss function and cross _ entry as a classification loss function.
And acquiring the recognition result, and outputting the result to the terminal equipment after processing the result.
The embodiment introduces the deep learning AI algorithm to the image processing for detecting the insurance state of the aircraft engine, greatly improves the insurance state detection efficiency of the aircraft engine, and reduces the error rate of identification. Does not need to regulate a large amount of parameters. Only engineering personnel are required to maintain a model file trained through the deep learning technology, when new data are added, the newly added data are only required to be added into the model to retrain the model, and then the model file is updated.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the scope of the invention, which is defined by the above description as merely a preferred embodiment of the invention, but shall not be construed as limiting the scope of the invention.
Claims (6)
1. The method for detecting the insurance state of the aircraft engine based on the deep learning is characterized by comprising mechanical arm equipment for clamping a camera, the camera and electronic equipment running a deep learning algorithm model, and specifically comprises the following steps:
s1 model pretraining
Training a deep learning model by utilizing a large number of aero-engine insurance state pictures and marking contents aiming at the pictures;
s2 obtaining the insurance status picture of the aeroengine
Shooting an engine insurance state picture through camera equipment clamped by the mechanical arm;
s3 feature extraction
Performing feature extraction on the whole picture by utilizing a deep learning network;
s4 object location and classification
According to the extracted features, determining the approximate position of a target object (including a fuse, a seal, a nut and the like) by using a rectangular frame, and judging the category;
s5 target contour cutting
Extracting the outline information of the target object according to the extracted features, the position and the category information;
s6 attribute feature extraction
Extracting attribute features by combining the original picture according to the category and contour information of the target object;
s7 attribute judgment
Judging the specific attributes of the target object, such as the stress direction of the nut pulled by the fuse, according to the attribute characteristics, the position of the complex, the winding direction, the arrangement space sequence of the object and other factors;
s8 customizing the output result
And synthesizing all the information, summarizing an insurance state result, and outputting the result to the terminal equipment after processing the result.
2. The method for detecting the insurance state of the aero-engine based on the deep learning as claimed in claim 1, wherein the insurance state of the aero-engine is photographed by a photographing device clamped by a mechanical arm.
3. The deep learning-based aircraft engine insurance state detection method of claim 1, wherein a plurality of different scales of feature map data are returned using the detection model.
4. The deep learning-based aircraft engine insurance state detection method of claim 1, wherein coordinates, classes and contours of target objects on the aircraft engine are detected using a trained deep learning target detection model.
5. The deep learning-based aircraft engine insurance state detection method according to claim 1, wherein in using the detection model, an attribute feature is extracted and an attribute feature determination is performed.
6. The method for detecting the insurance state of the aircraft engine based on the deep learning as claimed in claim 1, wherein in the detection model, the insurance state is judged according to multiple attributes of the target object on the picture, and the output result is customized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010117406.3A CN111325735A (en) | 2020-02-25 | 2020-02-25 | Aero-engine insurance state detection method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010117406.3A CN111325735A (en) | 2020-02-25 | 2020-02-25 | Aero-engine insurance state detection method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111325735A true CN111325735A (en) | 2020-06-23 |
Family
ID=71171112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010117406.3A Pending CN111325735A (en) | 2020-02-25 | 2020-02-25 | Aero-engine insurance state detection method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111325735A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117576635A (en) * | 2024-01-17 | 2024-02-20 | 中国石油集团川庆钻探工程有限公司 | Method for judging linear target tensioning state in video identification |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070031207A1 (en) * | 2005-08-04 | 2007-02-08 | Amir Rosenbaum | Lockwire header bolts |
| CN101263499A (en) * | 2005-07-11 | 2008-09-10 | 布鲁克斯自动化公司 | Intelligent condition monitoring and fault diagnostic system |
| CN101619683A (en) * | 2008-07-03 | 2010-01-06 | 通用汽车环球科技运作公司 | Ignition coil module fuse diagnosis |
| CN109086656A (en) * | 2018-06-06 | 2018-12-25 | 平安科技（深圳）有限公司 | Airport foreign matter detecting method, device, computer equipment and storage medium |
| CN109767497A (en) * | 2018-12-21 | 2019-05-17 | 杭州测质成科技有限公司 | A kind of detection method of automatic detection aerial blade surface quality |
| WO2019144337A1 (en) * | 2018-01-25 | 2019-08-01 | 大连理工大学 | Deep-learning algorithm-based self-adaptive correction method for full-envelope model of aero-engine |
| CN110243405A (en) * | 2019-06-25 | 2019-09-17 | 东北大学 | A kind of Aero-Engine Sensor Failure diagnostic method based on deep learning |
| CN110716520A (en) * | 2019-10-29 | 2020-01-21 | 中国航空工业集团公司西安飞行自动控制研究所 | Flight control servo actuator reliability evaluation modeling method based on multi-source information fusion |
-
2020
- 2020-02-25 CN CN202010117406.3A patent/CN111325735A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101263499A (en) * | 2005-07-11 | 2008-09-10 | 布鲁克斯自动化公司 | Intelligent condition monitoring and fault diagnostic system |
| US20070031207A1 (en) * | 2005-08-04 | 2007-02-08 | Amir Rosenbaum | Lockwire header bolts |
| CN101619683A (en) * | 2008-07-03 | 2010-01-06 | 通用汽车环球科技运作公司 | Ignition coil module fuse diagnosis |
| WO2019144337A1 (en) * | 2018-01-25 | 2019-08-01 | 大连理工大学 | Deep-learning algorithm-based self-adaptive correction method for full-envelope model of aero-engine |
| CN109086656A (en) * | 2018-06-06 | 2018-12-25 | 平安科技（深圳）有限公司 | Airport foreign matter detecting method, device, computer equipment and storage medium |
| CN109767497A (en) * | 2018-12-21 | 2019-05-17 | 杭州测质成科技有限公司 | A kind of detection method of automatic detection aerial blade surface quality |
| CN110243405A (en) * | 2019-06-25 | 2019-09-17 | 东北大学 | A kind of Aero-Engine Sensor Failure diagnostic method based on deep learning |
| CN110716520A (en) * | 2019-10-29 | 2020-01-21 | 中国航空工业集团公司西安飞行自动控制研究所 | Flight control servo actuator reliability evaluation modeling method based on multi-source information fusion |
Non-Patent Citations (2)
| Title |
|---|
| 孙军华 等: "Automatic Twining Direction Recognition of Lockwire on Engine" * |
| 罗毅等: "基于多摄像头的汽车保险丝盒检测与装配方法" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117576635A (en) * | 2024-01-17 | 2024-02-20 | 中国石油集团川庆钻探工程有限公司 | Method for judging linear target tensioning state in video identification |
| CN117576635B (en) * | 2024-01-17 | 2024-03-29 | 中国石油集团川庆钻探工程有限公司 | Method for judging linear target tensioning state in video identification |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105574550A (en) | Vehicle identification method and device | |
| CN112926405A (en) | Method, system, equipment and storage medium for detecting wearing of safety helmet | |
| CN111860277B (en) | Safety warning method for airspeed tube sleeve of civil aircraft based on color histogram feature | |
| CN112613569B (en) | Image recognition method, training method and device for image classification model | |
| CN111815576B (en) | Method, device, equipment and storage medium for detecting corrosion condition of metal part | |
| CN114170144A (en) | Power transmission line pin defect detection method, equipment and medium | |
| CN116503318A (en) | Aerial insulator multi-defect detection method, system and equipment integrating CAT-BiFPN and attention mechanism | |
| CN115063584A (en) | Pipeline steam leakage detection method and detection system based on deep learning | |
| CN112949457A (en) | Maintenance method, device and system based on augmented reality technology | |
| CN113906436A (en) | Method and apparatus for detecting smoke | |
| CN116071836A (en) | Deep learning-based crewman abnormal behavior detection and identity recognition method | |
| CN115019294A (en) | Pointer instrument reading identification method and system | |
| CN111832616A (en) | Method and system for identifying airplane model by using remote sensing image of multiple types of depth maps | |
| Xia et al. | An efficient and robust target detection algorithm for identifying minor defects of printed circuit board based on PHFE and FL-RFCN | |
| CN111325735A (en) | Aero-engine insurance state detection method based on deep learning | |
| CN116758493B (en) | Tunnel construction monitoring method and device based on image processing and readable storage medium | |
| CN114708582B (en) | AI and RPA-based electric power data intelligent inspection method and device | |
| CN111401142A (en) | Aero-engine blade metal surface etching character recognition method based on deep learning | |
| CN115719428A (en) | Face image clustering method, device, equipment and medium based on classification model | |
| CN115620083A (en) | Model training method, face image quality evaluation method, device and medium | |
| CN115410035A (en) | Air traffic controller unsafe behavior classification method based on monitoring video | |
| CN114565780A (en) | Target identification method and device, electronic equipment and storage medium | |
| CN113326951A (en) | Auxiliary detection device for aircraft outer surface cover screws and use method thereof | |
| CN113159230A (en) | Metallographic grade classification method based on deep learning | |
| CN110826473A (en) | Neural network-based automatic insulator image identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |