CN112634205A - Micro-feature detection algorithm based on image - Google Patents

Micro-feature detection algorithm based on image Download PDF

Info

Publication number
CN112634205A
CN112634205A CN202011412891.3A CN202011412891A CN112634205A CN 112634205 A CN112634205 A CN 112634205A CN 202011412891 A CN202011412891 A CN 202011412891A CN 112634205 A CN112634205 A CN 112634205A
Authority
CN
China
Prior art keywords
feature
layer
network
training
obtaining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011412891.3A
Other languages
Chinese (zh)
Inventor
孙长进
刘立庄
陈萍
孔彩凤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Kingen Intelligent Technology Co ltd
Original Assignee
Jiangsu Kingen Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Kingen Intelligent Technology Co ltd filed Critical Jiangsu Kingen Intelligent Technology Co ltd
Priority to CN202011412891.3A priority Critical patent/CN112634205A/en
Publication of CN112634205A publication Critical patent/CN112634205A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • G06T7/0006Industrial image inspection using a design-rule based approach
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a micro-feature detection algorithm based on an image, and particularly relates to the technical field of detection algorithms, wherein the technical scheme is as follows: s1, inputting a picture, obtaining a three-layer feature pyramid after a feature extraction network consisting of depth separable convolution, and obtaining a prediction result of the input picture, namely the offset, the category result and the confidence of each picture position frame after each layer of feature graph passes one layer of convolution layer; s2, obtaining the actual position of each prediction frame through the position offset; obtaining category information through logistic regression; s3, the confidence coefficient is the product of the probability of the object existing in the prediction frame and the intersection ratio of the real frame and the prediction frame, the invention has the advantages that: YOLOv3-Lite was detectable for either long range cracks, small cracks at rivets, or small cracks in aircraft engine interior blades that were dimly lit and cluttered in background.

Description

Micro-feature detection algorithm based on image
Technical Field
The invention relates to the field of detection algorithms, in particular to a micro-feature detection algorithm based on an image.
Background
The airplane fault can cause serious hidden danger to the flight safety if not eliminated in time, so that the airline companies are required to timely troubleshoot and maintain the structural damage of the airplane, the future development prospect of the civil aviation industry is wide at present, the number of airports and the number of airplanes in China are continuously increased, more and more groups select the airplanes as travel tools due to the convenience of the airplanes, but the airplane safety accidents can also occur every year, most of the reasons are the faults of the airplane, the mechanical property of the material can decay with the increase of the service life of the airplane, the crack damage is easy to generate, especially for the old airplane, the possibility of generating the damage is higher because of the long service life, therefore, in order to ensure the flight safety of the airplane, the airline company can perform maintenance work on one airplane for 3 times or more on average every day, wherein the maintenance work includes inspection before flight, after flight and inspection at stations for 1 time or more.
The prior art has the following defects: the existing method for detecting the damage of the airplane mainly comprises visual detection and nondestructive detection, wherein the nondestructive detection technology comprises ultrasonic detection, ray detection, penetration detection and other technologies, and in addition, detection means such as infrared, microwave, acoustic vibration, industrial CT and the like play an important role, the infrared detection technology is taken as an example, the method adopts the radiation principle to scan the temperature change caused by defects on the surface of the airplane and obtain the damage information according to the change, although the nondestructive detection technology is mature, the method still has problems, such as the incomplete training system of professional technicians, the problem of delay in the purchase and use of nondestructive detection equipment and the like, compared with the nondestructive detection, the visual detection is more convenient and easy to operate, the method is a main detection mode in the maintenance of the airplane and the cargo aircraft, the visual detection accounts for 90 percent and 80 percent respectively according to statistics, visual inspection is one of the most common ways to inspect and maintain the structures of an airplane, and all-around inspection is performed by crew members during the flying interval of the airplane, but the number of airplanes increases faster, and the training speed of the crew members is difficult to match.
Therefore, it is necessary to invent an image-based micro-feature detection algorithm.
Disclosure of Invention
The invention provides a micro-feature detection algorithm based on an image, which is characterized in that the offset, the category result and the confidence of a position frame of an input image are obtained by respectively passing each layer of feature map through one layer of convolution layer, then the actual position of each grid prediction frame can be obtained through the position offset, and category information is obtained through logistic regression, then the probability of crack damage in the frame is predicted according to a formula, and finally the final network prediction result is obtained through a non-maximum inhibition method, so that the problems of visual detection and nondestructive detection of a damage detection method are mainly solved.
In order to achieve the above purpose, the invention provides the following technical scheme: an image-based micro-feature detection algorithm, comprising the steps of:
s1, inputting a picture, obtaining a three-layer feature pyramid after a feature extraction network consisting of depth separable convolution, and obtaining a prediction result of the input picture, namely the offset, the category result and the confidence of each picture position frame after each layer of feature graph passes one layer of convolution layer;
s2, obtaining the actual position of each prediction frame through the position offset; obtaining category information through logistic regression;
s3, the confidence is the product of the probability of the object existing in the prediction box and the intersection of the real box and the prediction box:
Figure BDA0002816552000000021
s4, wherein Pr(Object) is the probability that there is fracture damage in the prediction box,
Figure BDA0002816552000000022
IoU of a prediction frame and a crack damage label frame, and finally obtaining a final network prediction result by a non-maximum value inhibition method;
s5, completing feature extraction of the Yolov3-Lite by depth separable convolution design, wherein the feature extraction structure of the Yolov3-Lite has 52 depth separable convolution layers;
s6, a three-layer feature pyramid of YOLOv3-Lite is used for detecting cracks with different sizes, the dimension of an input image is 416 x 416, a feature graph with 13 x 13 output dimension is obtained through the last layer of a feature extraction network and marked as f1, the 27 th layer is connected with f1 of up sampling, the feature graph with 26 x 26 output dimension is marked as f2, finally, the feature graph with 52 x 52 dimension is obtained through calculation of the 10 th layer of the network and is connected with f2 of up sampling, the connection part is constructed by a residual error network, the residual error network can combine low-level semantic information and high-level semantic information, under the connection mode, the network can effectively learn the crack features with different sizes, and finally, the three layers form the feature pyramid;
s7, dividing the data set into ten parts randomly in the training process, wherein nine parts are training sets, one part is a verification set, the network training comprises two stages, the first stage adopts a model pre-trained on the ImageNet data set, freezes the whole feature extraction layer, only trains the last three convolutional layers, the second stage trains the parameters of the whole network, the size of each batch in the first stage is set to be 10, the learning rate is set to be 0.001, the Adam optimization mode is adopted, the data set is called a round each time the network training is finished, if the network training is carried out for 3 rounds, the loss of the verification set is not reduced, the learning rate is reduced to be 1/10, the training is stopped when the error of the verification set is reduced to be less than 0 after 10 rounds of training, 300 rounds are iterated, the second stage adjusts the parameters of the whole model because the characteristic of the crack is far away from the characteristic of the object in the ImageNet, the Batchsize in the stage is set to be 4, the learning rate is set to 0.0001, adjusted in Adam's optimization and callback as in phase one, and the same end training condition is used for a total of 50 iterations in this phase.
Preferably, in S5, each depth separable convolutional layer includes a depth convolutional layer and a point-by-point convolutional layer, and all layers are followed by a Batch Normalization layer and a ReLU nonlinear layer.
Preferably, in S6, the receptive field of the 13 × 13 feature map is large for detecting large-sized cracks.
Preferably, in S6, the smaller receptive field of the 52 × 52 feature map can be used for detecting smaller cracks.
Preferably, in S6, the 26 × 26 feature map is between the 13 × 13 feature map and the 52 × 52 feature map.
Preferably, in S7, all training is run on the video card of NVIDIA Tesla K20 GPU, and the development environment is tensrflow 1.7.0.
The invention has the beneficial effects that:
the crack damage on the surface of the airplane body far away from the camera is small in proportion in the whole image, the position of the crack can still be accurately detected by the YOLOv3-Lite, and the YOLOv3-Lite can be detected no matter whether the crack is observed remotely and the crack is tiny at a rivet or the crack is tiny at the blade inside the airplane engine with dim light and messy background.
Drawings
FIG. 1 is a schematic diagram of a YOLOv3-Lite network structure provided by the present invention;
FIG. 2 is a schematic diagram of the results of the YOLOv3-Lite experiment provided by the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
Referring to fig. 1, the present invention provides an image-based micro-feature detection algorithm, which includes the following steps:
s1, inputting a picture, obtaining a three-layer feature pyramid after a feature extraction network consisting of depth separable convolution, and obtaining a prediction result of the input picture, namely the offset, the category result and the confidence of each picture position frame after each layer of feature graph passes one layer of convolution layer;
s2, obtaining the actual position of each prediction frame through the position offset; obtaining category information through logistic regression;
s3, the confidence is the product of the probability of the object existing in the prediction box and the intersection of the real box and the prediction box:
Figure BDA0002816552000000041
s4, wherein Pr(Object) is the probability that there is fracture damage in the prediction box,
Figure BDA0002816552000000051
IoU of a prediction frame and a crack damage label frame, and finally obtaining a final network prediction result by a non-maximum value inhibition method;
s5, the feature extraction part of YOLOv3-Lite is completed by depth separable convolution design, the feature extraction structure of YOLOv3-Lite has 52 depth separable convolution layers, each depth separable convolution layer comprises a depth convolution layer and a point-by-point convolution layer, and all the layers are followed by a Batch Normalization layer and a ReLU nonlinear layer.
S6, a three-layer feature pyramid of YOLOv3-Lite is used for detecting cracks with different sizes, the dimension of an input image is 416 x 416, a feature map with 13 x 13 output dimensions is obtained through the last layer of a feature extraction network and is marked as f1, the 27 th layer is connected with f1 of up sampling, the feature map with 26 x 26 output dimensions is marked as f2, finally, the feature map with 52 x 52 dimension is obtained through calculation of the 10 th layer of the network and is connected with f2 of up sampling, a connection part is constructed by a residual error network, the residual error network can combine low-level semantic information and high-level semantic information, under the connection mode, the network can effectively learn the crack features with different sizes, the last three layers form the feature pyramid, the sense field of the 13 x 13 feature map is large for detecting the cracks with large sizes, the sense field of the 52 x 52 feature map is smaller for detecting smaller cracks, the 26 × 26 feature map is intermediate between the 13 × 13 feature map and the 52 × 52 feature map.
Figure BDA0002816552000000052
Figure BDA0002816552000000061
Figure BDA0002816552000000071
Feature extraction network architecture
S7, dividing the data set into ten parts randomly in the training process, wherein nine parts are training sets, one part is a verification set, the network training comprises two stages, the first stage adopts a model pre-trained on the ImageNet data set, freezes the whole feature extraction layer, only trains the last three convolutional layers, the second stage trains the parameters of the whole network, the size of each Batch in the first stage is set to be 10, the learning rate is set to be 0.001, the Adam optimization mode is adopted, the data set is called a round each time the network training is finished, if the network training is carried out for 3 rounds, the loss of the verification set is not reduced, the learning rate is reduced to be 1/10, the training is stopped when the error of the verification set is reduced to be less than 0 after 10 rounds of training, 300 rounds are iterated, the second stage adjusts the parameters of the whole model because the characteristic of the crack is far away from the characteristic of the object in the ImageNet, the size of the Batch size in the stage is set to be 4, the learning rate is set to 0.0001, the learning rate is adjusted by adopting the optimization mode of Adam and the callback mode same as the first stage, the same training ending condition is adopted, 50 rounds of iteration are performed in the first stage, all training is performed on a video card of an NVIDIA Tesla K20 GPU, and the development environment is Tensorflow 1.7.0.
The using process of the invention is as follows: when the method is used, an experiment is firstly carried out on a video card of NVIDIA Tesla K20 GPU and a development environment Tensorflow 1.7.0, a data set is randomly divided into ten parts in a training process, nine parts are training sets, one part is a verification set, network training comprises two stages, the first stage adopts a model pre-trained on an ImageNet data set, a whole characteristic extraction layer is frozen, only the last three convolutional layers are trained, the second stage trains parameters of the whole network, the size of each batch in the first stage is set to be 10, the learning rate is set to be 0.001, an Adam optimization mode is adopted, the data set is called one round each time when the data set is traversed once, if the network training is carried out for 3 rounds, the loss of the verification set is not reduced, the learning rate is reduced to the original 1/10, the training is stopped when the error of the verification set is reduced to be less than 0 after the 10 rounds of training, 300 rounds of iteration are performed totally, and the characteristic of a crack is far different from the characteristic of an object in ImageNet, therefore, the second stage adjusts the whole model parameters, the size of the Batch size in the stage is set to be 4, the learning rate is set to be 0.0001, the learning rate is adjusted by adopting the optimization mode of Adam and the callback mode same as the stage one, and the same training ending condition is adopted, and 50 rounds of iteration are performed in the stage;
according to the experimental result of Yolov3-Lite, the left and right four columns are respectively divided into two groups, the left column of each group is the original image, and the right column is the detection result.
The above description is only a preferred embodiment of the present invention, and any person skilled in the art may modify the present invention or modify it into an equivalent technical solution by using the technical solution described above. Therefore, any simple modifications or equivalent substitutions made in accordance with the technical solution of the present invention are within the scope of the claims of the present invention.

Claims (6)

1. An image-based micro-feature detection algorithm, characterized by: the method comprises the following steps:
s1, inputting a picture, obtaining a three-layer feature pyramid after a feature extraction network consisting of depth separable convolution, and obtaining a prediction result of the input picture, namely the offset, the category result and the confidence of each picture position frame after each layer of feature graph passes one layer of convolution layer;
s2, obtaining the actual position of each prediction frame through the position offset; obtaining category information through logistic regression;
s3, the confidence is the product of the probability of the object existing in the prediction box and the intersection of the real box and the prediction box:
Figure FDA0002816551990000011
s4, wherein Pr(Object) is the probability that there is fracture damage in the prediction box,
Figure FDA0002816551990000012
IoU of a prediction frame and a crack damage label frame, and finally obtaining a final network prediction result by a non-maximum value inhibition method;
s5, completing feature extraction of the Yolov3-Lite by depth separable convolution design, wherein the feature extraction structure of the Yolov3-Lite has 52 depth separable convolution layers;
s6, a three-layer feature pyramid of YOLOv3-Lite is used for detecting cracks with different sizes, the dimension of an input image is 416 x 416, a feature graph with 13 x 13 output dimension is obtained through the last layer of a feature extraction network and marked as f1, the 27 th layer is connected with f1 of up sampling, the feature graph with 26 x 26 output dimension is marked as f2, finally, the feature graph with 52 x 52 dimension is obtained through calculation of the 10 th layer of the network and is connected with f2 of up sampling, the connection part is constructed by a residual error network, the residual error network can combine low-level semantic information and high-level semantic information, under the connection mode, the network can effectively learn the crack features with different sizes, and finally, the three layers form the feature pyramid;
s7, dividing the data set into ten parts randomly in the training process, wherein nine parts are training sets, one part is a verification set, the network training comprises two stages, the first stage adopts a model pre-trained on the ImageNet data set, freezes the whole feature extraction layer, only trains the last three convolutional layers, the second stage trains the parameters of the whole network, the size of each Batch in the first stage is set to be 10, the learning rate is set to be 0.001, the Adam optimization mode is adopted, the data set is called a round each time the network training is finished, if the network training is carried out for 3 rounds, the loss of the verification set is not reduced, the learning rate is reduced to be 1/10, the training is stopped when the error of the verification set is reduced to be less than 0 after 10 rounds of training, 300 rounds are iterated, the second stage adjusts the parameters of the whole model because the characteristic of the crack is far away from the characteristic of the object in the ImageNet, the size of the Batch size in the stage is set to be 4, the learning rate is set to 0.0001, adjusted in Adam's optimization and callback as in phase one, and the same end training condition is used for a total of 50 iterations in this phase.
2. An image-based micro-feature detection algorithm according to claim 1, wherein: in S5, each depth separable convolutional layer includes a depth convolutional layer and a point-by-point convolutional layer, and all layers are followed by a Batch Normalization and ReLU nonlinear layer.
3. An image-based micro-feature detection algorithm according to claim 2, wherein: in S6, the receptor field of the 13 × 13 feature map is large for detecting large-size cracks.
4. An image-based micro-feature detection algorithm according to claim 1, wherein: in S6, the smaller receptive field of the 52 × 52 feature map can be used to detect smaller cracks.
5. An image-based micro-feature detection algorithm according to claim 1, wherein: in S6, the 26 × 26 feature map is intermediate between the 13 × 13 feature map and the 52 × 52 feature map.
6. An image-based micro-feature detection algorithm according to claim 1, wherein: in S7, all training is run on the video card of NVIDIA Tesla K20 GPU, and the development environment is tensrflow 1.7.0.
CN202011412891.3A 2020-12-04 2020-12-04 Micro-feature detection algorithm based on image Pending CN112634205A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011412891.3A CN112634205A (en) 2020-12-04 2020-12-04 Micro-feature detection algorithm based on image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011412891.3A CN112634205A (en) 2020-12-04 2020-12-04 Micro-feature detection algorithm based on image

Publications (1)

Publication Number Publication Date
CN112634205A true CN112634205A (en) 2021-04-09

Family

ID=75307955

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011412891.3A Pending CN112634205A (en) 2020-12-04 2020-12-04 Micro-feature detection algorithm based on image

Country Status (1)

Country Link
CN (1) CN112634205A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110458161A (en) * 2019-07-15 2019-11-15 天津大学 A kind of mobile robot doorplate location method of combination deep learning
AU2020101011A4 (en) * 2019-06-26 2020-07-23 Zhejiang University Method for identifying concrete cracks based on yolov3 deep learning model

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2020101011A4 (en) * 2019-06-26 2020-07-23 Zhejiang University Method for identifying concrete cracks based on yolov3 deep learning model
CN110458161A (en) * 2019-07-15 2019-11-15 天津大学 A kind of mobile robot doorplate location method of combination deep learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
YADAN LI ET AL.: "YOLOv3-Lite: A Lightweight Crack Detection Network for Aircraft Structure Based on Depthwise Separable Convolutions", 《MDPI APPLIED SCIENCES》 *
燕红文;刘振宇;崔清亮;胡志伟;: "基于特征金字塔注意力与深度卷积网络的多目标生猪检测", 农业工程学报, no. 11 *

Similar Documents

Publication Publication Date Title
US9747564B1 (en) Aircraft maintenance and inspection with data analytics enhancement
CN109146849A (en) A kind of road surface crack detection method based on convolutional neural networks and image recognition
CN107709158A (en) System and method for checking surface automatically
EP2664548B1 (en) Damage assessment system and method of operating same
US9082209B1 (en) Method and apparatus for reworking inconsistencies on parts
CN115861263A (en) Insulator defect image detection method based on improved YOLOv5 network
CN111079645A (en) Insulator self-explosion identification method based on AlexNet network
Wanguo et al. Typical defect detection technology of transmission line based on deep learning
CN115147439A (en) Concrete crack segmentation method and system based on deep learning and attention mechanism
CN113744230B (en) Unmanned aerial vehicle vision-based intelligent detection method for aircraft skin damage
CN112634205A (en) Micro-feature detection algorithm based on image
CN117113066B (en) Transmission line insulator defect detection method based on computer vision
CN110659773A (en) Flight delay prediction method based on deep learning
Wagenmakers Aircraft performance engineering
CN114842315B (en) Looseness-prevention identification method and device for lightweight high-speed railway hub gasket
CN116664549A (en) Photovoltaic power station hot spot defect detection method based on feature perception
CN116206222A (en) Power transmission line fault detection method and system based on lightweight target detection model
Shao et al. Aircraft Skin Damage Detection and Assessment From UAV Images Using GLCM and Cloud Model
Bouarfa et al. Automated Drone-Based Aircraft Inspection
CN113870199A (en) Method for identifying defect detection of aircraft skin
Jiachen et al. Civil aircraft surface defects detection based on histogram of oriented gradient
Shailaja et al. A survey on autonomous damage detection on aircraft surfaces using deep learning models
Yasuda et al. Automated visual inspection of aircraft exterior using deep learning
Duvar et al. A Review on Visual Inspection Methods for Aircraft Maintenance
Xiaomei et al. Risk Index Prediction of Civil Aviation Based on Deep Neural Network.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination