CN112669292B - A method for detecting and classifying surface defects of aircraft skin spray paint - Google Patents

Abstract

The invention discloses a method for detecting and classifying surface defects of aircraft skin spray paint, comprising the following steps: S1, collecting an image of the aircraft skin spray paint surface; S2, classifying defects on the image collected in step S1 based on the surface smoothness Detection; S3. Multi-classification detection of surface defects of aircraft skin spray paint based on a simplified GoogLeNet convolutional neural network model; S4. Output multi-classification results. The method for detecting and classifying surface defects of aircraft skin spray paint provided by the present invention integrates the traditional image defect detection two-classification method and the deep learning-based GoogLeNet convolutional neural network model of the defect multi-classification framework. Combined, it not only effectively improves the computational efficiency and robustness of the algorithm, but also effectively shortens the execution time of the algorithm, improves the detection accuracy of aircraft skin surface defects, and can effectively meet the requirements of intelligent detection and detection of aircraft skin paint surface defects. Classification needs.

Description

A method for detecting and classifying surface defects of aircraft skin spray paint

technical field

The invention relates to a method for realizing detection and classification of surface defects of aircraft skin spray paint, and belongs to the technical field of defect detection.

Background technique

The surface coating process of aircraft skin is complex. Once the coating and the base material are debonded and cracks occur during use, it will not only affect the appearance, but also seriously affect the operation safety of the aircraft. Therefore, the detection of aircraft skin surface defects is to inspect the aircraft. An important means of factory quality. There are many types of surface defects on aircraft skins, among which the more typical ones include particles, scratches, garbage, flow, orange peel, and super-thick flanging. In most cases, the size of surface defects of aircraft skin is measured in millimeters, and the defect area is small and unevenly dispersed. Traditional defect detection mainly relies on manual visual inspection, which not only has limitations such as high labor cost, low work efficiency, and poor real-time performance, but also the inspection effect is easily affected by artificial subjective factors such as vision and physical status. With the improvement of the degree of automation of aircraft manufacturing, the artificial visual detection method can no longer meet the needs of development. Therefore, there is an urgent need in the field to realize automatic and intelligent detection methods for aircraft skin surface defects.

At present, the automatic detection of aircraft skin surface defects is still in its infancy in China. The existing detection methods often have problems such as large amount of calculation, poor accuracy and low reliability, and it is difficult to meet industrial requirements. Therefore, how to shorten the The algorithm execution time and the improvement of the detection accuracy of aircraft skin surface defects have become technical problems that need to be solved urgently.

SUMMARY OF THE INVENTION

In view of the above problems existing in the prior art, the purpose of the present invention is to provide a method for detecting and classifying surface defects of aircraft skin spray paint.

For realizing the above-mentioned purpose of the invention, the technical scheme adopted in the present invention is as follows:

A method for detecting and classifying surface defects of aircraft skin spray paint, comprising the following steps:

S1. Collect an image of the painted surface of the aircraft skin;

S2. Perform two-category defect detection on the image collected in step S1 based on the surface smoothness, that is: first preprocess the collected image; then perform block detection and positioning on the preprocessed image to find pixels with defects; then use The information entropy function estimates the surface smoothness of the pixels with defects and the pixels of the whole image, and judges whether the image has defects based on the surface smoothness. ;

S3. Perform multi-classification detection on the surface defects of aircraft skin spray paint based on the simplified GoogLeNet convolutional neural network model, namely: first construct the simplified GoogLeNet convolutional neural network model, and then use the constructed simplified GoogLeNet convolutional neural network model to perform the second step in step S2. The defect images obtained by classification are subjected to multi-classification detection; the GoogLeNet convolutional neural network model includes five layers, and the five layers are the first layer, the second layer, the third layer, the fourth layer, and the fifth layer. , the first and second layers are ordinary convolutional layers, the third layer consists of Inception-A and Inception-B modules, and the fourth layer consists of Inception-A, Inception-B, Inception-C, Inception-D and Inception- The E module is composed, and the fifth layer is composed of Inception-A and Inception-B modules; the simplified GoogLeNet convolutional neural network model refers to the GoogLeNet convolutional neural network model that removes the Inception-B and Inception-C modules; in the present invention , the adopted simplified GoogLeNet convolutional neural network model has a faster running speed and can effectively shorten the algorithm execution time under the premise that the accuracy rate is not reduced compared with the GoogLeNet convolutional neural network model.

S4, output multi-classification results.

In an embodiment, in step S2, the preprocessing of the image collected in step S1 includes: Gaussian filtering (to denoise the image), histogram equalization (to enhance the image), Preprocessing of Ostu adaptive threshold segmentation and erosion.

In one embodiment, in step S2, Blob is used to perform block detection on the preprocessed image.

A preferred solution, step S2, the operation of performing two-class defect detection on the image collected in step S1 based on the surface smoothness is as follows: first, the image collected in step S1 is subjected to Gaussian filtering, histogram equalization, Ostu adaptive threshold segmentation and Preprocessing of corrosion; then use Blob to perform block detection on the preprocessed image to find pixels with defects; then use the information entropy function to estimate the surface smoothness of the pixels with defects and the pixels of the whole image, set the surface smoothness The degree threshold (the surface smoothness threshold represents the number of defective pixels, when the number of defective pixels reaches this number, it is determined that the image is defective) to determine whether the image has defects, and the image is divided into defective images and non-defective images. , to achieve binary classification defect detection for images.

In one embodiment, the following operations are specifically included in step S3:

S31. Data set production: use the collected images of the aircraft skin spray paint surface for data set production, and divide the data set into a training sample data set and a test sample data set;

S32. Build the GoogLeNet convolutional neural network model: The GoogLeNet convolutional neural network model includes five layers, and the five layers are the first layer, the second layer, the third layer, the fourth layer, and the fifth layer. Among them, the first layer layer and the second layer are ordinary convolutional layers, the third layer consists of Inception-A and Inception-B modules, and the fourth layer consists of Inception-A, Inception-B, Inception-C, Inception-D and Inception-E modules , the fifth layer consists of Inception-A and Inception-B modules;

S33. Build a simplified GoogLeNet convolutional neural network model: remove the Inception-B and Inception-C modules of the GoogLeNet convolutional neural network model in step S32, and build a simplified GoogLeNet convolutional neural network model;

S34. Train the simplified GoogLeNet convolutional neural network model: input the images in the training sample data set into the simplified GoogLeNet convolutional neural network model for feature recognition to obtain the optimal simplified GoogLeNet convolutional neural network model;

S35. Test the simplified GoogLeNet convolutional neural network model: input the test sample data set into the trained simplified GoogLeNet convolutional neural network model to verify the accuracy;

S36. Multi-classification detection: The defect images classified in step S2 are output to the simplified GoogLeNet convolutional neural network model after the test, and the multi-classification detection of surface defects of aircraft skin spray paint is realized by simplifying the GoogLeNet convolutional neural network model.

In a preferred solution, the data set produced in step S31 includes images of various common defect types on the painted surface of the aircraft skin, and the common defect types include but are not limited to bubbles, particles, garbage, flow, and flanging. In the process of making the dataset, the specific defect types of the images can be distinguished by professional surface painting workers.

A preferred solution, the simplified GoogLeNet convolutional neural network model constructed in step S33 is the first convolutional layer, the second convolutional layer, the third Inception-A module, the fourth Inception-A module, and the fourth Inception-D module, the fourth Inception-E module, the fifth Inception-A module; the operations of each convolutional layer and module are as follows:

The first convolution layer first uses 7*7 convolution kernels, the number of convolution kernels is 64, and then uses 3*3 kernels for maximum pooling, and the sliding step size is 2;

The second convolution layer first uses 3*3 convolution kernels, the number of convolution kernels is 192, and the sliding step size is 1, and then uses 3*3 kernels for maximum pooling, and the sliding step size is 2;

The third Inception-A module includes four branches. The four branches are: use a 1*1 convolution kernel, the number of convolution kernels is 64, and the sliding step size is 1; 2) First, use a 1*1 convolution Kernel, the number of convolution kernels is 96, and then a 3*3 convolution kernel is used, the number of convolution kernels is 128, and the sliding step size is 1; 3) First, a 1*1 convolution kernel is used, the convolution kernel The number is 16, and then 5*5 convolution kernels are used, the number of convolution kernels is 32, and the sliding step size is 1; 4) First, use 3*3 kernels for maximum pooling, and then use 1*1 volumes Product kernel, the number of convolution kernels is 32, and the sliding step size is 1; finally connect the output results of the four branches, and continue to the next step;

The fourth Inception-A module is the same as the third Inception-A module;

The fourth Inception-D module is the same as the third Inception-A module;

The fourth Inception-E module includes five branches. The five branches are as follows: 1) Use a 1*1 convolution kernel, the number of convolution kernels is 64, and the sliding step size is 1; 2) First, use a 1*1 convolution kernel Convolution kernel, the number of convolution kernels is 96, and then a 3*3 convolution kernel is used, the number of convolution kernels is 128, and the sliding step size is 1; 3) First, a 1*1 convolution kernel is used, the volume The number of accumulation kernels is 16, then 5*5 convolution kernels are used, the number of convolution kernels is 32, and the sliding step size is 1; 4) First use 3*3 kernels for maximum pooling, and then use 1*1 The number of convolution kernels is 32, and the sliding step size is 1; 5) First use 5*5 kernels for average pooling, the sliding step size is 3, and then use 1*1 convolution kernels, the volume The number of product kernels is 32, the sliding step size is 1, and then the fully connected layer is used twice, and then the Softmax activation function is used to assist the classification; finally, the output results of the first five branches are connected, and the next step is continued;

The fifth Inception-A module is the same as the third Inception-A module;

In each convolutional layer and module, the ReLU activation function is used after using the convolution kernel and pooling operation.

Compared with the prior art, the present invention has the following significant beneficial effects:

The method for detecting and classifying surface defects of aircraft skin spray paint provided by the invention firstly performs two-class defect detection on the collected images of the aircraft skin spray paint surface based on the surface smoothness, so as to realize the defect area in the detection of aircraft skin spray paint surface defects. It can accurately locate and accurately distinguish the defective parts and normal parts of the aircraft skin paint surface; then based on the simplified GoogLeNet convolutional neural network model, the aircraft skin paint surface defects are multi-classified and detected, and the classification and identification of different defect categories are realized. The framework can solve the defects of ordinary deep learning network intelligent detection or identification of known categories, has a strong tolerance for unknown defects, has greater flexibility and adaptability, and has high accuracy. efficiency, can effectively shorten the execution time of the algorithm, and has high accuracy; the method provided by the present invention can overcome the disadvantages of traditional manual visual inspection, and can reduce the waste of raw materials in the aircraft production process, improve the protection performance, prolong the service life of the aircraft skin, reduce the The later maintenance cost and guarantee of safety and reliability are of great significance, and it also fills the gap in the research on the detection of paint defects on the surface of large passenger aircraft. Compared with the existing technology, significant progress and unexpected results have been achieved.

Description of drawings

FIG. 1 is a diagram of a defect image detected by a binary classification method according to an embodiment of the present invention.

Detailed ways

The technical solution of the present invention will be further described in detail and completely below with reference to specific embodiments.

Example

A method for detecting and classifying surface defects of aircraft skin spray paint provided by the present invention comprises the following steps:

S1. Collect the image of the painted surface of the skin of the aircraft, for example, collect the image of the painted surface of the skin of the C919 type civil aircraft on the market;

S2. Perform two-class defect detection on the image collected in step S1 based on the surface smoothness, as follows:

First, preprocess the image collected in step S1 with Gaussian filtering, histogram equalization, Ostu adaptive threshold segmentation and erosion; then use Blob to perform block detection on the preprocessed image to find pixels with defects; then use the information The entropy function estimates the surface smoothness of the pixels with defects and the pixels of the whole image, and sets the surface smoothness threshold (the surface smoothness threshold represents the number of pixels with defects, when the number of defective pixels reaches this number, it is determined as this. The image is defective) to determine whether the image has defects, and the image is divided into defective images and non-defective images to realize the two-class defect detection of the image. The defect images detected by the two-class classification are shown in Figure 1, including common defects such as bubbles, particles, garbage, flow, and flanging. In this step, only the various images in Figure 1 are judged to be defective images. As for the specific defect types, refer to Subsequent multi-classification is confirmed.

One of the key and difficult points in the detection of surface defects of aircraft skin spray paint lies in the accurate positioning of the defect area. The present invention performs block detection on the preprocessed image through Blob, so that the defect area of the image can be accurately located.

The second key and difficulty in the detection of surface defects of aircraft skin paint is how to distinguish the defect parts of aircraft skin paint from normal parts. This application introduces an image surface smoothness evaluation method based on information entropy, in which information entropy can be used to measure The richness of image information, the information entropy based on the richness of the image information is used to evaluate the surface smoothness of the image, which provides a standard for measuring the surface defects of the aircraft skin paint, so that the defect parts of the aircraft skin paint can be accurately compared with the normal parts. Distinguishing: In this embodiment, the accuracy rate of the two-category test is 98.3%, the accuracy is high, and the result of the two-category test is accurate.

S3. Perform multi-classification detection on surface defects of aircraft skin spray paint based on the simplified GoogLeNet convolutional neural network model: the specific steps include the following:

S31. Data set production: use the collected images of the painted surface of the aircraft skin for the production of the data set, and divide the data set into a training sample data set and a test sample data set; during the production of the data set, the surface Professional painting workers distinguish the types of defects in the collected images. The types of defects include common defect types such as bubbles, particles, garbage, flow, and flanging, and then the divided images are divided into training sample data sets and test sample data. set;

S32. Build the GoogLeNet convolutional neural network model: The GoogLeNet convolutional neural network model includes five layers, and the five layers are the first layer, the second layer, the third layer, the fourth layer, and the fifth layer. Among them, the first layer layer and the second layer are ordinary convolutional layers, the third layer consists of Inception-A and Inception-B modules, and the fourth layer consists of Inception-A, Inception-B, Inception-C, Inception-D and Inception-E modules , the fifth layer consists of Inception-A and Inception-B modules;

S33. Build a simplified GoogLeNet convolutional neural network model: remove the Inception-B and Inception-C modules of the GoogLeNet convolutional neural network model in step S32, and build a simplified GoogLeNet convolutional neural network model;

Therefore, the simplified GoogLeNet convolutional neural network model constructed in this embodiment is: the first convolutional layer, the second convolutional layer, the third Inception-A module, the fourth Inception-A module, and the fourth Inception-D module, the fourth Inception-E module, the fifth Inception-A module; the operations of each convolutional layer and module are as follows:

The first convolution layer first uses 7*7 convolution kernels, the number of convolution kernels is 64, and then uses 3*3 kernels for maximum pooling, and the sliding step size is 2;

The second convolution layer first uses 3*3 convolution kernels, the number of convolution kernels is 192, and the sliding step size is 1, and then uses 3*3 kernels for maximum pooling, and the sliding step size is 2;

The third Inception-A module includes four branches. The four branches are: use a 1*1 convolution kernel, the number of convolution kernels is 64, and the sliding step size is 1; 2) First, use a 1*1 convolution Kernel, the number of convolution kernels is 96, and then a 3*3 convolution kernel is used, the number of convolution kernels is 128, and the sliding step size is 1; 3) First, a 1*1 convolution kernel is used, the convolution kernel The number is 16, and then 5*5 convolution kernels are used, the number of convolution kernels is 32, and the sliding step size is 1; 4) First, use 3*3 kernels for maximum pooling, and then use 1*1 volumes Product kernel, the number of convolution kernels is 32, and the sliding step size is 1; finally connect the output results of the four branches, and continue to the next step;

The fourth Inception-A module is the same as the third Inception-A module;

The fourth Inception-D module is the same as the third Inception-A module;

The fourth Inception-E module includes five branches. The five branches are as follows: 1) Use a 1*1 convolution kernel, the number of convolution kernels is 64, and the sliding step size is 1; 2) First, use a 1*1 convolution kernel Convolution kernel, the number of convolution kernels is 96, and then a 3*3 convolution kernel is used, the number of convolution kernels is 128, and the sliding step size is 1; 3) First, a 1*1 convolution kernel is used, the volume The number of accumulation kernels is 16, then 5*5 convolution kernels are used, the number of convolution kernels is 32, and the sliding step size is 1; 4) First use 3*3 kernels for maximum pooling, and then use 1*1 The number of convolution kernels is 32, and the sliding step size is 1; 5) First use 5*5 kernels for average pooling, the sliding step size is 3, and then use 1*1 convolution kernels, the volume The number of product kernels is 32, the sliding step size is 1, and then the fully connected layer is used twice, and then the Softmax activation function is used to assist the classification; finally, the output results of the first five branches are connected, and the next step is continued;

The fifth Inception-A module is the same as the third Inception-A module;

In each convolutional layer and module, the ReLU activation function is used after using the convolution kernel and pooling operation;

S34. Train the simplified GoogLeNet convolutional neural network model: input the images in the training sample data set into the simplified GoogLeNet convolutional neural network model for feature recognition to obtain the optimal simplified GoogLeNet convolutional neural network model;

S35. Test the simplified GoogLeNet convolutional neural network model: input the test sample data set into the trained simplified GoogLeNet convolutional neural network model to verify the accuracy;

S36. Multi-classification detection: output the defect images classified in step S2 into the simplified GoogLeNet convolutional neural network model after the test, and realize multi-classification detection of the surface defects of aircraft skin spray paint by simplifying the GoogLeNet convolutional neural network model. Through this step, it is possible to further determine the specific defect type of the second-classified defect image in step S2, for example, whether the defect image is a bubble defect or a particle, garbage, flow or flanging defect.

The third key and difficulty in the detection of surface defects of aircraft skin spray paint is the classification and identification of different defect categories. The present invention adopts the simplified GoogLeNet network based on Inception-V4 to realize the classification and identification of different defect categories, and the whole framework can solve the problem of ordinary deep learning network. Intelligent detection or identification of known types of defects, strong tolerance for unknown defects, greater flexibility and adaptability, high accuracy, improved computing efficiency while satisfying network learning performance, and effectively shortened algorithm execution time . In this embodiment, the multi-classification test accuracy rate is 99.7%, the accuracy rate is high, and the multi-classification result is accurate.

S4. Output multi-classification results to complete defect detection and classification on the painted surface of aircraft skin.

The method for detecting and classifying surface defects of aircraft skin spray paint provided by the present invention integrates the two-classification method based on traditional image defect detection and the multi-class defect classification framework based on the simplified GoogLeNet convolutional neural network model based on deep learning. The classification results are shown in Figure 1. As shown in the figure, the classification results include five defect types: bubbles, flanging, particles, flowing and garbage. The combination of two-classification and multi-classification effectively improves the computational efficiency and robustness of the algorithm, thereby effectively shortening the algorithm. The execution time improves the accuracy of aircraft skin surface defect detection, and can effectively meet the needs of intelligent detection and classification of aircraft skin paint surface defects. It is of great significance to prolong the service life, reduce maintenance costs in the later period and ensure safety and reliability.

Finally, it should be pointed out here that the above are only some preferred embodiments of the present invention, and should not be construed as limiting the protection scope of the present invention, and some non-essential improvements and adjustments made by those skilled in the art according to the above-mentioned content of the present invention All belong to the protection scope of the present invention.

Claims (1)

1. a method for realizing the detection and classification of aircraft skin spray paint surface defects, is characterized in that, comprises the steps: S1. Collect an image of the painted surface of the aircraft skin; S2. Perform two-class defect detection on the image collected in step S1 based on the surface smoothness, as follows: First, preprocess the image collected in step S1 with Gaussian filtering, histogram equalization, Ostu adaptive threshold segmentation and erosion; then use Blob to perform block detection on the preprocessed image to find pixels with defects; then use the information The entropy function estimates the surface smoothness of the pixels with defects and the pixels of the whole image, sets the surface smoothness threshold to determine whether the image has defects, divides the image into defective images and non-defective images, and realizes the two-class defect detection of the image. ; S3. Perform multi-classification detection on surface defects of aircraft skin spray paint based on the simplified GoogLeNet convolutional neural network model: the specific steps include the following: S31. Data set production: The collected images of the painted surface of the aircraft skin are used for the production of the data set, and the data set is divided into a training sample data set and a test sample data set; in the production process of the data set, the surface is sprayed first Professional workers distinguish the types of defects in the collected images, including bubbles, particles, garbage, flow, and flanging, and then divide the classified images into training sample data sets and test sample data sets respectively; S32. Build the GoogLeNet convolutional neural network model: The GoogLeNet convolutional neural network model includes five layers, and the five layers are the first layer, the second layer, the third layer, the fourth layer, and the fifth layer. Among them, the first layer layer and the second layer are convolutional layers, the third layer consists of Inception-A and Inception-B modules, and the fourth layer consists of Inception-A, Inception-B, Inception-C, Inception-D and Inception-E modules, The fifth layer consists of Inception-A and Inception-B modules; S33. Build a simplified GoogLeNet convolutional neural network model: remove the Inception-B and Inception-C modules of the GoogLeNet convolutional neural network model in step S32, and build a simplified GoogLeNet convolutional neural network model; The constructed simplified GoogLeNet convolutional neural network model is in order: the first convolutional layer, the second convolutional layer, the third Inception-A module, the fourth Inception-A module, the fourth Inception-D module, and the fourth Inception-E module module, the fifth Inception-A module; the operations of each convolutional layer and module are as follows: The first convolution layer first uses 7*7 convolution kernels, the number of convolution kernels is 64, and then uses 3*3 kernels for maximum pooling, and the sliding step size is 2; The second convolution layer first uses 3*3 convolution kernels, the number of convolution kernels is 192, and the sliding step size is 1, and then uses 3*3 kernels for maximum pooling, and the sliding step size is 2; The third Inception-A module includes four branches. The four branches are as follows: 1) Use a 1*1 convolution kernel, the number of convolution kernels is 64, and the sliding step size is 1; 2) First, use a 1*1 convolution kernel Convolution kernel, the number of convolution kernels is 96, and then a 3*3 convolution kernel is used, the number of convolution kernels is 128, and the sliding step size is 1; 3) First, a 1*1 convolution kernel is used, the volume The number of accumulation kernels is 16, then 5*5 convolution kernels are used, the number of convolution kernels is 32, and the sliding step size is 1; 4) First use 3*3 kernels for maximum pooling, and then use 1*1 The number of convolution kernels is 32, and the sliding step size is 1; finally connect the output results of the four branches, and continue to the next step; The fourth Inception-A module is the same as the third Inception-A module; The fourth Inception-D module is the same as the third Inception-A module; The fourth Inception-E module includes five branches. The five branches are as follows: 1) Use a 1*1 convolution kernel, the number of convolution kernels is 64, and the sliding step size is 1; 2) First, use a 1*1 convolution kernel Convolution kernel, the number of convolution kernels is 96, and then a 3*3 convolution kernel is used, the number of convolution kernels is 128, and the sliding step size is 1; 3) First, a 1*1 convolution kernel is used, the volume The number of accumulation kernels is 16, then 5*5 convolution kernels are used, the number of convolution kernels is 32, and the sliding step size is 1; 4) First use 3*3 kernels for maximum pooling, and then use 1*1 The number of convolution kernels is 32, and the sliding step size is 1; 5) First use 5*5 kernels for average pooling, the sliding step size is 3, and then use 1*1 convolution kernels, the volume The number of product kernels is 32, the sliding step size is 1, and then the fully connected layer is used twice, and then the Softmax activation function is used to assist the classification; finally, the output results of the first five branches are connected, and the next step is continued; The fifth Inception-A module is the same as the third Inception-A module; In each convolutional layer and module, the ReLU activation function is used after using the convolution kernel and pooling operation; S34, train the simplified GoogLeNet convolutional neural network model: input the images in the training sample data set into the simplified GoogLeNet convolutional neural network model for feature recognition, and obtain the simplified GoogLeNet convolutional neural network model; S35. Test the simplified GoogLeNet convolutional neural network model: input the test sample data set into the trained simplified GoogLeNet convolutional neural network model to verify the accuracy; S36. Multi-classification detection: input the defect image classified by the second classification in step S2 into the simplified GoogLeNet convolutional neural network model after the test, and realize multi-classification detection of surface defects of aircraft skin spray paint by simplifying the GoogLeNet convolutional neural network model; S4. Output multi-classification results to complete defect detection and classification on the painted surface of aircraft skin.

Images