CN114092467A

CN114092467A - Scratch detection method and system based on lightweight convolutional neural network

Info

Publication number: CN114092467A
Application number: CN202111453822.1A
Authority: CN
Inventors: 何静媛; 李宇昂; 周吉祥
Original assignee: Chongqing Shouyu Technology Co ltd; Chongqing University
Current assignee: Chongqing Shouyu Technology Co ltd; Chongqing University
Priority date: 2021-12-01
Filing date: 2021-12-01
Publication date: 2022-02-25

Abstract

The invention provides a scratch detection method and system based on a lightweight convolutional neural network. The method comprises the following steps: collecting a sample, and preprocessing the sample; building a lightweight network model, wherein the lightweight network comprises a backbone network consisting of 2 to 4 stages for scratch feature identification, the output of each stage is respectively input into a prediction module corresponding to each stage one by one for carrying out confidence degree preliminary prediction, and the output of each prediction module is connected with a fusion layer for fusing the confidence degrees of the preliminary predictions; and carrying out scratch detection on the preprocessed sample in the lightweight network model. The method removes the last stage of the VGG network, reduces the parameter quantity of the network and improves the detection speed.

Description

Scratch detection method and system based on lightweight convolutional neural network

Technical Field

The invention relates to the technical field of edge detection, in particular to a scratch detection method and system based on a lightweight convolutional neural network.

Background

The scratch defect detection task of the workpiece is based on an image edge detection technology, and the image edge detection is one of the most key technical problems in the fields of image processing and computer vision, and has high practical value. Edge detection methods can be broadly classified into two categories: the traditional method and the method based on deep learning are poor in effect and robustness due to the fact that the traditional method adopts manual feature setting to extract edges. Convolutional Neural Networks (CNNs) are favored in various fields by virtue of their powerful feature extraction capabilities, and the rapid update of CNNs technology further attracts more and more researchers to be added. Although the CNNs-based method has made great progress in the task of edge detection, there are still problems in the field of scratch defect detection focusing on practicability and real-time performance, for example, most edge detection methods based on CNNs use a common natural image data set for training, the data volume is sufficient, while scratch detection may have the problems of limited data acquisition and insufficient sample diversity, and a more serious problem is that related research of researchers in the field focuses more on accuracy, and tends to spend more parameters and reasoning time for obtaining a small amount of accuracy improvement, so that the network is more and more complex, the parameter volume is also more and more large, and the reasoning speed is slow, which makes it difficult to deploy into actual scratch detection applications.

Disclosure of Invention

In order to overcome the defects in the prior art, the invention aims to provide a scratch detection method and system based on a lightweight convolutional neural network.

In order to achieve the above object, the present invention provides a scratch detection method based on a lightweight convolutional neural network, including the steps of:

collecting a sample, and preprocessing the sample;

building a lightweight network model, wherein the lightweight network comprises a backbone network consisting of 2-4 stages for scratch feature identification, the output of each stage is respectively input into a prediction module corresponding to each stage one by one for carrying out confidence degree preliminary prediction, and the output of each prediction module is connected with a fusion layer for fusing the confidence degrees of the preliminary predictions;

and carrying out scratch detection on the preprocessed sample in the lightweight network model.

Most of the current edge detection networks use a VGG network as a backbone network. The VGG network consists of 5 stages, and the method reduces the number of network layers and speeds up the speed.

The preferable scheme of the scratch detection method is as follows: 4 stages are provided, wherein the first two stages respectively consist of a cavity convolution layer and a ShuffleNet Bottleneeck module layer; the last two stages respectively consist of a cavity convolution layer and two ShuffleNet Bottleneeck module layers; every two stages are connected through a downsampling operation, and the feature graph output by each stage is added through a convolution compression channel and then output.

The method removes the last stage of the VGG network, reduces the parameter number of the network, improves the detection speed, and increases the size of the receptive field reduced by reducing the number of network layers by using the cavity convolution instead of the normal convolution, thereby improving the effect; and a ShuffleNet Bottleneeck module is adopted to replace a part of convolution modules, so that the detection speed is improved.

The preferable scheme of the scratch detection method is as follows: after the characteristics of the stage are output, the characteristics output by the high-level stage are amplified by using an up-sampling operation to obtain a resolution ratio, then the resolution ratio is combined with the characteristics output by the low-level stage according to channels, and the combined characteristics are used for primarily predicting confidence. This can supply information to the underlying network, making the detection result more accurate.

The preferable scheme of the scratch detection method is as follows: when scratch detection is carried out, in a training stage, fusion layer output is used as a prediction result, and in a testing stage, a prediction module corresponding to a backbone network and a highest stage is taken for prediction. The fusion layer can improve the accuracy to a certain extent, but also increases the calculated amount and the reasoning time. Therefore, the scratch detection method is used in a training stage and a testing stage, namely, the scratch detection method is not used in the actual detection process, and the fusion layer is abandoned, so that a large amount of repeated operation is avoided, the network parameters are reduced, and the network detection speed is accelerated.

The preferable scheme of the scratch detection method is as follows: and when the outputs of all the prediction modules are fused in the training stage, adding the output of the high-level prediction module after the detach and the output of the low-level prediction module. The method is equivalent to using different supervision information for different stages, so that the method can better guide the learning of the stages of different levels of the network, and the different stages of the network can automatically learn the characteristics of corresponding scales.

The preferable scheme of the scratch detection method is as follows: and after the scratch is detected, processing the connected domains of the detection result, calculating the minimum external rectangle of each connected domain, and determining whether the connected domain is reserved or not according to the length and width thresholds of the set rectangle. The processing can effectively filter error results generated by network sensitivity.

The invention also provides a scratch detection system, which comprises a processor and a memory, wherein the processor is in communication connection with the memory, and the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the corresponding operation of the scratch detection method.

Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic view of a scratch detection process;

FIG. 2 is a schematic diagram of the network architecture of the present invention;

FIG. 3 is a schematic illustration of a scratch;

FIG. 4 is a schematic illustration of a scratch marking;

FIG. 5 is a diagram illustrating the results of network training;

FIG. 6 is a graph showing the test results.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.

As shown in fig. 1, the present invention provides an embodiment of a scratch detection method based on a lightweight convolutional neural network, which specifically includes the following steps:

firstly, collecting and preprocessing a sample:

a total of 10 images of different workpieces at different exposures were taken for training and 5 images were taken for testing using a mapping apparatus, and figure 3 shows a map of the workpiece with scratches. As shown in fig. 4, the marking tool is used to mark the scratch in the image. The original image size is 5472x3648, since the image is too large, the inference speed is greatly reduced, and therefore all images are scaled to 2736x1824 size with no loss of accuracy guaranteed. The scaled image is still too large for graphics card training, so the training image and the annotations are both cropped from the scaled image to 456x456 size and the pixel values of the image are normalized to [0,1 ]. The training data after cutting was 240 pieces in total, and the test data was 120 pieces. In order to enhance the network generalization, tools are used for making pseudo data, and an imgauge library is used for data expansion, and operations such as rotation, translation and distortion are carried out on the image. The final data used for training were 1020 and 120 pieces of test data.

Second, the construction of the network

In this embodiment, a pytorch deep learning library is used to construct a CNN network, and a network structure is shown in fig. 2, where dil _ block represents a hole convolution module, b _ block represents a shefflenet Bottleneck module, and 1x1_ conv represents a convolution with a convolution kernel size of one by one, which is used to compress a channel, fusion information and up represent an upsampling module, which is used to improve the resolution of a feature map, and p _ block represents a prediction module, which is composed of a 1x1_ conv and an upsampling module, and is used to predict and output a confidence map of a specific scale. The main network consists of 4 stages, the output channel of each stage is 64,128,256 and 512, the first two stages consist of two module layers: a hole convolution with an expansion rate of 2 and a shuffle lattice module grouped as 2; the last two stages consist of 3 modules: a hole convolution with an expansion rate of 2 is followed by two shuffle lattice blocks grouped by 2. Every two stages are connected through a downsampling operation, the downsampling operation is maxpool, and the convolution layers in the stages are connected through a batch normal layer and a relu activation layer. All feature maps output by each stage respectively pass through a convolution compression channel of 1x1, and then are added to obtain multi-scale information reinforced edge information, and the added features are used as the output of each stage. And respectively sending the characteristic output of each stage to a prediction module which is in one-to-one correspondence with each stage, and outputting confidence coefficient of the scratch by the prediction module. In order to supplement information to the underlying network, the features output by the high-level stage are amplified by twice the resolution by using an upsampling operation, then the features are combined with the features of the low-level stage according to channels, and the combined features are used for predicting the initial prediction confidence coefficient in a training stage or the prediction in a testing stage. In addition, in order to better guide the learning of stages of different levels of the network, a cascade structure training network is used when the output of each prediction module is fused in the training stage, the structure adds the output of the prediction module of the high level and the output of the prediction module of the low level after the prediction module of the high level outputs the detail, which is equivalent to that different stages use different supervision information, so that the different stages of the network can automatically learn the characteristics of corresponding scales. The backbone network has 4 stages, so 4 confidence maps are generated, and finally the results are fused by a convolution layer of 1x 1. The model will therefore yield 5 results, with the final fusion layer omitted from the network architecture diagram. The term detach as used herein means: the detach operation of the Pythrch stops the tentor variable from computing the gradient, i.e., the variable can no longer be propagated backwards and is not used to update the network parameters.

Training of model

Training by using processed data, optimizing the model by adopting an SGD optimizer, setting the batch size to be 4, and setting the initial learning rate to be 2 multiplied by 10^-4Loss using cross entropy loss, training iterates 500 epochs total, with the training results shown in fig. 5. In the training process of the model, each output confidence map of the model corresponds to one loss, and the loss is 5 in total, weighted sum is carried out on the loss, and back propagation is carried out on the loss to update the network parameters. The training and testing of the invention are based on a Linux operating system and use 1080 video cards.

Testing of model

FIG. 1 is a flow chart of scratch defect detection, different from the conventional method, in the testing stage, the invention removes a plurality of prediction modules of stages, and only takes the prediction module of the backbone network and the highest stage to perform reasoning test to accelerate reasoning speed. The test uses the manufactured 120 test data sets, after the scratch is detected, the detection result is post-processed, namely connected domain processing, the minimum circumscribed rectangle of the size of each connected domain is calculated, whether the connected domain is reserved or not is determined according to the set length and width thresholds of the rectangles, and the result is shown in fig. 6. Compared with RCF and BDCN methods, the number of the reasoning frames of the invention reaches 64 frames per second, which is respectively improved by 32 frames per second and 44 per second compared with the two methods.

Fifth, evaluation of the model

As shown in table 1, the proposed lightweight scratch detection method achieves better inference speed and accuracy than RCF, losing some accuracy compared to BDCN method, but the inference speed exceeds 3 times it. The test result proves the effectiveness of the method and meets the requirement of a scratch detection task.

TABLE 1 test results

Method	DSC(％)	Speed (frame/second)
			RCF	33.7	32
BDCN	39.5	20
			Method for producing a composite material	37.8	64

The invention also provides an embodiment of a scratch detection system, which comprises a processor and a memory, wherein the processor is in communication connection with the memory, and the memory is used for storing at least one executable instruction, and the executable instruction enables the processor to execute the corresponding operation of the scratch detection method.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A scratch detection method based on a lightweight convolutional neural network is characterized by comprising the following steps:

collecting a sample, and preprocessing the sample;

building a lightweight network model, wherein the lightweight network comprises a backbone network consisting of 2 to 4 stages for scratch feature identification, the output of each stage is respectively input into a prediction module corresponding to each stage one by one for carrying out confidence degree preliminary prediction, and the output of each prediction module is connected with a fusion layer for fusing the confidence degrees of the preliminary predictions;

2. The scratch detection method based on the lightweight convolutional neural network as claimed in claim 1, wherein there are 4 stages, and the first two stages are respectively composed of a hole convolutional layer and a shuffle bottleeck module layer; the last two stages respectively consist of a cavity convolution layer and two ShuffleNet Bottleneeck module layers; every two stages are connected through a downsampling operation, and feature graphs output by each module layer of each stage are added through convolution compression channels and then output.

3. The scratch detection method based on the lightweight convolutional neural network of claim 2, wherein the expansion rate of the hole convolutional layer is 2, and the groups of the shefflenet bottleeck modules are 2; the convolution kernel size of the convolution compressed channel is 1x 1.

4. The scratch detection method based on the lightweight convolutional neural network as claimed in claim 1, wherein after the features of the stage are output, the features output by the high-level stage are amplified by using an up-sampling operation to obtain a resolution, and then are combined with the features output by the low-level stage according to channels, and the combined features are used for the initial prediction confidence.

5. The scratch detection method based on the lightweight convolutional neural network according to claim 1, wherein in the scratch detection, the fused layer output is used as a prediction result in a training stage, and in a testing stage, a prediction module corresponding to the backbone network and the highest stage is used for prediction.

6. The scratch detection method based on the lightweight convolutional neural network according to claim 1 or 5, wherein when the outputs of the prediction modules are fused in the training stage, the output of the higher-level prediction module is added to the output of the lower-level prediction module after the socket.

7. The scratch detection method based on the lightweight convolutional neural network of claim 1, wherein the convolutional layers in the stage are connected through a batch normal layer and a relu activation layer.

8. The scratch detection method based on the lightweight convolutional neural network of claim 1, wherein after the scratch is detected, connected domain processing is performed on the detection result, the minimum bounding rectangle of the size of each connected domain is calculated, and whether the connected domain is reserved or not is determined according to the length and width thresholds of the set rectangles.

9. The scratch detection method based on the lightweight convolutional neural network as claimed in claim 1, wherein after the sample is collected, the scratch in the sample is marked, the sample is clipped, and the pixel value of the sample is normalized to [0,1 ].

10. A scratch detection system comprising a processor and a memory, said processor and memory being communicatively coupled, said memory storing at least one executable instruction that causes said processor to perform operations corresponding to the scratch detection method of any of claims 1-9.