CN114332047A

CN114332047A - Construction method and application of surface defect detection model

Info

Publication number: CN114332047A
Application number: CN202111666451.5A
Authority: CN
Inventors: 李斌; 李威风; 唐立新; 牛通之; 张泽丰; 邱园红
Original assignee: Huazhong University of Science and Technology
Current assignee: Huazhong University of Science and Technology
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2022-04-12

Abstract

The invention belongs to the field of image processing, and particularly relates to a construction method and application of a surface defect detection model, which comprises the following steps: the method comprises the steps that first m layers of coding blocks cascaded in a coding module extract features of a training image step by step to obtain shallow features; fusing and inputting all shallow layer features into a defect positioning module, obtaining position features by using bar pooling, adding the position features and the shallow layer features of the last level, performing feature extraction step by using the last n layers of coding blocks cascaded in a coding module to obtain deep layer features, fusing all the deep layer features, inputting into a defect shape detection module, obtaining shape features by using pyramid pooling and local bar pooling, and controlling a decoding module to perform up-sampling, fusion and dimension reduction on the shape features and part or all of the level features layer by layer to obtain a defect area; and updating the neural network parameters under the constraint of the defect position label, the shape label and the area label, and performing iterative training. The method not only focuses on the defect position, but also strengthens the defect shape adaptation, and realizes accurate segmentation of the defect.

Description

Construction method and application of surface defect detection model

Technical Field

The invention belongs to the field of image processing, and particularly relates to a construction method and application of a surface defect detection model.

Background

Regular or irregular processing textures are left on the surface of the workpiece due to different processing methods, so that the defect detection difficulty is increased, and meanwhile, the shape difference of defects on the surface of the workpiece is large due to the influence of various random factors in the processing process, so that the defect detection is greatly interfered. Therefore, how to accurately detect defect regions with different shapes on the surface of a workpiece with irregular texture becomes an urgent problem to be solved in an industrial production line.

In the defect detection method based on machine vision, the defect detection method can be divided into a traditional defect detection method and a defect detection method based on a neural network according to different feature extraction methods.

In the conventional defect detection method, different defect detection methods are designed according to different defect characteristics, such as a more classical threshold method and a method adopting coefficient reconstruction, and the method has poor adaptability, different defect detection models are required for different types of defects, and the detection method is required to be redesigned for defects with larger differences.

In the defect detection method based on the neural network, due to the enhancement of large-scale data acquirability and the appearance of high-performance computing hardware, the defect detection method is considered to be feasible by adopting the neural network, and typical methods comprise PGANet adopting single model detection or HDCB-Net adopting multi-model cascade.

The multi-model cascading method adopts a plurality of models to cascade and detect the defect area, is mainly used for detecting the local target area in the large image, is complex in deployment, and is accompanied by cutting operation in the detection process, so that information is easily lost, and the detection accuracy is reduced.

The single model detection method adopts a mode of collecting a plurality of submodels to enhance the feature extraction capability of the network so as to enhance the defect detection accuracy, and the method avoids the risk of potential information loss in the multi-model cascade method, but the existing method has no network structure for injecting the position and the shape of the defect.

Disclosure of Invention

Aiming at the defects and the improvement requirements of the prior art, the invention provides a construction method of a surface defect detection model and application thereof, aiming at realizing that the defect position is concerned and the defect shape adaptation is strengthened in the defect detection process, thereby realizing accurate segmentation of the defects.

To achieve the above object, according to an aspect of the present invention, there is provided a method for constructing a surface defect inspection model, including:

s1, obtaining a neural network framework of the surface defect detection model, which comprises: the device comprises an encoding module, a defect positioning module, a defect shape detection module and a decoding module;

s2, inputting the images in the training set into the coding module, and performing feature extraction step by step on the front m-layer coding blocks cascaded in the coding module to obtain shallow features of different levels;

s3, sampling all the shallow features to the same scale, connecting the shallow features into a whole, and inputting the whole into the defect positioning module, wherein the defect positioning module extracts the related information of the defect position in the features by using a strip pooling mode to obtain the position features representing the defect position;

s4, adding the position features and the shallow features of the last level, extracting features step by utilizing the last n layers of coding blocks cascaded in the coding module to obtain deep features of different levels, down-sampling all the deep features to the same scale, fusing and inputting the deep features to the defect shape detection module;

s5, controlling the defect shape detection module to extract relevant information of defect shapes in the characteristics by using simplified pyramid pooling and local strip pooling to obtain shape characteristics representing the defect shapes and outputting the shape characteristics to the decoding module;

s6, controlling the decoding module to perform up-sampling, fusion and dimensionality reduction on the received features and part or all of the hierarchical features obtained by the encoding module layer by layer to finally obtain a defect region;

s7, calculating a loss function to update neural network parameters based on the position feature and the defect position label, the shape feature and the defect shape label and the defect area label, and repeatedly executing S2 until a termination condition is reached to finish the training and construction of the surface defect detection model.

Further, the S2 further includes:

and controlling another coding block in the coding module to perform feature re-extraction on the first-level features obtained by coding, updating the first-level features and using the first-level features as shallow features.

Further, in S3, the defect locating module is specifically configured to:

fusing the received features with each layer of features by using one coding block, then performing dimensionality reduction and grouping on the fused features by using different convolution units, giving the grouping number according to the defect number in a single image, then performing pooling on the grouped features respectively by using row bar pooling and column bar pooling, and performing sampling fusion on the pooled features to obtain the features reflecting the defect positions.

Further, in S4, the position features are subjected to feature extraction step by using the last n-layer coding blocks cascaded in the coding module, so as to obtain deep features of different levels, specifically:

and adding the position characteristics and the shallow layer characteristics of the last level, and performing characteristic extraction step by using the last n layers of coding blocks cascaded in the coding module to obtain the deep layer characteristics of different levels.

Further, the defect shape detection module comprises a simplified pyramid pooling sub-module and a local strip pooling sub-module; in S5, the defect shape detection module is specifically configured to:

the simplified pyramid pooling sub-module adopts square pooling checks of different sizes to perform pooling operation on the feature V obtained in the S4, the pooled features are up-sampled to the size same as the feature V, then a first convolution unit is utilized to fuse the up-sampled pooled features and reduce the dimension to obtain a feature V1, the process is constrained by a defect shape label, and the used features during calculation constraint are as follows: reducing the dimension of the feature V1 by adopting a second convolution unit to obtain a one-dimensional feature V11;

the local strip-shaped pooling submodule respectively performs pooling operation on the feature V obtained by the S4 by adopting local row pooling kernels and local column pooling kernels with different shapes, upsamples two pooled features to the same size as the feature V, then adds the two upsampled pooled features and fuses the features by using a third convolution unit to obtain a feature V2, the process is constrained by a defect shape label, wherein the used features are as follows: dimensionality reduction is carried out on the feature V2 by adopting a fourth convolution unit to obtain a one-dimensional feature V21;

connecting the fusion feature obtained in the step S4, the feature V1 and the feature V2 into a whole, fusing by using a fifth convolution unit, and performing feature dimensionality reduction to obtain a feature V';

and outputting the feature V ' as a defect shape feature W to the decoding module, or adding the feature V11 and the feature V21, taking softmax, multiplying the result by the feature V ', connecting the multiplied result and the feature V ' into a whole, sending the whole into a sixth convolution unit for dimensionality reduction and fusion, obtaining the defect shape feature W, and outputting the defect shape feature W to the decoding module.

Further, the neural network framework further comprises a fine extraction module;

the shape feature obtained in S5 is first input to the fine extraction module; and the fine extraction module performs expansion convolution on the shape characteristics, connects the output result and the shape characteristics into a whole, performs dimensionality reduction fusion by using a seventh convolution unit to obtain characteristics Z, and inputs the characteristics Z into the decoding module.

Further, the loss function loss in S7 is:

loss＝λloss_loc+μ₁loss_shp+μ₂loss_shp+νloss_seg；

therein, loss_loc、loss_shpAnd loss_segRespectively representing position loss, shape loss and segmentation loss, lambda, mu₁、μ₂And ν represents the weight of the three losses, respectively.

Further, the weight mu corresponding to the two processes of the character tower pooling and the local strip pooling in the shape loss₁、μ₂May be set to be the same.

The invention also provides a surface defect detection method, which is used for detecting the surface defects by adopting the surface defect detection model constructed by the construction method of the surface defect detection model.

The present invention also provides a computer-readable storage medium comprising a stored computer program, wherein when the computer program is executed by a processor, the apparatus on which the storage medium is located is controlled to execute a method for constructing a surface defect detection model as described above and/or a method for detecting surface defects as described above.

Generally, by the above technical solution conceived by the present invention, the following beneficial effects can be obtained:

(1) the invention provides a convolution neural network model formed by combining a defect positioning module, a defect shape detection module and a fine extraction module, so that the defect position is concerned and the defect shape adaptation is strengthened in the defect detection process, and the requirement on accurate segmentation of the defect is met.

(2) According to the neural network established by the invention, as the defect positioning module positions the region where the defect is located, the interference of background noise in the image can be effectively avoided, and the combination of square pooling and local strip pooling effectively adapts to the detection of defects of different shapes, thereby accurately segmenting the defect region.

(3) The invention has the advantages of end-to-end detection and two-stage detection, and has the key technology that the position of the defect is determined by using strip pooling, and the shape of the defect is determined by using local strip pooling and simplified pyramid pooling.

Drawings

FIG. 1 is a schematic diagram of a neural network framework focusing on defect location and shape features according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a neural network focusing on defect location and shape features provided by an embodiment of the present invention;

FIG. 3 is a schematic diagram of a defect localization module according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of a defect localization module according to an embodiment of the present invention;

FIG. 5 is a partial schematic view of a defect shape module provided in accordance with an embodiment of the present invention; wherein, (a) is a simplified gold seed tower pooling module structure schematic diagram; (b) is a schematic structural diagram of a local strip-shaped pooling module;

FIG. 6 is a functional diagram of a defect shape detection module according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a defect shape detection module according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of a fine extraction module according to an embodiment of the present invention;

FIG. 9 illustrates a partial defect detection effect provided by an embodiment of the present invention; wherein, (a) is the original picture; (b) a DANet detection effect graph is obtained; (c) an SPNet detection effect graph is shown; (d) a U-Net detection effect graph is obtained; (e) a RefineNet detection effect graph is obtained; (f) is a detection effect graph of the method of the invention; (g) is a label graph;

fig. 10 is a diagram of the detection effect of each sub-module according to the embodiment of the present invention, where (a) is an original image, (b) is a shape label diagram, (c) is a position label diagram, (d) is a feature diagram after detection processing by the position detection module, (e) is a feature diagram after processing by the shape detection module, and (f) is a final detection result.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

Example one

A method for constructing a surface defect detection model, as shown in fig. 1, includes:

s2, inputting the images in the training set into a coding module, and carrying out feature extraction on the front m layers of coding blocks cascaded in the coding module step by step to obtain shallow features of different levels;

s3, sampling all shallow features to the same scale, connecting the shallow features into a whole, and inputting the whole into a defect positioning module, wherein the defect positioning module extracts relevant information of defect positions in the features by using a strip pooling mode to obtain position features representing the defect positions;

s4, adding the position characteristics and the shallow characteristics of the last level, extracting the characteristics step by utilizing the last n layers of coding blocks cascaded in the coding module to obtain the deep characteristics of different levels, down-sampling all the deep characteristics to the same scale, fusing and inputting the deep characteristics to the defect shape detection module;

s7, calculating a loss function to update the neural network parameters based on the position feature and the defect position label, the shape feature and the defect shape label and the defect area label, and repeatedly executing S2 until a termination condition is reached to complete the construction of the surface defect detection model.

The method mainly comprises two stages of encoding and decoding, wherein in the encoding stage, a defect positioning module is utilized for shallow features and defect position label constraint is combined, so that the shallow features of a model pay more attention to the positions of defects; for the high-level features in the encoding stage, combining the shape perception module with the constraint of the defect shape label to enable the model to acquire different receptive field features and focus on defect detail information in the high-level features; and meanwhile, the generalization of the enhanced model is further extracted from the features by utilizing the expansion convolution in the coding stage. In the decoding stage, the jump connection is utilized to fuse the output of each layer, and the final defect area is obtained by decoding.

It should be noted that the dashed box and the dashed arrow in fig. 1 are optional.

In one embodiment, the method comprises the following steps:

and S1, processing the acquired qualified industrial image and the acquired unqualified industrial image to generate an image data set, wherein the qualified industrial image is defined as an OK sample, and the unqualified industrial image is defined as an NG sample. And mixing the OK sample and the NG sample according to a certain proportion, wherein the value range of the proportion is that OK/NG is 2/3-4/5.

(S2) the mixed data is used as a data set for training and testing the network, wherein the proportion of the training set is 0.7-0.8, the proportion of the testing set is 0.2-0.3, the NG samples in the training set and the testing set are required to be marked with defect shape labels, and a minimum parallel axis circumscribed rectangle corresponding to a defect area is generated according to the marked defect shape labels to serve as a position label.

The design network structure of S2 comprises defect positioning module, defect shape detection module, fine extraction module, and coding and decoding structure.

And S3, inputting the training set images in the S1 into an encoder for encoding, obtaining encoding characteristics of different levels, and dividing the encoding characteristics into shallow-layer characteristics and high-layer characteristics.

And S4, inputting the shallow feature in the S3 into a defect positioning module to extract the position of the defect, and obtaining the feature reflecting the position of the defect.

S5, inputting the high-level features in S3 to a defect shape detection module to extract defect shape information and obtain corresponding output features.

And S6, inputting the features output in S5 into a fine extraction module, further obtaining more accurate feature information, and obtaining corresponding output features.

And S7, fusing and decoding the features output in S6 and the shallow feature and the high feature in S3 to obtain a final defect area.

S8 the above process is implemented in a continuous training process, which is constrained by a loss function, which includes position loss, shape loss, and segmentation loss. The three losses jointly form training losses, and the proportion of the three losses is adjusted according to different emphasis points.

With reference to fig. 2, in detail, a method for constructing a surface defect online detection model includes the following steps:

step 1, inputting images in a training set into a neural network for encoding to obtain features of different levels, wherein the features output by each encoding block (such as a VGG block) in FIG. 1 are all one-level features and are divided into shallow features and deep features.

And 2, acquiring the position information of the defect by using a defect positioning module. In order to better integrate different levels of features, another VGG block is required to be used for feature extraction again for the first level of features, shallow features with different scales are sampled to the same scale, and then the shallow features are fused and input to a defect positioning module to extract related information of defect positions in the features.

Wherein, for the defect location module implemented by using the strip pooling, a feasible implementation manner thereof is shown in fig. 3. In the figure, the features of the first three layers are subjected to dimension reduction and grouping by using an encoding block, for example, VGG is adopted to fuse the features of each layer, and different convolution units (for example, 1 multiplied by 1, the input and output sizes are the same) are used for performing dimension reduction and grouping, wherein the grouping number can be given according to the number of defects in a single image. And then, respectively pooling the characteristics by using row strip pooling and column strip pooling, and sampling and fusing the pooled characteristics to obtain the characteristics of the reaction defect positions. The process is constrained by a defect location penalty function.

And 3, adding the features processed in the step 2 with the third layer of features, continuously extracting high-layer features by using a coding block, down-sampling the high-layer features with different scales to the same scale, and fusing and inputting the high-layer features to a defect shape detection module for extracting related information of defect shapes in the features.

And 4, constructing the shape characteristics of the defect by adopting a simplified pyramid pooling submodule and a local strip pooling submodule on the characteristics input into the defect shape detection module in the step 3.

Specifically, for the feature V input into the pyramid pooling sub-module, pooling operation is performed on the feature by using square pooling kernels of different sizes, the pooled feature is up-sampled to the same size as the feature V, and then the up-sampled feature is fused and dimensionality-reduced by using a convolution unit to obtain the feature V1. The process is constrained by shape labels, where the features used in computing the constraint are: and reducing the dimension of the V1 by a convolution unit to obtain a one-dimensional characteristic V11.

Performing pooling operation on the features V input into the local strip-shaped pooling sub-module by adopting local row pooling kernels and local column pooling kernels with different shapes, upsampling the pooled features until the features V have the same size, adding the two upsampled pooling features, and fusing the features by using a convolution unit to obtain the features V2. The process is constrained by shape labels, where the features used in computing the constraint are: reducing the dimension of V2 by using a convolution unit to obtain a one-dimensional feature V₂₁。

And (3) fusing the features subjected to the high-order fusion in the step (3), the features V1 subjected to the simplified pyramid pooling and the features V2 subjected to the local strip pooling by using a certain convolution unit, and performing feature dimensionality reduction to obtain features V' (which can be directly output as the shape features W).

Preferably, the feature V11 and the feature V21 are added, softmax is taken, and then multiplied by the feature V ', and the multiplied result and the feature V' are sent to a certain convolution unit for dimensionality reduction and fusion to obtain the feature W.

And 5, sending the features W processed in the step 4 into a fine extraction module for further extraction, wherein the fine extraction module comprises two expansion convolutions of 3 multiplied by 3, and the expansion rates are respectively 2 and 4. And (4) carrying out dimension reduction fusion on the output results of the two expansion convolutions and the feature W processed in the step (4) by using a certain convolution unit again to obtain a feature Z.

And 6, performing up-sampling, fusion and dimensionality reduction on the feature Z processed in the step 5 and the optional layer feature layer by layer to finally obtain a defect region. The process is constrained by the defect label, and the optional layers are selected according to actual needs.

Further, the label constraint loss function of step 2, step 4, and step 6 is:

loss＝λloss_loc+μ₁loss_shp+μ₂loss_shp+νloss_seg

wherein loss_loc,loss_shpAnd loss_segRespectively substituted by position loss, shape loss and segmentation loss, lambda, mu₁,μ₂And ν represents the weight of the three losses. Weight μ in two sub-blocks in shape loss₁,μ₂May be set to be the same.

The present invention is described in further detail below with reference to examples.

The invention is described in detail by taking the detection of the cylindrical surface defect of the commutator as an example, but the application object of the invention is not limited to the detection, the commutator is an important part in the motor, periodic and aperiodic texture characteristics are left on the outer surface due to external circle turning and grinding in the production process, different defect characteristics are left on the surface defect characteristics due to the change of a processing cutter and a die, and different contrast characteristics are presented, and the invention can effectively detect the cylindrical surface defect in the commutator under the conditions.

Step 1, dividing the acquired commutator image into a training set and a test set, labeling labels, and drawing out a defect area, wherein the training set accounts for 80% of the total data volume, and the test set accounts for 20% of the total data volume. Inputting the images in the training set into a neural network for coding to obtain features of different levels, taking fig. 2 as an example, coding the images by using VGG blocks, wherein the features output by each VGG block in the graph are all level features, the total level is 5, and the number is represented by (i), (iv).

And 2, fusing the characteristics of the first three layers by using a defect positioning module to obtain the position information of the defect. In order to better integrate different levels of features, a VGG block is used for feature re-extraction of a first layer of features, the first three layers of features with different scales are sampled to the same scale, and then the features are connected into a whole and input to a defect positioning module to extract related information of defect positions in the features.

The defect localization module is implemented using strip pooling, as shown in FIG. 3. In fig. 3, the features of the first three layers are connected into a whole, one VGG encoding block is used for fusing the features of each layer, and then a plurality of 1 × 1 convolutions are used for reducing the dimension and grouping the features, wherein the grouping number can be given according to the number of defects in a single image. And then, respectively pooling the features by using row strip pooling and column strip pooling, sampling and fusing the pooled features to obtain the features of the reaction defect positions, and recording the features as Y. The process is constrained by a defect location penalty function.

It should be noted that the defect localization module is specifically configured to: the received features are subjected to row bar pooling and column bar pooling to extract features reflecting the positions of defects, and the specific principle is shown in fig. 4:

the gray area in (a) in fig. 4 is assumed to be a certain defective area. After row bar pooling and column bar pooling, two vectors are obtained, shown in fig. 4 (b), which are multiplied by a matrix to obtain the result, fig. 4 (c). Comparing (a) and (c) in fig. 4, the defect region shown by the gray area in (c) in fig. 4 is substantially the same as the minimum outer parallel axis rectangle of the original defect region in (a) in fig. 4 (shown by the rectangular box). This indicates that defect location features are obtained instead of defect shape features after being processed by the defect location module. This is more compact than conventional methods using FasterRCNN or YOLO.

And 3, adding the features processed in the step 2 with the third layer of features, continuously extracting the high-layer features by using a VGG block, and similarly connecting the high-layer features (the fourth and fifth layer of features) into a whole and inputting the integrated features into a defect shape detection module to extract the related information of the defect shape in the features.

And 4, constructing the shape features of the defects by adopting a simplified pyramid pooling submodule ((a) in fig. 5) and a partial strip pooling submodule ((b) in fig. 5) for the features input into the defect shape detection module in the step 3.

Specifically, pooling operation is performed on the features V input into the pyramid pooling submodule by pooling kernels with the sizes of 2 × 2 and 4 × 4 respectively, the pooled features are up-sampled to the same size as the input features V, and then dimension reduction and fusion are performed on the up-sampled features by using 1 × 1 convolution and a VGG block to obtain the features V1. The process is constrained by shape labels, where the features used in computing the constraint are: and reducing the dimension of the V1 by a convolution unit to obtain a one-dimensional characteristic V11.

For the features V input into the local strip-shaped pooling sub-module, pooling operation is carried out on the features by adopting local row pooling kernels (1 × 4) and local column pooling kernels (4 × 1) with different shapes, the pooled features are up-sampled to the same size of the input features V, and then the two up-sampled pooled features are added and fused by a VGG block to obtain the features V2. The process is constrained by shape labels, where the features used in computing the constraint are: reducing the dimension of V2 by adopting 1 x 1 convolution to obtain one-dimensional characteristic V₂₁。

And (4) fusing the high-order fused features in the step (3), fusing the simplified pyramid pooled features in the step (5) and the local bar pooled features in the step (6) by using a VGG block, and reducing the dimension of the features.

And (3) fusing the high-order fused features in the step (3), the features V1 subjected to the simplified pyramid pooling and the features V2 subjected to the local strip pooling by using VGG blocks, and performing feature dimensionality reduction to obtain features V'.

It should be noted that the present invention proposes to adopt pyramid pooling submodule, which simplifies the calculation, and compared with the original pyramid pooling model, the model combines the independent convolution after each pooling in the original pyramid pooling and the up-sampling into one body, and then the convolution is performed, modified into that each pooling is directly up-sampled, and then the pooling is combined into one body, and the convolution is unified. The change effectively reduces the calculation amount of the algorithm and simplifies the complexity of the model.

The local strip-shaped pooling provided by the invention has the following advantages: compared with strip pooling, local strip pooling not only retains the advantages of detecting strip defects, but also mitigates the effects of background noise when detecting defects. The first and second columns in fig. 6 show the differences between the three pooled nuclei (light gray areas are defect areas during pooling, dark gray areas are disturbed areas in the pooled nuclei). FIG. 6 shows the characteristics of strip pooling, partial strip pooling, and square pooling. The strip pooling has an advantage over the strip defect, the local strip pooling reduces the interference of small areas while maintaining the strip pooling advantage, and the square pooling has a better effect over dense shapes. And the detection of defects with different shapes can be adapted by combining the defects.

The defect detection shape of the invention combines the square pooling nucleus and the strip pooling nucleus, and gives full play to the respective advantages of the two pooling nuclei, and fig. 6 shows the characteristics, in particular: the strip-shaped pooling nucleus and the square pooling nucleus are combined, and the advantages of the two pooling nuclei are integrated, so that the network has stronger adaptability to complex shapes and defects. As shown in fig. 6, local stripe pooling can accommodate stripe defects, while square pooling has some advantages for complex defects. Thus, both pooling nuclei can be used simultaneously to accommodate defects of different shapes as much as possible.

Preferably, the feature V11 and the feature V21 are added, softmax is taken, and then multiplied by the feature V ', and the multiplied result and the feature V' are sent to 1 × 1 convolution for dimensionality reduction fusion to obtain the feature W, which is shown in fig. 7.

And 5, sending the features W processed in the step 4 into a fine extraction module (shown in figure 8) for further extraction, wherein the fine extraction module comprises two 3 x 3 expansion convolutions, and the expansion rates are 2 and 4 respectively. And (4) reusing the output results of the two expansion convolutions and the feature W processed in the step (4) to perform dimension reduction fusion on the features by using the 1 multiplied by 1 convolution to obtain the feature Z.

And 6, fusing the features processed in the step 5 and the front 4 layers of features in the step 1 layer by layer, and up-sampling to obtain a final defect area, wherein a part of defect detection effects are shown in a graph 9. This process is constrained by the defect label.

Further, the label constraint loss function of step 2, step 4, and step 6 is:

loss＝λloss_loc+μ₁loss_shp+μ₂loss_shp+νloss_seg

wherein loss_loc,loss_shpAnd loss_segRespectively substituted by position loss, shape loss and segmentation loss, lambda, mu₁,μ₂And ν represents the weight of the three losses. Weight μ in two sub-blocks in shape loss₁,μ₂Are typically set to be the same. Fig. 10 shows the effect diagram of each sub-module, and it can be seen from the diagram that the defect location module effectively extracts the position feature of the defect, and the defect shape detection module accurately detects the defect shape feature, which proves the effectiveness of the module provided by the present invention.

It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for constructing a surface defect detection model is characterized by comprising the following steps:

s7, calculating a loss function to update the neural network parameters based on the position feature and the defect position label, the shape feature and the defect shape label and the defect area label, and repeatedly executing S2 until a termination condition is reached to finish the construction of the surface defect detection model.

2. The method for constructing a surface defect inspection model according to claim 1, wherein the step S2 further comprises:

3. The method for constructing a surface defect detection model according to claim 1, wherein in S3, the defect localization module is specifically configured to:

fusing the received features with each layer of features by using one coding block, then performing dimension reduction and grouping on the fused features by using different convolution units, setting the grouping number according to the number of defects in a single image, then performing pooling on the grouped features respectively by using row bar pooling and column bar pooling, and performing sampling fusion on the pooled features to obtain the features reflecting the positions of the defects.

4. The method according to claim 1, wherein in S4, the position features are subjected to feature extraction step by using last n layers of coding blocks cascaded in the coding module to obtain deep features of different levels, specifically:

5. The method for constructing the surface defect detection model according to claim 1, wherein the defect shape detection module comprises a simplified pyramid pooling sub-module and a local strip pooling sub-module; in S5, the defect shape detection module is specifically configured to:

the local strip-shaped pooling submodule respectively performs pooling operation on the feature V obtained by the S4 by adopting local row pooling kernels and local column pooling kernels with different shapes, samples the pooled features up to the same size as the feature V, adds the two types of the pooled features after sampling, and fuses the features by using a third convolution unit to obtain a feature V2, wherein the process is constrained by a defect shape label, and the used features during calculation and constraint are as follows: dimensionality reduction is carried out on the feature V2 by adopting a fourth convolution unit to obtain a one-dimensional feature V21;

connecting the fusion feature V obtained in the step S4, the feature V1 and the feature V2 into a whole, fusing by using a fifth convolution unit, and performing feature dimensionality reduction to obtain a feature V';

6. The method for constructing the surface defect detection model according to claim 5, wherein the neural network framework further comprises a fine extraction module;

7. The method of claim 1, wherein the loss function loss in S7 is:

loss＝λloss_loc+μ₁loss_shp+μ₂loss_shp+νloss_seg；

8. The method according to claim 7, wherein the weight μ corresponding to the two processes of pyramid pooling and local bar pooling in the shape loss is the weight μ₁、μ₂May be set to be the same.

9. A surface defect inspection method, characterized in that the surface defect inspection model constructed by the method for constructing a surface defect inspection model according to any one of claims 1 to 8 is used for surface defect inspection.

10. A computer-readable storage medium, comprising a stored computer program, wherein when the computer program is executed by a processor, the computer program controls a device on which the storage medium is located to perform a method of constructing a surface defect detection model according to any one of claims 1 to 8 and/or a method of detecting surface defects according to claim 9.