CN116071338A - Method, device and equipment for detecting surface defects of steel plate based on YOLOX - Google Patents
Method, device and equipment for detecting surface defects of steel plate based on YOLOX Download PDFInfo
- Publication number
- CN116071338A CN116071338A CN202310133204.1A CN202310133204A CN116071338A CN 116071338 A CN116071338 A CN 116071338A CN 202310133204 A CN202310133204 A CN 202310133204A CN 116071338 A CN116071338 A CN 116071338A
- Authority
- CN
- China
- Prior art keywords
- module
- defect detection
- steel plate
- yolox
- outputting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 121
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 79
- 239000010959 steel Substances 0.000 title claims abstract description 79
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 87
- 238000012549 training Methods 0.000 claims abstract description 38
- 101100295091 Arabidopsis thaliana NUDT14 gene Proteins 0.000 claims abstract description 14
- 238000007781 pre-processing Methods 0.000 claims abstract description 10
- 238000012937 correction Methods 0.000 claims abstract description 7
- 238000012360 testing method Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 8
- 238000005520 cutting process Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000002787 reinforcement Effects 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 239000011800 void material Substances 0.000 claims description 2
- 230000004927 fusion Effects 0.000 description 9
- 238000004519 manufacturing process Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000000605 extraction Methods 0.000 description 6
- BATBOVZTQBLKIL-UHFFFAOYSA-N [1-(5,8-dihydroxy-1,4-dioxonaphthalen-2-yl)-4-methylpent-3-enyl] 3-methylbut-2-enoate Chemical compound C1=CC(O)=C2C(=O)C(C(OC(=O)C=C(C)C)CC=C(C)C)=CC(=O)C2=C1O BATBOVZTQBLKIL-UHFFFAOYSA-N 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention relates to a method for detecting surface defects of a steel plate based on YOLOX, which comprises the steps of constructing a defect detection model based on a YOLOX network, and initializing epoch, learning rate and model weight; inputting the training set into a defect detection model, preprocessing the training set, inputting the training set into a backbone network, and outputting three effective feature maps with different sizes through a Focus module and four dark net modules which are sequentially connected in series; respectively inputting the three effective feature graphs into three CBAM modules with corresponding sizes, carrying out self-adaptive feature correction, and then respectively inputting three branches of a bottleneck network, wherein each branch comprises an RFB module and an ASPP module which are connected in series, outputting three enhanced feature graphs, and respectively inputting the three enhanced feature graphs into corresponding decoupling heads to output a prediction result; decoding the prediction result, dynamically matching positive and negative samples by using the SimOTA, calculating a total loss value of the positive and negative samples, and updating model weights based on the total loss value; and repeatedly inputting the sample data into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model.
Description
Technical Field
The invention relates to the technical field of steel plate defect detection, in particular to a method, a device and equipment for detecting steel plate surface defects based on YOLOX.
Background
The development of deep learning greatly promotes the development of defect detection, the steel plate defect detection method based on deep learning abandons the part of the manual design feature extraction algorithm in traditional machine learning, adopts a network self-learning feature mode, trains in interaction with sample data, realizes automatic feature extraction by updating network architecture parameters through counter propagation, omits the step of the manual design feature extraction algorithm, and further improves the precision and efficiency, and is widely applied to production detection in the industrial field at present. The single-stage detection network taking the YOLO series as the main part takes an important role in the target detection field, the picture is directly input into the model, the neural network can acquire a corresponding result only by learning the picture once, the speed is ensured, the accuracy is also high, and the performance and the speed are greatly optimized again after a plurality of versions of iterations, so that the model becomes a main stream model in the target detection field based on deep learning.
The model in the prior art is YOLOX, adopts a backbone network and a bottleneck network which are the same as YOLOv3, uses mosaic enhancement and hybrid enhancement in the aspect of data preprocessing, adopts a CSP structure still in the backbone network, mainly utilizes a residual structure to extract features, fuses and splices the extracted feature information with the original input, and outputs the generated 3 features with different dimensions to the bottleneck network part. The section then performs 3-dimensional feature fusion on the input feature map and outputs it to the decoupling head. Three branches are formed through the decoupling head, the three types of output are stacked respectively for the category, the foreground and the background of the prediction frame and the coordinate information of the prediction frame, the output is decoded, a positive sample is obtained through the SimOTA, and the prediction frame is optimized through calculating a loss value. The activation function used in the whole model network is a combination of linear rectification ReLU and weighted linear rectification Si LU, and the latter is most common, and the whole is an end-to-end YOLO model.
At present, the steel plate can generate different types of defects such as cracks, scratches, spots, pits and the like in the production process, so that the appearance is influenced, the strength is reduced, and the rust is easy to generate. The existing detection network is insufficient in detection precision and accuracy of small targets, characteristics of the small targets are not fully considered, and as the number of network layers is deepened, the characteristics of the small targets are continuously reduced until the characteristics disappear, so that various small target defects can be detected by greatly detecting errors, detection is very unreliable, production efficiency and cost are greatly affected, and the detection network cannot be applied to the industry.
Disclosure of Invention
Therefore, the invention aims to solve the technical problem of insufficient detection precision and accuracy of small targets in the prior art.
In order to solve the technical problems, the invention provides a method for detecting surface defects of a steel plate based on YOLOX, which comprises the following steps:
inputting sample data to be detected into a backbone network of a defect detection model constructed based on a YOLOX network, and outputting three effective feature maps with different sizes through a second dark module, a third dark module and a fourth dark module through a Focus module and four dark modules which are sequentially connected in series;
inputting three effective feature graphs with different sizes into three CBAM modules with corresponding sizes respectively, multiplying the channel attention feature graphs and the space attention feature graphs generated by the CBAM modules with the input effective feature graphs, and outputting after self-adaptive feature correction;
respectively inputting the output of the three CBAM modules into three branches of a bottleneck network, wherein each branch comprises an RFB module and an ASPP module which are connected in series, and outputting three enhancement feature graphs;
inputting the three enhancement feature maps to corresponding decoupling heads respectively; the decoupling head convolves the input enhancement feature map, decomposes the enhancement feature map into a regression branch, a confidence coefficient branch and a classification branch, and outputs three prediction targets, wherein the prediction targets comprise coordinate information of a prediction frame, a foreground, a background and an object type; and stacking the three prediction targets and outputting a prediction result.
In one embodiment of the present invention, before the inputting the sample data to be detected into the backbone network of the defect detection model constructed based on the YOLOX network, the method further includes:
obtaining the picture data of the surface defects of the steel plate, and dividing the picture data into a training set and a testing set;
initializing epoch, learning rate and model weight of a defect detection model constructed based on the YOLOX network; inputting the training set into a defect detection model, and preprocessing to obtain a sample training set;
the preprocessing comprises the steps of rotating, cutting, scaling, translating and Gri dMask enhancing the picture data of the surface defects of the steel plate.
In one embodiment of the invention, the dark modules each comprise a BaseConv module and a csclayer module connected in series along the forward direction of propagation;
the BaseConv module comprises a convolution unit, a batch regularization unit and an activation function unit which are sequentially connected in series along the positive propagation direction;
the fourth dark net module further comprises an SPP module, and the SPP module is arranged behind the CSPLlayer module;
the SPP module convolves the input characteristic images, inputs the convolved characteristic images into a plurality of pooling units with different pooling core sizes for pooling, inputs the pooled characteristic images and the convolved characteristic images into a SELayer layer for processing, and outputs an effective characteristic image through the BaseConv module.
In one embodiment of the present invention, the RFB module decomposes the input reinforcement feature map into three branches with different expansion rates, and each branch is processed by convolution kernels with different sizes, and then performs splicing output on the three branches through 3×3 hole convolution.
In one embodiment of the present invention, the ASPP module convolves the output of the RFB module with a plurality of parallel hole convolution layers having different sampling rates, and then merges the output enhancement feature map.
In one embodiment of the present invention, after outputting the prediction result, the method further includes:
decoding the prediction result, dynamically matching positive and negative samples by using the SimOTA, calculating a total loss value of the positive and negative samples, and optimizing model weight based on the total loss value;
repeatedly inputting sample data to be detected in the sample training set into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model;
and inputting sample data to be detected in the test set into a final defect detection model to obtain a steel plate surface defect detection result.
In one embodiment of the present invention, the S imOTA dynamically matches positive and negative samples comprising:
screening a positive sample candidate region as a real frame;
calculating the coincidence degree I OU of the real frame and each prediction frame;
summing the first 10 predicted frames with the highest overlap ratio with the I OU of the real frame to obtain the k value of the real frame, rounding k downwards, taking the first k predicted frames as positive samples, and taking the rest as negative samples;
calculating a cost matrix by using the positive samples and the negative samples, and dynamically matching the positive samples and the negative samples;
the cost matrix is expressed as:
wherein ,
and />
Classification loss and positioning loss, respectively.
In one embodiment of the present invention, the calculating the positive and negative sample total loss value includes:
training confidence branches and classification branches by using BCEWithLog itsLoss function to obtain foreground and background loss L obj And classification loss L cls ；
Training regression branches by using the sI OU function to obtain positioning loss L reg ；
Total loss value:
wherein lambda is the balance coefficient of the positioning loss, defaults to 5.0, N pos Anchor points for being divided into positive samples.
The embodiment of the invention also provides a device for detecting the surface defects of the steel plate based on the YOLOX, which comprises the following steps:
the data acquisition module is used for acquiring the picture data of the surface defects of the steel plate;
the preprocessing module is used for carrying out rotation, cutting, scaling, translation and Gri dMask enhancement on the multiple steel plate surface defect pictures to obtain sample pictures;
the backbone network module is used for processing the input sample pictures through the Focus module and the four dark net modules and outputting three effective feature pictures with different sizes;
the bottleneck network module is used for processing the effective feature map through the CBAM module, the RFB module and the ASPP module with corresponding sizes and outputting an enhanced feature map;
the decoupling module is used for decomposing the enhanced feature map and outputting three prediction targets including coordinate information, foreground, background and object types; stacking the three prediction targets and outputting a prediction result;
the model optimization module is used for decoding the prediction result, dynamically matching positive and negative samples by using the S imaOTA, calculating the total loss value of the positive and negative samples, and optimizing model weight based on the total loss value;
model acquisition module: repeating the steps, inputting the sample picture into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model;
and the test set prediction module inputs the test set into the final defect detection model to obtain a steel plate surface defect detection result.
The embodiment of the invention also provides a device for detecting the surface defects of the steel plate based on the YOLOX, which comprises the following components:
the acquisition device is used for acquiring a picture of the surface defect of the steel plate;
the device based on the method for detecting the surface defects of the steel plate according to the YOLOX is in communication connection with the acquisition device and is used for realizing the steps of the method for detecting the surface defects of the steel plate according to the YOLOX when executing the computer program and outputting the detection result of the surface defects of the steel plate;
and the display device is in communication connection with the device of the YOLOX-based steel plate surface defect detection method and is used for acquiring and displaying the steel plate surface defect detection result.
Compared with the prior art, the technical scheme of the invention has the following advantages:
according to the method for detecting the surface defects of the steel plate based on the YOLOX, the characteristic of the small target is subjected to enhanced extraction and characteristic fusion based on a YOLOX network; the CBAM module is used for respectively carrying out convolution on a space and a channel, generating attention characteristic images in two dimensions of the channel and the space in a serialization manner, multiplying the attention characteristic images with the input characteristic images, carrying out self-adaptive characteristic correction, and improving detection precision; decomposing an input reinforcement feature map into three branches with different expansion rates by utilizing an RFB module, and superposing the three branches with different receptive fields to enlarge the receptive fields of a defect detection model network; constructing hollow convolution kernels of different receptive fields by using different hollowness through an ASPP module to obtain multi-scale object information; compared with the original YOLOX network, the detection precision of small target defects such as cracks, scratches, inclusions, pitting corrosion, oxide scale and the like on the surface of the steel plate is greatly improved, MAP values are improved to more than three points, the small target defect detection effect is remarkable, the production cost of the steel plate is reduced, and the product quality and the production efficiency are improved.
Drawings
In order that the invention may be more readily understood, a more particular description of the invention will be rendered by reference to specific embodiments thereof that are illustrated in the appended drawings, in which
FIG. 1 is a flow chart of steps of a method for detecting defects on a surface of a YOLOX-based steel plate according to the present invention;
FIG. 2 is a schematic diagram of a defect detection model according to the present invention;
fig. 3 is a schematic diagram of the composition structure of the BaseConv module provided by the present invention;
FIG. 4 is a schematic diagram of the composition structure of the SPP module according to the present invention;
FIG. 5 is a schematic diagram of the structure of a decoupling head according to the present invention;
FIG. 6 is a schematic diagram of test results on a training set of a prior art YOLOX-based defect detection technique;
FIG. 7 is a schematic diagram of the test results of the method for detecting surface defects of a steel plate based on Yolox provided by the invention on a training set.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and specific examples, which are not intended to be limiting, so that those skilled in the art will better understand the invention and practice it.
Referring to fig. 1, a flowchart of steps of a YOLOX-based steel plate surface defect detection method according to the present invention includes:
s1: inputting sample data to be detected into a backbone network of a defect detection model constructed based on YOLOX, and outputting three effective feature maps with different sizes through a second dark module, a third dark module and a fourth dark module through a Focus module and four dark modules which are sequentially connected in series;
referring to fig. 2, a schematic structural diagram of a defect detection model provided by the present invention is shown; the defect detection model comprises a backbone network, a bottleneck network and a decoupling head; the backbone network comprises a Focus module and four dark net modules; the bottleneck network comprises three branches, and each branch comprises a CBAM module, an RFB module and an ASPP module which are sequentially connected in series;
the Focus module is used for obtaining a value at every other pixel in an input sample picture, obtaining four independent feature layers, stacking the four feature layers, concentrating the wide-high information into channel information, expanding the input channel by four times, changing the spliced feature layers into twelve channels relative to the original three channels, and enhancing the feature extraction capability; then, the data are convolved, standardized in batches and activated by the BaseConv module and then output to the first dark net module.
Referring to fig. 3, the BaseConv module includes a convolution unit, a batch regularization unit and an activation function unit sequentially connected in series along a positive propagation direction, where the activation function adopts an S i LU function.
The first dark net module, the second dark net module, the third dark net module and the fourth dark net module all comprise a BaseConv module and a CSPLlayer module which are connected in series along the forward transmission direction;
the CSPLlayer module is used for carrying out residual calculation after carrying out 1×1 convolution on the input characteristic image; and stacking the output after residual calculation with the output subjected to the 1×1 convolution only, and obtaining the output after stacking through the 1×1 convolution.
An SPP module is added in a CSPLlayer module of the fourth dark net module; the SPP module is an SPPBott eNeck module, and referring to FIG. 4, the SPP module performs pooling through pooling check feature images with different sizes, so that the receptive field of a network can be remarkably improved; and the SELAyer is added in the SPP module, the multi-scale space information is extracted, and the sensitivity to channel characteristics is improved while the fusion of more space characteristics is realized.
The CSPLayer modules of the second dark net module, the third dark net module and the fourth dark net module output an effective feature map which is a first feature map, a second feature map and a third feature map respectively, and the effective feature maps are sent to a bottleneck network; the shape of the first feature map, the second feature map and the third feature map is f1= (80,80,256), f2= (40,40,512), f3= (20,20,1024);
s2: inputting three effective feature graphs with different sizes into three CBAM modules with corresponding sizes respectively, multiplying the channel attention feature graphs and the space attention feature graphs generated by the CBAM modules with the input effective feature graphs, and outputting after self-adaptive feature correction;
and the CBAM module is used for respectively carrying out convolution processing on the space and the channel to extract the characteristics, generating attention characteristic map information in the two dimensions of the channel and the space in a serialization way, and then multiplying the two kinds of characteristic map information with the original input characteristic map to carry out self-adaptive characteristic correction.
S3: respectively inputting the output of the three CBAM modules into three branches of a bottleneck network, wherein each branch comprises an RFB module and an ASPP module which are connected in series, and outputting three enhancement feature graphs;
the RFB module is used for deforming each input characteristic image into 3 branches with different expansion rates, the bottom layer of each branch is processed through convolution kernels with the sizes of 1x1,3x3 and 5x5 respectively, then the branches are subjected to cavity convolution with the size of 3x3, so that different branches have different receptive fields, and finally the branches with different receptive fields are overlapped, so that the receptive fields of a network are greatly improved;
the ASPP module uses multiple parallel hole convolution layers with different sampling rates to construct hole convolution kernels of different receptive fields through different hole rates, and features extracted by each sampling rate are further processed in separate branches and fused to generate a final result. Constructing convolution kernels of different receptive fields through different void ratios to obtain multi-scale object information;
s4: inputting the three enhancement feature maps to corresponding decoupling heads respectively; each decoupling head outputs three prediction targets, wherein the prediction targets comprise coordinate information of a prediction frame, a foreground, a background and an object type; stacking the three prediction targets and outputting a prediction result;
referring to fig. 5, a decoupling head Yo l head convolutionally decomposes an input enhancement feature map into a regression branch, a confidence branch and a classification branch, respectively acquires coordinate information, foreground and background of a prediction frame and object category, stacks the obtained prediction frame and outputs a prediction result;
specifically, three prediction targets, reg (h, w, 4), can be obtained for each enhancement feature map, and the three prediction targets are respectively used for judging regression parameters, namely coordinate information, of each feature point, and a prediction frame can be obtained after the regression parameters are adjusted; obj (h, w, 1) is used to determine whether each feature point contains an object; cl s (h, w, num_cl assembly) is used to determine the kind of object contained in each feature point. Three prediction targets are stacked to obtain a prediction result of each feature layer as (h, w,4+1+num_c l asses), wherein 4 represents a regression parameter of each feature point, 1 represents whether an object is contained, num_cl errors represent the type of the object contained in each feature point, and (h, w) represents the length and width of a prediction frame.
Based on the above embodiment, in this embodiment, before inputting the sample data to be detected into the backbone network of the defect detection model constructed based on the YOLOX network, the method further includes:
obtaining the picture data of the surface defects of the steel plate, and dividing the picture data into a training set and a testing set;
constructing a defect detection model based on a YOLOX network, and initializing epoch, learning rate and model weight; inputting the training set into a defect detection model, and preprocessing to obtain a sample training set;
the preprocessing comprises the steps of rotating, cutting, scaling, translating and Gri dMask enhancing the picture data of the surface defects of the steel plate.
Based on the foregoing embodiment, in this embodiment, after outputting the prediction result, the method further includes:
decoding the prediction result, dynamically matching positive and negative samples by using the S imoa, calculating a total loss value of the positive and negative samples, and optimizing model weight based on the total loss value; repeatedly inputting sample data to be detected in the sample training set into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model; and inputting sample data to be detected in the test set into a final defect detection model to obtain a steel plate surface defect detection result.
The dynamically matching positive and negative samples by using the S imaOTA comprises the following steps: screening a positive sample candidate region as a real frame gt; calculating the coincidence degree IOU of the real frame gt and each prediction frame anchor; summing the first 10 prediction frames anchor with the highest overlap ratio with the IOU of the real frame to obtain the k value of the real frame, rounding down k, taking the first k anchors as positive samples, and taking the rest as negative samples; calculating a cost matrix by using the positive samples and the negative samples, and dynamically matching the positive samples and the negative samples;
the cost matrix is expressed as:
wherein ,
and />
Classification loss and positioning loss, respectively.
Training confidence branches and classification branches by using BCEWithLog itsLoss function to obtain foreground and background loss L obj And classification loss L cls The method comprises the steps of carrying out a first treatment on the surface of the Training regression branches by using the sI OU function to obtain positioning loss L reg ；
Total loss value:
wherein lambda is the balance coefficient of the positioning loss, defaults to 5.0, N pos Anchor points for being divided into positive samples.
Specifically, the size of the surface defect picture input by the sample training set is set to 640×640×3, and the size of the image is changed to 320×320×64 through the Focus structure and the BaseConv operation; then sequentially passing through the first dark module, the second dark module, the third dark module and the fourth dark module to change the image size into 160×160×128, 80×80×256, 40×40×512 and 20×20×1024; adding an SPPBott l eNeck module into a fourth dark module to complete the construction of a backbone network; 3 effective feature graphs with different scales are respectively led out of the second dark net module, the third dark net module and the fourth dark net module, a bottleneck network is input for feature fusion, and 3 enhancement feature graphs after feature fusion are input into 3 YOLO Head for classification prediction; the specific operation of feature fusion of the bottleneck network is as follows: the method comprises the steps of outputting 3 effective feature graphs with different dimensions into a coat 1, a coat 2 and a coat 3 after passing through CBAM modules with corresponding dimensions respectively, and obtaining an enhanced feature graph P1 after Conv2D of the coat 3 led out from a fourth dark net module; p1 is subjected to up-sampling and Concat connection with the fet 2 led out from the third dark net module, and an enhanced feature layer P2 is obtained through a CSPLlayer and Concat structure after connection; similarly, performing Concat connection on the P2 up-sampling and the fet 1 led out from the second dark net module, and obtaining an enhanced feature layer P3 through a CSPLlayer structure after connection; in addition, the reinforced feature layer is subjected to top-to-bottom rebinning, which comprises the following specific operations: direct downsampling of P3 is connected with P2 to obtain an enhanced feature layer P4 after passing through a CSPLlayer structure, downsampling of P4 is continuously connected with P1 to obtain an enhanced feature layer P5 after passing through a CSPLlayer structure; finally, after feature fusion, 5 enhancement feature layers P1 to P5 are obtained; the enhancement characteristic layers P3, P4 and P5 are correspondingly input into RFB modules and ASPP modules with the sizes of 80 multiplied by 256, 40 multiplied by 512 and 20 multiplied by 1024, and then classified prediction is carried out in a YOLO Head network with the sizes of three corresponding to realize the detection of the surface defects of the steel plate.
According to the method for detecting the surface defects of the steel plate based on the YOLOX, the characteristics of the surface defects of the steel plate are fully considered, and the characteristics of small targets are subjected to enhanced extraction and characteristic fusion; the Gri dMask enhancement method is added during pretreatment, so that reasonable balance can be achieved between deleting and reserving region information on the image, and the image context information finally input into the model network is not lost and the robustness of the network is high; the additional SELayer in the SPP module is used for extracting multi-scale space information, so that feature fusion in more spaces is realized and the sensitivity to channel features is improved; a CBAM module (a channel and a spatial attention module) is added in a bottleneck network, convolution processing is carried out on the space and the channel respectively to extract characteristics, attention characteristic map information is generated in the two dimensions of the channel and the space in a serialization mode, and then the two characteristic map information is multiplied with the original input characteristic map to carry out self-adaptive characteristic correction; the RFB module is added in the bottleneck network, each input enters the RFB module to form 3 branches with different expansion rates, and then the branches with different receptive fields are overlapped to enlarge the receptive fields of the network; an ASPP module is added, and cavity convolution kernels of different receptive fields are constructed through different cavity rates so as to obtain multi-scale object information; the decoupling head adopts an S IOU function for iterative optimization of the coordinate information of the prediction frame, the vector angle between the required regression is fully considered, the punishment index is redefined, and the accuracy of the coordinate information is obviously improved. Compared with the existing model, the MAP value of the defect detection model provided by the invention is improved by more than 3 percent, the detection accuracy of small target defects such as cracks, scratches, inclusions, pitting corrosion, oxide scale and the like on the surface of the steel plate is greatly improved, the detection effect is obvious, the production cost of the steel plate is reduced, and the product quality and the production efficiency are improved.
Referring to FIG. 6, the results of the prior art YOLOX model on the test set are shown; referring to FIG. 7, the test results of the method for detecting defects on the surface of a steel plate based on YOLOX provided by the invention are shown on a test set; according to fig. 6 and 7, compared with the existing detection method, the detection method for the surface defects of the steel plate based on YOLOX provided by the invention has the advantages that the detection accuracy is improved by more than 3 points, the detection effect on small target defects is remarkable, the production cost of the steel plate is reduced, and the product quality and the production efficiency are improved.
Based on the above embodiment, the present invention also provides a YOLOX-based steel plate surface defect detection device, including:
the data acquisition module 100 is used for acquiring the picture data of the surface defects of the steel plate;
the preprocessing module 200 is used for carrying out rotation, cutting, scaling, translation and Gri dMask enhancement on the multiple steel plate surface defect pictures to obtain sample pictures;
the backbone network module 300 is configured to process an input sample picture through the Focus module and the four dark net modules, and output three effective feature maps with different sizes;
the bottleneck network module 400 is configured to process the effective feature map through a CBAM module, an RFB module, and an ASPP module with corresponding sizes, and output an enhanced feature map;
the decoupling module 500 is configured to decompose the enhanced feature map, and output three prediction targets, including coordinate information, foreground, background, and object types; stacking the three prediction targets and outputting a prediction result;
the model optimization module 600 is configured to decode the prediction result, dynamically match positive and negative samples using S imOTA, calculate a total loss value of the positive and negative samples, and optimize the model weight based on the total loss value;
model acquisition module 700: repeating the steps, inputting the sample picture into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model;
the test set prediction module 800 inputs the test set into the final defect detection model to obtain the steel plate surface defect detection result.
The YOLOX-based steel plate surface defect detecting device of the present embodiment is used to implement the foregoing YOLOX-based steel plate surface defect detecting method, and thus the foregoing description of the embodiment of the YOLOX-based steel plate surface defect detecting device may be found in the foregoing example portions of the YOLOX-based steel plate surface defect detecting method, for example, the backbone network module 300 and the decoupling module 500 are respectively used to implement steps S1 and S4 in the foregoing YOLOX-based steel plate surface defect detecting method, and the bottleneck network module 400 is used to implement steps S2 and S3 in the foregoing YOLOX-based steel plate surface defect detecting method, so that the detailed description thereof may be referred to corresponding examples of each portion and will not be repeated herein.
Based on the above embodiments, the present invention also provides a YOLOX-based steel plate surface defect detecting apparatus, including:
the acquisition device is used for acquiring a picture of the surface defect of the steel plate;
the device for detecting the surface defects of the steel plate based on the YOLOX is in communication connection with the acquisition device and is used for realizing the steps of the method for detecting the surface defects of the steel plate based on the YOLOX when executing the computer program and outputting the detection result of the surface defects of the steel plate;
and the display device is in communication connection with the YOLOX-based steel plate surface defect detection device and is used for acquiring and displaying the steel plate surface defect detection result.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The method, the device and the equipment for detecting the surface defects of the steel plate based on the YOLOX provided by the invention are described in detail. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the method of the present invention and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
Claims (10)
1. A YOLOX-based steel sheet surface defect detection method, characterized by comprising:
inputting sample data to be detected into a backbone network of a defect detection model constructed based on a YOLOX network, and outputting three effective feature maps with different sizes through a second dark module, a third dark module and a fourth dark module through a Focus module and four dark modules which are sequentially connected in series;
inputting three effective feature graphs with different sizes into three CBAM modules with corresponding sizes respectively, multiplying the channel attention feature graphs and the space attention feature graphs generated by the CBAM modules with the input effective feature graphs, and outputting after self-adaptive feature correction;
respectively inputting the output of the three CBAM modules into three branches of a bottleneck network, wherein each branch comprises an RFB module and an ASPP module which are connected in series, and outputting three enhancement feature graphs;
inputting the three enhancement feature maps to corresponding decoupling heads respectively; the decoupling head convolves the input enhancement feature map, decomposes the enhancement feature map into a regression branch, a confidence coefficient branch and a classification branch, and outputs three prediction targets, wherein the prediction targets comprise coordinate information of a prediction frame, a foreground, a background and an object type; and stacking the three prediction targets and outputting a prediction result.
2. The YOLOX-based steel sheet surface defect detection method according to claim 1, wherein before inputting the sample data to be detected into a backbone network of a defect detection model constructed based on a YOLOX network, further comprising:
obtaining the picture data of the surface defects of the steel plate, and dividing the picture data into a training set and a testing set;
initializing epoch, learning rate and model weight of a defect detection model constructed based on the YOLOX network; inputting the training set into a defect detection model, and preprocessing to obtain a sample training set;
the pretreatment comprises the steps of rotating, cutting, scaling, translating and GridMask enhancing of the steel plate surface defect picture data.
3. The YOLOX-based steel plate surface defect detection method according to claim 1, wherein the dark modules each comprise a BaseConv module and a CSPLayer module connected in series in a forward propagation direction;
the BaseConv module comprises a convolution unit, a batch regularization unit and an activation function unit which are sequentially connected in series along the positive propagation direction;
the fourth dark net module further comprises an SPP module, and the SPP module is arranged behind the CSPLlayer module;
the SPP module convolves the input characteristic images, inputs the convolved characteristic images into a plurality of pooling units with different pooling core sizes for pooling, inputs the pooled characteristic images and the convolved characteristic images into a SELayer layer for processing, and outputs an effective characteristic image through the BaseConv module.
4. The YOLOX-based steel plate surface defect detection method of claim 1, wherein the RFB module decomposes the input reinforcement feature map into three branches with different expansion rates, each branch is subjected to convolution kernel processing with different sizes, and then three branches are spliced and output through 3x3 cavity convolution.
5. The YOLOX-based steel plate surface defect detection method of claim 1, wherein the ASPP module convolves the RFB module output with a plurality of parallel void convolution layers having different sampling rates, and then fuses the output enhancement feature map.
6. The YOLOX-based steel plate surface defect detection method according to claim 1, wherein after outputting the prediction result, further comprising:
decoding the prediction result, dynamically matching positive and negative samples by using the SimOTA, calculating a total loss value of the positive and negative samples, and optimizing model weight based on the total loss value;
repeatedly inputting sample data to be detected in the sample training set into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model;
and inputting sample data to be detected in the test set into a final defect detection model to obtain a steel plate surface defect detection result.
7. The YOLOX-based steel plate surface defect detection method of claim 6, wherein the SimOTA dynamically matching positive and negative samples comprises:
screening a positive sample candidate region as a real frame;
calculating the coincidence degree IOU of the real frame and each prediction frame;
summing the first 10 predicted frames with the highest overlap ratio with the IOU of the real frame to obtain the k value of the real frame, rounding down k, taking the first k predicted frames as positive samples and taking the rest as negative samples;
calculating a cost matrix by using the positive samples and the negative samples, and dynamically matching the positive samples and the negative samples;
the cost matrix is expressed as:
wherein ,
and />
Classification loss and positioning loss, respectively.
8. The YOLOX-based steel plate surface defect detection method of claim 7, wherein the calculating positive and negative sample total loss values comprises:
training confidence branches and classification branches by using BCEWITHLogitsLoss function to obtain foreground and background loss L obj And classification loss L cls ；
Training regression branches by using SIOU function to obtain positioning loss L reg ；
Total loss value:
wherein lambda is the balance coefficient of the positioning loss, defaults to 5.0, N pos Anchor points for being divided into positive samples.
9. A device for YOLOX-based steel sheet surface defect detection method according to any one of claims 1 to 8, comprising:
the data acquisition module is used for acquiring the picture data of the surface defects of the steel plate;
the preprocessing module is used for carrying out rotation, cutting, scaling, translation and GridMask enhancement on the multiple steel plate surface defect pictures to obtain sample pictures;
the backbone network module is used for processing the input sample pictures through the Focus module and the four dark net modules and outputting three effective feature pictures with different sizes;
the bottleneck network module is used for processing the effective feature map through the CBAM module, the RFB module and the ASPP module with corresponding sizes and outputting an enhanced feature map;
the decoupling module is used for decomposing the enhanced feature map and outputting three prediction targets including coordinate information, foreground, background and object types; stacking the three prediction targets and outputting a prediction result;
the model optimization module is used for decoding the prediction result, dynamically matching positive and negative samples by using the SimOTA, calculating the total loss value of the positive and negative samples, and optimizing model weight based on the total loss value;
model acquisition module: repeating the steps, inputting the sample picture into the defect detection model for training until the training times reach epoch, and outputting a final defect detection model;
and the test set prediction module inputs the test set into the final defect detection model to obtain a steel plate surface defect detection result.
10. A YOLOX-based steel plate surface defect detecting apparatus, characterized by comprising:
the acquisition device is used for acquiring a picture of the surface defect of the steel plate;
the YOLOX-based steel plate surface defect detecting method according to claim 9, wherein the device is communicatively connected to the collecting device, and is used for implementing the above-mentioned YOLOX-based steel plate surface defect detecting method when executing the computer program, and outputting a steel plate surface defect detecting result;
and the display device is in communication connection with the device of the YOLOX-based steel plate surface defect detection method and is used for acquiring and displaying the steel plate surface defect detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310133204.1A CN116071338A (en) | 2023-02-20 | 2023-02-20 | Method, device and equipment for detecting surface defects of steel plate based on YOLOX |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310133204.1A CN116071338A (en) | 2023-02-20 | 2023-02-20 | Method, device and equipment for detecting surface defects of steel plate based on YOLOX |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116071338A true CN116071338A (en) | 2023-05-05 |
Family
ID=86181799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310133204.1A Pending CN116071338A (en) | 2023-02-20 | 2023-02-20 | Method, device and equipment for detecting surface defects of steel plate based on YOLOX |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116071338A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117079195A (en) * | 2023-10-13 | 2023-11-17 | 中国科学技术大学 | Wild animal identification method and system based on image video |
| CN117853783A (en) * | 2023-12-12 | 2024-04-09 | 济南大学 | Single board defect identification method and system based on deep learning |
| CN118032820A (en) * | 2024-04-02 | 2024-05-14 | 深圳市艾兰特科技有限公司 | SMT full-automatic online X-RAY nondestructive testing system |
| CN118296468A (en) * | 2024-05-30 | 2024-07-05 | 厦门锋元机器人有限公司 | Aluminum welding defect detection method and system based on artificial intelligence |
-
2023
- 2023-02-20 CN CN202310133204.1A patent/CN116071338A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117079195A (en) * | 2023-10-13 | 2023-11-17 | 中国科学技术大学 | Wild animal identification method and system based on image video |
| CN117079195B (en) * | 2023-10-13 | 2024-02-23 | 中国科学技术大学 | Wild animal identification method and system based on image video |
| CN117853783A (en) * | 2023-12-12 | 2024-04-09 | 济南大学 | Single board defect identification method and system based on deep learning |
| CN118032820A (en) * | 2024-04-02 | 2024-05-14 | 深圳市艾兰特科技有限公司 | SMT full-automatic online X-RAY nondestructive testing system |
| CN118296468A (en) * | 2024-05-30 | 2024-07-05 | 厦门锋元机器人有限公司 | Aluminum welding defect detection method and system based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116071338A (en) | Method, device and equipment for detecting surface defects of steel plate based on YOLOX | |
| CN109190752B (en) | Image semantic segmentation method based on global features and local features of deep learning | |
| CN112465748B (en) | Crack identification method, device, equipment and storage medium based on neural network | |
| CN110378222B (en) | Method and device for detecting vibration damper target and identifying defect of power transmission line | |
| CN108664981B (en) | Salient image extraction method and device | |
| CN114627383B (en) | Small sample defect detection method based on metric learning | |
| CN110111334B (en) | Crack segmentation method and device, electronic equipment and storage medium | |
| CN110570396A (en) | industrial product defect detection method based on deep learning | |
| CN113628178B (en) | Steel product surface defect detection method with balanced speed and precision | |
| CN112819748B (en) | Training method and device for strip steel surface defect recognition model | |
| CN115908772A (en) | Target detection method and system based on Transformer and fusion attention mechanism | |
| CN115035334B (en) | Multi-classification change detection method and system for multi-scale fusion double-time-phase remote sensing image | |
| CN117036243A (en) | Method, device, equipment and storage medium for detecting surface defects of shaving board | |
| CN115661097A (en) | Object surface defect detection method and system | |
| CN116071315A (en) | Product visual defect detection method and system based on machine vision | |
| CN115147418A (en) | Compression training method and device for defect detection model | |
| CN117541587B (en) | Solar panel defect detection method, system, electronic equipment and storage medium | |
| CN113516640A (en) | CT image fine crack segmentation device and method based on classification branches | |
| CN117670860A (en) | Photovoltaic glass defect detection method and device | |
| CN110033443B (en) | Display panel defect detection method | |
| CN111882545A (en) | Fabric defect detection method based on bidirectional information transmission and feature fusion | |
| CN116596866A (en) | Defect detection method based on high-resolution image and storage medium | |
| CN116503885A (en) | Table identification method and device, electronic equipment and storage medium | |
| CN115100402A (en) | Small sample segmentation model, small target detection method, system, medium, and apparatus | |
| CN115330705A (en) | Skin paint surface defect detection method based on adaptive weighting template NCC |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |