CN115294089A - Steel surface defect detection method based on improved YOLOv5 - Google Patents
Steel surface defect detection method based on improved YOLOv5 Download PDFInfo
- Publication number
- CN115294089A CN115294089A CN202210990916.0A CN202210990916A CN115294089A CN 115294089 A CN115294089 A CN 115294089A CN 202210990916 A CN202210990916 A CN 202210990916A CN 115294089 A CN115294089 A CN 115294089A
- Authority
- CN
- China
- Prior art keywords
- image
- module
- surface defect
- steel
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 99
- 238000001514 detection method Methods 0.000 title claims abstract description 89
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 76
- 239000010959 steel Substances 0.000 title claims abstract description 76
- 238000002372 labelling Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 9
- 239000000463 material Substances 0.000 claims abstract description 7
- 238000000034 method Methods 0.000 claims description 18
- 238000009826 distribution Methods 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 7
- 238000005520 cutting process Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 230000007306 turnover Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 4
- 238000007670 refining Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 6
- 230000006870 function Effects 0.000 description 11
- 230000006872 improvement Effects 0.000 description 8
- 238000011160 research Methods 0.000 description 7
- 238000005096 rolling process Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- XEEYBQQBJWHFJM-UHFFFAOYSA-N Iron Chemical compound [Fe] XEEYBQQBJWHFJM-UHFFFAOYSA-N 0.000 description 2
- 238000002679 ablation Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 206010039509 Scab Diseases 0.000 description 1
- 238000005299 abrasion Methods 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 238000005242 forging Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 229910052742 iron Inorganic materials 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002923 metal particle Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 238000013138 pruning Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 239000009728 shiwei Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/0008—Industrial image inspection checking presence/absence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/225—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on a marking or identifier characterising the area
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a steel surface defect detection method based on improved YOLOv5, which comprises the following steps: 1. acquiring an image of the surface of a steel material to be detected; 2. carrying out defect labeling on the image by using the image, and forming a steel surface defect data set required by a model by using the obtained labeling file and the original image; 3. performing data enhancement on the data set; 4. constructing a steel surface defect detection model: adding a small target detection layer for identifying image characteristics in a small range on the basis of three detection layers of an original YOLOv5 network, introducing a Trans module into the tail end and the neck part of a backbone of the YOLOv5 network, replacing a part of a CONV module and a CSP2_1 module, and introducing a CBAM module into the neck part of the YOLOv5 network so as to be matched with the Trans module and the Conv module to enhance the performance of the network; 5. training a steel surface defect detection model; 6. and predicting a steel surface defect detection model. The invention enhances the detection effect on the defects of the small targets and effectively focuses on task target detection.
Description
Technical Field
The invention relates to defect detection, in particular to a steel surface defect detection method based on improved YOLOv 5.
Background
At present, in the aspect of steel surface defect detection operation, the following detection methods are commonly used: 1. manual detection method; 2. eddy current testing; 3. infrared detection method; 4. and (4) magnetic flux leakage detection method. The latter three of them cannot be widely used due to the reasons of technology or cost, etc., so that the most manual detection methods are adopted at present. However, this detection method is labor-consuming and inefficient. In recent years, with the rapid development of technologies such as machine vision and deep learning, the related applications thereof are becoming wider and wider, and many researches on defect detection of machine vision have been carried out. The detection method based on machine vision is low in cost and high in efficiency, and can obtain detection precision close to manual work through gradual research and improvement.
In order to solve the problem of detecting surface defects of steel plates, hot rolled steel or cold rolled strip steel, a plurality of researches using a traditional image detection or deep learning method are carried out at present. The director provides an improved two-dimensional maximum class variance threshold segmentation algorithm in a thesis, greatly improves the efficiency of defect image segmentation, designs a classifier based on a support vector machine, realizes classification of defect features and obtains a better classification result. And decomposing the steel plate image into sub-bands through Tetrolet transformation, extracting matrix characteristics of the sub-bands, and finally classifying the characteristic vectors subjected to dimension reduction through a support vector machine. The Huchao method is characterized in that 43-dimensional features including morphology, gray scale and texture are extracted from the surface defect features of the steel plate, feature selection is carried out by combining a Fisher criterion and a PCA principal component analysis method, and finally classification is carried out by using a binary tree support vector machine (BT-SVM). The method comprises the steps that the image of the steel plate surface defect is preprocessed by the Hu-Ji through a filtering algorithm, histogram equalization and a Sobel edge detection operator, then the characteristic dimension with high contribution degree is extracted through a Fisher criterion, and finally the defect is identified through an AdaBoost.BK algorithm, so that the highest classification precision reaches 85.89%. Although the method has a certain effect on detecting the surface defects of the secondary steel, the method adopts a traditional algorithm in the aspects of image feature extraction and feature identification, so that the detection accuracy of the model is low and the improvement is difficult.
In recent years, the emergence of technologies such as artificial neural networks, deep learning, and the like has achieved a lot of remarkable achievements in the field of image processing. Therefore, many researchers have combined deep learning techniques with defect detection work to achieve many better results. Yangxi and the like provide a strip steel surface defect detection algorithm based on CNN, the judgment of whether the strip steel surface has defects is realized, the accuracy rate reaches more than 99.99%, lijinu improves a U-Net semantic segmentation network in research, a residual error network structure based on grouping convolution and uniform feature recombination is provided to realize multi-scale feature extraction, a defect fine segmentation network Nabla-Net is obtained through transition feature and dense connection improvement, and an attention mechanism is introduced to enable a space to be subjected to self-adaptive adjustment and a deep supervision mechanism to realize network pruning. Comparison of U-Net and Res-UNet in steel image defect detection work is performed by Shiwei Geniste et al, and the results show that Res-UNet is more accurate in recognition of defect positions and defect types of steel images. Some of the above methods focus on the classification of target defects, and some focus on the segmentation of images, but no one combines the two methods to make them effectively locate while classifying defects, which will greatly limit their application in industrial production.
The Wang Li proposes that the surface defects of the strip steel are detected by taking Faster R-CNN as a framework and ResNet-101 as an image feature extraction network, and the network is trained by using a transfer learning method, so that an optimization model ensures the stationarity and convergence of network training. Since the method uses a two-stage detection method, the detection speed needs to be improved. The improved YOLOv3 algorithm is used in the defect research of the metal surface, the main improvement comprises the steps of preprocessing an image by using histogram equalization, simulating a real working environment by using a data enhancement method and optimizing a loss function to improve the accuracy of classification of the surface defect by using a model. In order to improve the steel flaw detection speed, the Liuyang provides an R-Tiny-Yolov3 algorithm based on Tiny-Yolov3, a residual error network and a Spatial Pyramid Pooling (SPP) module are added into the algorithm, and CIOU is selected as a loss function, the steel flaw detection precision of the algorithm reaches 71.5%, the detection speed reaches 39.8 frames per second, and the real-time detection requirement of embedded equipment can be met. The above researches respectively improve the performance of each aspect of defect detection in different ways, but the researches are insufficient on various unique defects aiming at the steel surface, which results in the incompleteness of defect detection types or the low detection precision of partial defect types, thereby resulting in the non-ideal overall performance.
In the process of steel production and processing, various defects can be left on the surface of the steel due to insufficient process or operation errors, and the defects on the surface of the steel are seven defects in general: 1. the crack defects are mostly caused by the bubbles, internal cracks or surface impurities contained under the skin of the produced ingot which are broken or extended during the rolling process. 2. The scratch is mostly caused by the accumulation of iron scale or other metal particles and foreign matters in the guide and guard device. And the surface of the steel is scratched due to improper installation of the guide plate. 3. The folding defects are mainly formed by the fact that burrs, creases, sharp corners and the like in the steel forging and rolling process are rolled into the steel. 4. The lug defect characteristic refers to a bulge formed along the rolling direction caused by improper adjustment of a rolling mill, deviation, looseness or overlarge size of a guide plate, roll movement and the like. 5. The scab (heavy skin) defect is a thin and flat layered structure formed by the sharp edges and corners existing in the original steel ingot, residues on the surface of a steel billet, pits, heavy skin or cleaned pits after rolling. 6. The crater defects are mainly distributed at the end parts of the rolled steel, and are mainly caused by the craters left after flame cutting or water welding. 7. The end burr defect is mainly caused by serious abrasion of the saw blade or improper adjustment of the saw blade when the steel material is sawed. Therefore, the defects on the surface of steel are various in types, the difference of the characteristics of the defects of different types is large, so that the precision difference of different defects in detection is large, the precision of defect detection of a part of small targets is unsatisfactory, and meanwhile, the background of a defect image on the surface of steel is complex, the texture is disordered, and the condition of coverage crossing between the detected targets can also occur.
Disclosure of Invention
The invention aims to provide a steel surface defect detection method based on improved YOLOv5, which enhances the detection effect on small target defects and can effectively focus on the detection of task targets.
In order to achieve the purpose, the invention adopts the following technical scheme to realize the purpose:
a steel surface defect detection method based on improved YOLOv5 comprises the following steps:
step 2, carrying out defect labeling on the collected steel surface image by using an image labeling tool, and forming a steel surface defect data set required by a model by using the obtained labeling file and the original image;
step 4, constructing a steel surface defect detection model: adding a small target detection layer for identifying image characteristics in a small range on the basis of three detection layers of an original YOLOv5 network, introducing a Trans module into the tail end and the neck part of a backbone of the YOLOv5 network, replacing a part of a CONV module and a CSP2_1 module, and introducing a CBAM module into the neck part of the YOLOv5 network so as to be matched with the Trans module and the Conv module to enhance the performance of the network;
step 5, training a steel surface defect detection model;
and 6, predicting a steel surface defect detection model.
Further, the defect labeling of step 2 includes labeling the type of the defect and its position in the image.
Further, the data enhancement in step 3 is performed by adding gaussian noise, randomly horizontally flipping images, randomly vertically flipping images, rotating images, cropping images, and dithering colors.
Further, the adding of the gaussian noise specifically includes: a) Generating a random number matrix conforming to a gaussian distribution using np. b) Adding the original image and the generated random number matrix to obtain an image added with Gaussian noise;
the random horizontal turnover image is used for horizontally turning over the image and the labeling boundary frame with the probability of 0.5;
the random vertical turnover image vertically turns over the image and the labeling boundary frame with the probability of 0.5;
the rotating image is to rotate the image in any direction so as to increase the data volume;
the cutting image is highlighted by removing unnecessary parts in the picture and only reserving important parts through a slicing function of a NumPy matrix;
the color dithering is to randomly adjust the saturation, brightness and contrast of the original picture to generate a new image.
Further, the Trans module introduced in step 4 comprises a multi-head attention layer and a full connection layer which are connected through a residual error network, so that the characteristic information is effectively extracted, and the calculation and storage expenses are reduced;
the CBAM module includes two sequential sub-modules connected by a residual network for refining the feature map through CBAM and enhancing the extraction of features.
The method for detecting the surface defect of the steel product based on the improved YOLOv5 as claimed in claim 1, wherein the predicting of the surface defect of the steel product in the step 6 specifically comprises: firstly, a trained surface defect detection model is used for reasoning, then a product surface image with a concentrated verification is input, and finally the detected defect type and position are output and displayed.
The invention has the following beneficial effects:
according to the invention, a small target detection layer is added on the basis of the original YOLOv5 network, so that the detection effect on the small target defects is enhanced, the network can effectively resist against a complex background by introducing a Trans module and a CBAM module, the detection of a task target is effectively focused, the overall average detection accuracy (mAP) of the improved YOLOv5 network is improved by about 6% compared with the original YOLOv5 network, and the mAP is improved by about 10% compared with a Faster-RCNN model taking Resnet50 as a backbone network. Therefore, the detection method of the invention not only has high precision in the detection of the steel surface defects, but also can effectively avoid the condition of overlapping and crossing among detection targets caused by complex background and disordered texture factors of the steel surface defect images.
The detection method can also be used in other industrial application scenes, particularly in the field of target detection with small targets and complex backgrounds.
Drawings
FIG. 1: comparison of detection layers of the invention;
FIG. 2 is a schematic diagram: the structure of the Trans module of the invention;
FIG. 3: the structure diagram of the CBAM module of the invention;
FIG. 4: the structure diagram of the improved YOLOv5 network;
FIG. 5 is a schematic view of: the invention relates to a steel surface defect data set;
FIG. 6: the confusion matrix of the present invention;
FIG. 7: the invention relates to a histogram of steel surface defect detection condition;
FIG. 8: the curve graph of the change condition of the mAP along with the iteration times is shown;
FIG. 9: the invention discloses a schematic diagram of an IoU calculation method.
Detailed Description
The following examples are given to illustrate the present invention in further detail, but are not intended to limit the scope of the present invention.
As shown in fig. 1 to 5, a method for detecting surface defects of a steel material based on improved YOLOv5 includes the following steps:
step 2, marking the acquired steel surface image with an image marking tool to mark the defects, mainly marking the types and positions of the defects in the image, and forming a steel surface defect data set required by a model by using the obtained marking file and the original image;
wherein: the data set contains 6 common steel surface defects: crazing, inclusions, patches, pitted surfaces, rolled-in scales, and scratches;
wherein: the gaussian noise is a type of noise whose probability density function of noise distribution follows gaussian distribution (normal distribution), and appears at all positions in an image, and is mainly caused by dark field of view and uneven brightness during shooting, and besides, the noise of electronic components and their mutual influence are also important causes of gaussian noise, and the probability density function of gaussian noise is shown as the following formula:
wherein z represents the gray scale value of an image pixel; μ represents an average or expected value of the pixel values; σ denotes the standard deviation of the pixel, σ 2 Represents the square of the standard deviation;
the np.random.normal () function that can generate a random number conforming to a gaussian distribution (normal distribution) is provided in NumPy;
random. Normal () function prototype is as follows:
output＝np.random.normal(loc,
scale
[,size])
the meaning of each return value and parameter in the function is respectively as follows: loc: mean of gaussian distribution; scale: standard deviation of gaussian distribution; size: (ii) the output random data dimension;
the function is used for generating a random number which is in a specified shape and accords with Gaussian distribution, and returning a generated result through a value, wherein the 1 st parameter corresponds to the center of the whole Gaussian distribution; the 2 nd parameter corresponds to the width of the Gaussian distribution; the larger the scale value, the flatter the distribution; otherwise, the distribution is towering; the 3 rd parameter is an optional parameter, the output size can be set according to requirements, and if the output size is not specified, only a single value is output. When the parameter loc is set to 0 and scale is set to 1, the standard normal distribution is represented;
the adding of the gaussian noise specifically comprises: a) Generating a random number matrix conforming to a gaussian distribution using np. b) Adding the original image and the generated random number matrix to obtain an image added with Gaussian noise;
the random horizontal turnover image is used for horizontally turning over the image and the labeling boundary frame with the probability of 0.5;
the random vertical turnover image vertically turns over the image and the labeling boundary frame with the probability of 0.5;
the image rotation is to rotate the image in any direction (for example, up and down, left and right) so as to increase the data amount;
the cutting image is highlighted by removing unnecessary parts in the picture and only reserving important parts through a slicing function of a NumPy matrix;
the color dithering is to randomly adjust the saturation, brightness and contrast of the original picture to generate a new image;
step 4, constructing a steel surface defect detection model, which specifically comprises the following steps:
4.1 Adding a small target detection layer on the basis of three detection layers of the original YOLOv5 network for identifying image features in a small range;
4.2 A Trans module is introduced into the terminal and the neck part of the YOLOv5 network backbone and replaces part of the CONV module and CSP2_1 module;
wherein: the structure of the Trans module is shown in fig. 2, the Trans module includes two sublayers, namely a multi-head attention layer and a full connection layer, which are connected through a residual network, the LayerNorm and Dropout in the two sublayers can help the network convergence and effectively prevent the network overfitting, and the multi-head attention structure not only can help the current node to focus on the current pixel, but also can effectively obtain the semantics of the context; the reason why the Trans module is applied to the tail end and the neck part of the backbone is that the resolution of the picture information at this stage is low, so that the calculation and storage expenses can be reduced while the feature information can be effectively extracted;
4.3 Then introducing the CBAM module into the neck part of the YOLOv5 network so as to cooperate with the Trans module and the Conv module to enhance the performance of the network;
wherein: the CBAM module is a simple and effective attention module, which can be integrated into the CNN architecture and can be trained in an end-to-end manner, and for given feature data, it sequentially infers an attention map along two independent dimensions of channel and space, and then multiplies the attention map by the input image features for adaptive feature refinement, and the structure of the CBAM module is as shown in fig. 3, which uses two sequential sub-modules for refining the feature map passing through the CBAM, and simultaneously uses a residual network to connect the two, thereby enhancing the feature extraction. The module is mainly applied to a neck part and is matched with a Trans module and a Conv module to be used for enhancing the performance of a network; the introduction of the CBAM module and the Trans module is more favorable for capturing global information, resisting chaotic information and effectively processing complex backgrounds; the structure of YOLOv5 modified by introducing the Trans module and CBAM module is shown in fig. 4;
step 5, training a steel surface defect detection model; after the original dataset is enhanced by data in step 3, the new dataset contains 1800 samples in total, wherein: 1422 training sets and 378 verification sets;
and 6, predicting the steel surface defect detection model, reasoning by using the trained surface defect detection model, inputting the product surface image with concentrated verification, and finally outputting and displaying the detected defect type and position.
Preferably, the hardware environment of the steel surface defect detection method based on the improved YOLOv5 in the embodiment is as follows:
1) A processor: intel i7-7700U CPU @2.80GHz;
2) Computer memory: 16GB;
3) A display card: GPU NVIDIA GeForce GTX1050ti.
Preferably, the software environment of the steel surface defect detection method based on the improved YOLOv5 in the embodiment is as follows:
1) Operating the system: windows10 bit;
2) And (3) developing a language: python 3.7.12;
3) Module library: pitor 1.10.2, numpy 1.19.5, opencv-python 4.5, cuda 10.2.95, tensorboard 2.8.0
The evaluation indexes of the steel surface defect detection method based on the improved YOLOv5 in the embodiment are as follows:
regarding the evaluation index of the target detection field, firstly, the accuracy of the detection frame is considered, the index is mainly measured by an IoU index, namely the intersection ratio of the areas of the detection frame and the target frame, as shown in the following figure, in the actual operation, whether the prediction in the step 6 is effective is judged according to the size of the IoU, the prediction is judged to be correct when the IoU is more than or equal to 0.5, and the calculation method of the IoU is shown in FIG. 9;
the division of the detection result can be represented by a confusion matrix shown in fig. 7, and can be divided into the following four cases TP (correctly detected positive samples), FN (incorrectly detected positive samples), FP (incorrectly detected negative samples), TN (correctly detected negative samples) according to the combination of the prediction result and the actual situation, and the statistical results of the above four cases can obtain the following three indexes: precision (Precision, P), recall (Recall, R), and accuracy (Aaccuracy, a); the comprehensive evaluation indexes on the basis are Average Precision (AP) and Average Precision mean (mAP), wherein: AP represents the average precision of all detection results under the same category, and mAP represents the average value of the average precision of all detection categories;
in order to prove the detection effect of the improved model and verify the function of specific improvement measures, the present embodiment further sets the existing YOLOv5s network group and the fast-RCNN network group, which are respectively called as V5-0 and FF-RCNN as comparison groups, and also sets three ablation experimental groups, including: adding a small target detection layer group, a Trans module and CBAM module group and a final optimization group which are respectively and sequentially called as V5-1, V5-2 and V5-3; the overall results of the experiment are shown in table 1:
table 1: overall results of the experiment
According to the data in Table 1, the optimized YOLOv5s network (V5-3) shows an improvement of mAP of about 6% compared to the original YOLOv5s network (V5-0), and an improvement of mAP of about 10% compared to the optimized YOLOv5s network (V5-3) compared to fast-RCNN; meanwhile, the comparison with the results of the ablation experimental groups (V5-1 and V5-2) shows that the improved YOLOv5s network of the invention has good effect on the improvement measures of the original network, and the precision is improved by about 4-5%.
The original YOLOv5s network (V5-0), the small target detection layer group (V5-1), the Trans module and CBAM module group (V5-2) and the improved YOLOv5s network (V5-3) are added for detecting the defects on the surface of the steel products, and the detection conditions are shown in FIG. 7.
As shown in fig. 8, the improved YOLOv5s network can effectively improve the detection capability for both rolled-in _ scale (rolled scale) and scratches by adding a small target detection layer, and effectively enhance the network's capability for processing complex background information by introducing an attention module, which makes the best of the two to obtain a good detection result, and the optimal weight in fig. 8 is generated at 230 th iteration.
Claims (6)
1. A method for detecting surface defects of steel based on improved YOLOv5 is characterized by comprising the following steps:
step 1, acquiring an image of the surface of a steel material to be detected through an industrial camera;
step 2, carrying out defect labeling on the collected steel surface image by using an image labeling tool, and forming a steel surface defect data set required by a model by using the obtained labeling file and the original image;
step 3, performing data enhancement on the obtained data set and randomly dividing the enhanced data set sample into a training set and a verification set;
step 4, constructing a steel surface defect detection model: adding a small target detection layer for identifying image characteristics in a small range on the basis of three detection layers of an original YOLOv5 network, introducing a Trans module into the tail end and the neck part of a backbone of the YOLOv5 network and replacing part of a CONV module and a CSP2_1 module, and introducing a CBAM module into the neck part of the YOLOv5 network so as to be matched with the Trans module and the Conv module to enhance the performance of the network;
step 5, training a steel surface defect detection model;
and 6, predicting a steel surface defect detection model.
2. The improved YOLOv 5-based steel surface defect detection method as claimed in claim 1, wherein the defect labeling of step 2 comprises labeling the type of the defect and its position in the image.
3. The method for detecting the defects on the surface of the steel material based on the improved YOLOv5 as claimed in claim 1, wherein the data enhancement in the step 3 is performed by means of Gaussian noise addition, random horizontal flip images, random vertical flip images, image rotation, image cutting and color dithering.
4. The improved YOLOv 5-based steel surface defect detection method according to claim 3, wherein the adding of Gaussian noise specifically comprises: a) Generating a random number matrix conforming to a gaussian distribution using np. b) Adding the original image and the generated random number matrix to obtain an image added with Gaussian noise;
the random horizontal turnover image is used for horizontally turning over the image and the labeling boundary box at the probability of 0.5;
the random vertical turnover image vertically turns over the image and the labeling boundary frame with the probability of 0.5;
the rotation image is to rotate the image in any direction so as to increase the data amount;
the cutting image is highlighted by removing unnecessary parts in the picture and only reserving important parts through a slicing function of a NumPy matrix;
the color dithering is to randomly adjust the saturation, brightness and contrast of the original picture to generate a new image.
5. The improved YOLOv 5-based steel surface defect detection method according to claim 1, wherein the Trans module introduced in the step 4 comprises a multi-head attention layer and a full connection layer which are connected through a residual error network, so that characteristic information can be effectively extracted, and calculation and storage costs can be reduced;
the CBAM module includes two sequential sub-modules connected by a residual network for refining the feature map through CBAM and enhancing the extraction of features.
6. The method for detecting the surface defect of the steel product based on the improved YOLOv5 as claimed in claim 1, wherein the predicting of the surface defect of the steel product in the step 6 specifically comprises: firstly, a trained surface defect detection model is used for reasoning, then a product surface image with a concentrated verification is input, and finally the detected defect type and position are output and displayed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210990916.0A CN115294089A (en) | 2022-08-18 | 2022-08-18 | Steel surface defect detection method based on improved YOLOv5 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210990916.0A CN115294089A (en) | 2022-08-18 | 2022-08-18 | Steel surface defect detection method based on improved YOLOv5 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115294089A true CN115294089A (en) | 2022-11-04 |
Family
ID=83829670
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210990916.0A Pending CN115294089A (en) | 2022-08-18 | 2022-08-18 | Steel surface defect detection method based on improved YOLOv5 |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115294089A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116630301A (en) * | 2023-06-20 | 2023-08-22 | 盐城工学院 | Strip steel surface small target defect detection method and system based on super resolution and YOLOv8 |
| CN117274192A (en) * | 2023-09-20 | 2023-12-22 | 重庆市荣冠科技有限公司 | Pipeline magnetic flux leakage defect detection method based on improved YOLOv5 |
| CN117333491A (en) * | 2023-12-01 | 2024-01-02 | 北京航空航天大学杭州创新研究院 | Steel surface defect detection method and system |
-
2022
- 2022-08-18 CN CN202210990916.0A patent/CN115294089A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116630301A (en) * | 2023-06-20 | 2023-08-22 | 盐城工学院 | Strip steel surface small target defect detection method and system based on super resolution and YOLOv8 |
| CN117274192A (en) * | 2023-09-20 | 2023-12-22 | 重庆市荣冠科技有限公司 | Pipeline magnetic flux leakage defect detection method based on improved YOLOv5 |
| CN117333491A (en) * | 2023-12-01 | 2024-01-02 | 北京航空航天大学杭州创新研究院 | Steel surface defect detection method and system |
| CN117333491B (en) * | 2023-12-01 | 2024-03-15 | 北京航空航天大学杭州创新研究院 | Steel surface defect detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115294089A (en) | Steel surface defect detection method based on improved YOLOv5 | |
| CN111860171B (en) | Method and system for detecting irregular-shaped target in large-scale remote sensing image | |
| CN111652085B (en) | Object identification method based on combination of 2D and 3D features | |
| CN104866862B (en) | A kind of method of belt steel surface area-type defect recognition classification | |
| CN110532946B (en) | Method for identifying axle type of green-traffic vehicle based on convolutional neural network | |
| CN108765412A (en) | A kind of steel strip surface defect sorting technique | |
| CN113643228B (en) | Nuclear power station equipment surface defect detection method based on improved CenterNet network | |
| CN109101976B (en) | Method for detecting surface defects of arc-extinguishing grid plate | |
| CN113256601B (en) | Pavement disease detection method and system | |
| CN109215009B (en) | Continuous casting billet surface image defect detection method based on deep convolution neural network | |
| CN111968098A (en) | Strip steel surface defect detection method, device and equipment | |
| CN112614105B (en) | Depth network-based 3D point cloud welding spot defect detection method | |
| CN115601355A (en) | Method and device for detecting and classifying product surface defects and storage medium | |
| CN114926407A (en) | Steel surface defect detection system based on deep learning | |
| CN110751619A (en) | Insulator defect detection method | |
| CN113240623A (en) | Pavement disease detection method and device | |
| CN112819748A (en) | Training method and device for strip steel surface defect recognition model | |
| CN114972316A (en) | Battery case end surface defect real-time detection method based on improved YOLOv5 | |
| Wen-ming et al. | Welding defect detection of x-ray images based on faster r-cnn model | |
| CN116958052A (en) | Printed circuit board defect detection method based on YOLO and attention mechanism | |
| CN111597939B (en) | High-speed rail line nest defect detection method based on deep learning | |
| Xu et al. | The steel surface multiple defect detection and size measurement system based on improved yolov5 | |
| CN117593244A (en) | Film product defect detection method based on improved attention mechanism | |
| CN116258908A (en) | Ground disaster prediction evaluation classification method based on unmanned aerial vehicle remote sensing image data | |
| CN115359306A (en) | Intelligent identification method and system for high-definition images of railway freight inspection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |