CN116721078A - Strip steel surface defect detection method and device based on deep learning - Google Patents
Strip steel surface defect detection method and device based on deep learning Download PDFInfo
- Publication number
- CN116721078A CN116721078A CN202310678776.8A CN202310678776A CN116721078A CN 116721078 A CN116721078 A CN 116721078A CN 202310678776 A CN202310678776 A CN 202310678776A CN 116721078 A CN116721078 A CN 116721078A
- Authority
- CN
- China
- Prior art keywords
- strip steel
- steel surface
- surface defect
- network
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 116
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 101
- 239000010959 steel Substances 0.000 title claims abstract description 101
- 238000001514 detection method Methods 0.000 title claims abstract description 52
- 238000013135 deep learning Methods 0.000 title claims abstract description 19
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 28
- 238000004519 manufacturing process Methods 0.000 claims abstract description 17
- 230000008569 process Effects 0.000 claims abstract description 9
- 238000012360 testing method Methods 0.000 claims abstract description 8
- 238000012795 verification Methods 0.000 claims abstract description 8
- 230000003044 adaptive effect Effects 0.000 claims description 42
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 5
- 238000003709 image segmentation Methods 0.000 claims description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 239000000284 extract Substances 0.000 claims description 2
- 230000010354 integration Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 230000005284 excitation Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000001125 extrusion Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000000087 stabilizing effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The application discloses a strip steel surface defect detection method and device based on deep learning, wherein the method comprises the following steps: collecting original images of the surface of the strip steel on a strip steel production line; marking the defect types of the collected original images, and integrating the original images and marking data into a strip steel surface defect data set; dividing the strip steel surface defect data set in a certain proportion to obtain a training set, a verification set and a test set; constructing a strip steel surface defect detection model, wherein the model comprises a main network, a focusing characteristic pyramid network and a detection head; training a network by using the strip steel surface defect data set to obtain a trained model, and storing a weight file in the training process; and detecting the strip steel surface defect image by using the trained model to obtain the type and position information of the defect in the image. The application effectively improves the accuracy and speed of the detection of the surface defects of the strip steel.
Description
Technical Field
The application relates to the field of machine vision image processing, in particular to a strip steel surface defect detection method and device based on deep learning.
Background
As one of important raw materials in the manufacturing industry, the strip steel is widely applied to the fields of automobile industry, aerospace, instrument manufacturing and the like. At present, with the vigorous development of the high-precision industry, the requirements of manufacturing industries on the quality of strip steel products are continuously increasing. However, due to the influence of factors such as production equipment, production environment and manufacturing level, various defects on the surface of the strip steel inevitably occur in the production process, hidden dangers are easily caused when the low-quality strip steel with the defects is used for subsequent production and processing, and even the strip steel is directly scrapped in serious cases, so that the production benefits of the whole industrial chain are destroyed. Therefore, the defects on the surface of the strip steel can be detected and treated in time, so that the defects of the strip steel can be effectively controlled, the economic loss of enterprises can be reduced, and the development of related industries can be promoted. Therefore, the research on the defect automatic detection algorithm based on deep learning has very important practical value for improving the quality of the strip steel. In the prior art, the defect detection technology based on deep learning solves the problem of manual quality inspection to a certain extent, but can hardly meet the actual demand in the actual industrial scene, and the detection level still has the defects: on one hand, complex noise exists on the surface defect of the complex strip steel, and the algorithm is extremely easy to cause the problem of missing detection and missing detection on the defect. On the other hand, in the actual industrial scenes with high real-time requirements, most of the existing algorithms have slower overall detection speed and slow down the production process of the strip steel.
Disclosure of Invention
Aiming at the problems in the prior art, the application aims to provide a strip steel surface defect detection method and device based on deep learning, which effectively improve the accuracy and speed of strip steel surface defect detection.
The application adopts the following technical scheme:
in one aspect, a method for detecting surface defects of strip steel based on deep learning includes:
step S1, collecting original images of the surface of strip steel on a strip steel production line;
s2, marking the defect types of the collected original image, and integrating the original image and marking data into a strip steel surface defect data set;
s3, dividing the strip steel surface defect data set in a certain proportion to obtain a training set, a verification set and a test set;
s4, constructing a strip steel surface defect detection model, wherein the model comprises a main network, a focusing characteristic pyramid network and a detection head;
s5, training a network by using the strip steel surface defect data set to obtain a trained model, and storing a weight file in the training process;
s6, detecting a strip steel surface defect image by using a trained model to obtain the type and position information of the defect in the image;
wherein, the backbone network extracts three different scale feature layers C 1 、C 2 And C 3 C is carried out by 1 、C 2 And C 3 Transmitting to a focusing type characteristic pyramid network; the focusing type characteristic pyramid network is used for completing recalibration of the characteristic in the neck network to obtain a characteristic layer P 1 、P 2 And P 3 The method comprises the steps of carrying out a first treatment on the surface of the In the detection head part, for the feature layer P 1 、P 2 And P 3 And respectively splitting to realize decoupling prediction.
Preferably, the marking of defect types is performed on the collected original image, and the original image and marking data are integrated into a strip steel surface defect data set, which specifically includes:
carrying out data annotation by using labelImg software to obtain an annotation file, wherein the format of the annotation file is XML; and integrating the original image and the labeling data into a strip steel surface defect data set.
Preferably, the backbone network comprises image segmentation, linear embedding, SWTR structure blocks and merging blocks, and performs feature extraction in four stages; in the first stage, the input image is segmented by using image segmentation and linear embedding, marks are linearly embedded into each image block, and the linear image blocks are extracted by using the attention mechanism of the SWTR structural blocks; in the second stage, the transmitted feature layer is downsampled by using a merging block, and feature extraction is realized on the feature layer by using an SWTR structure block to obtain a feature layer C1; repeating the operation of the second stage in the third stage and the fourth stage to obtain feature layers C2 and C3; and transmitting the three different-scale feature layers C1, C2 and C3 extracted by the backbone network to a focusing feature pyramid network.
Preferably, the focusing characteristic pyramid network comprises a CBS convolution module, an adaptive focusing block, an adaptive focusing residual block and upsampling; in a focusing type characteristic pyramid network, three characteristic layers with different scales are utilized for carrying out enhanced characteristic extraction; in the small feature layer of 7×7 pixels, for C 3 Performing continuous two adaptive focusing blocks and up-sampling to obtain a characteristic layer C 2 'A'; for C 2 ' continue to use 1 adaptive focusing block, upsample to obtain feature layer P 1 'A'; for C 1 Channel adjustment using 1 adaptive focusing block and P 1 ' 1-time addition operation to obtain a feature layer P 1 The method comprises the steps of carrying out a first treatment on the surface of the In the middle feature layer of 14×14 pixels, for C 2 Obtaining a feature layer P using consecutive 1 adaptive focusing block and 1 adaptive focusing residual block 2 ' P is to 2 ' and C 2 ' 1-time addition operation to obtain a feature layer P 2 The method comprises the steps of carrying out a first treatment on the surface of the In the case of a large feature layer of 28 x 28 pixels, for an adjusted C 1 Downsampling of the CBS convolution module 1 time and using two consecutive adaptive focusing blocks for it, then for P 2 ' use 1 adaptive focusing residual block, use cascade operation to perform channel adjustment of CBS convolution module on feature layer, use 1 adaptive focusing residual block again and use C with C 3 Adding the adjusted characteristic layer to obtain a characteristic layer P 3 。
Preferably, in the detection header part, 3 yoloxhaead modules are used for feature layer P 1 、P 2 And P 3 Dividing to realize decoupling prediction, wherein one side of decoupling uses a CBS convolution module with a 1-to-3 x 3 convolution kernel size and adjusts channels into the number of defect types to predict classification results; the other side uses a base convolution module of 3 x 3 convolution kernel size and adapts the channels to predict the four points of the anchor frame and determine the existence of the target, respectively.
Preferably, the Loss function Loss used in the training of the strip steel surface defect detection model is as follows:
Loss＝λLoss reg +Loss cls +Loss obj
wherein, loss reg Representing regression loss, calculated using IoU loss; loss (Low Density) cls Representing Loss of classification, loss obj Representing target losses, all calculated using BCEWithLogits losses; lambda is the weight and Adam is used by the optimizer.
In another aspect, a strip steel surface defect detection device based on deep learning includes:
the original image acquisition module is used for acquiring original images of the surface of the strip steel on a strip steel production line;
the defect data set integration module is used for marking the defect types of the collected original images and integrating the original images and marking data into a strip steel surface defect data set;
the defect data set dividing module is used for dividing the strip steel surface defect data set in a certain proportion to obtain a training set, a verification set and a test set;
the model construction module is used for constructing a strip steel surface defect detection model, and the model comprises a main network, a focusing characteristic pyramid network and a detection head;
the model training module is used for training the network by using the strip steel surface defect data set to obtain a trained model, and storing a weight file in the training process;
and the strip steel surface detection module is used for detecting the strip steel surface defect image by using the trained model to obtain the type and position information of the defect in the image.
Compared with the prior art, the application has the following beneficial effects:
1. according to the application, an anchor-frame-free YOLOX network is used, SWTR is introduced as a main network, the reasoning time is quickened in the detection, the global modeling capacity of the network is fully improved, the network can capture the long-distance dependence of the characteristics more effectively, the defect target area is detected more completely finally, and the detection efficiency of the detection equipment in an actual strip steel generation scene can be fully improved by the efficient combination mode;
2. according to the application, the focusing of the network on the SWTR captured different scale features can be effectively improved by designing the focusing feature pyramid network. The pyramid structure based on convolution integrally enhances the flexibility of the network to capture the characteristics, and simultaneously performs certain local connection and characteristic recalibration on the defect characteristics so as to continuously adjust the focusing of the network on the defects; the continuous pyramid transmission structure effectively integrates the characteristics of the feature layers at different stages, so that the local space aggregation capability of the detection network is obviously improved, and the position and type of the defect can be accurately detected when the network is used for dealing with strip steel surface defect targets with changeable forms;
3. the self-adaptive focusing block is designed to realize self-adaption and characteristic recalibration of the network to the strip steel surface defect form, so that the network is more flexible when facing a defect target, the network is fully focused on the effective characteristics of the strip steel surface defect, and the problems of easy detection lack and detection omission when detecting the strip steel defect target with large scale change and large irrelevant noise are solved.
Drawings
FIG. 1 is a flow chart of a method for detecting surface defects of strip steel based on deep learning according to an embodiment of the application;
FIG. 2 is a schematic diagram of a CBS convolution module embodying the present application;
FIG. 3 is a schematic diagram of an adaptive feature extractor according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an adaptive focusing block according to an embodiment of the present application;
FIG. 5 is a schematic diagram of an adaptive focusing residual block according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a Yoloxhead module according to an embodiment of the application;
FIG. 7 is a schematic diagram of an extrusion excitation block according to an embodiment of the present application;
FIG. 8 is a schematic diagram of a network for detecting surface defects of a strip steel according to an embodiment of the present application;
fig. 9 is a block diagram of a device for detecting surface defects of a strip steel based on deep learning according to an embodiment of the application.
Detailed Description
The application will be further illustrated with reference to specific examples. It is to be understood that these examples are illustrative of the present application and are not intended to limit the scope of the present application. Furthermore, it should be understood that various changes and modifications can be made by one skilled in the art after reading the teachings of the present application, and such equivalents are intended to fall within the scope of the application as defined in the appended claims.
In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It should be noted that, without conflict, the embodiments of the present application and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, and the described embodiments are merely some, rather than all, embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
The terms first, second, third and the like in the description and in the claims of the application and in the above-described figures, are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Referring to fig. 1, the method for detecting the surface defects of the strip steel based on deep learning in the embodiment includes the following steps:
step S1, collecting original images of the surface of the strip steel on a strip steel production line.
And S2, marking the defect types of the collected original image, and integrating the original image and marking data into a strip steel surface defect data set.
Specifically, labelImg software is used for data annotation, and an annotation file is obtained, and the format of the annotation file is XML. Integrating the original image and the labeling data into a strip steel surface defect data set;
and step S3, dividing the strip steel surface defect data set in a certain proportion to obtain a training set, a verification set and a test set.
And S4, constructing a strip steel surface defect detection model, wherein the model comprises a main network, a focusing type characteristic pyramid network and a detection head.
Specifically, the system comprises the following basic modules:
CBS convolution modules, including convolution, batch normalization, and the Silu activation function, are shown in FIG. 2.
An adaptive feature extractor comprising a CBS convolution module, a deformable convolution and a squeeze excitation block is shown in fig. 3.
An adaptive focusing block comprising a CBS convolution module and an adaptive feature extractor is shown in fig. 4.
The adaptive focusing residual block, comprising an adaptive focusing block and a residual edge, is shown in fig. 5.
A focused feature pyramid network comprising CBS convolution module, adaptive focusing block, adaptive focusing residual block, and upsampling, as shown in fig. 6.
The yoloxhaad module, including CBS convolution module, is used to decouple classification and regression tasks and adjust channels to achieve predictions of defect type, target, location, as shown in fig. 7.
The backbone network SWTR (Swin Transformer) comprises an image block, a linear embedding block, an SWTR structure block and a merging block, and is used for carrying out global self-attention on the image so as to realize efficient feature extraction and improve global modeling capacity of the whole network. The CBS convolution module comprises convolution, batch normalization and activation functions. The method is used for channel adjustment and preliminary feature extraction. The self-adaptive feature extractor comprises a CBS convolution module, a deformable convolution and extrusion excitation block and is used for effectively focusing irregular and widely-distributed strip steel surface defect features and achieving more comprehensive and complete capture of defects. The self-adaptive focusing block comprises a CBS convolution module and a self-adaptive feature extractor, and is used for extracting deep features of the network. The adaptive focusing residual block comprises an adaptive focusing block and a residual edge and is used for stabilizing the expression of output characteristics. The focusing type feature pyramid network comprises a CBS convolution module, an adaptive focusing block, an adaptive focusing residual block and upsampling, and is used for establishing local connection for a feature layer so as to promote the capture of the key defect features by the network. The yoloxhaad module comprises a CBS convolution module for decoupling classification and regression tasks and adjusting channels to realize prediction of defect types, targets and positions.
Referring to fig. 8, the constructed network model performs the steps of:
step S4-1, three different scale feature layers are extracted through a backbone network Swin Transformer, and the method specifically comprises the following steps: and performing blocking operation on the input image, and splitting the image into 4 blocks of 48-dimensional feature layers according to the input proportion. Next, the 4 feature layers are linearly embedded, the feature dimensions of each feature layer are mapped into 96 dimensions and a fully connected operation is achieved. Then, feature extraction is performed using a window-based attention mechanism and a sliding window operation in the SWTR structure block, and an initial feature layer is obtained. And carrying out block merging on the initial characteristic layers, and carrying out downsampling compression on the characteristic layers across single pixels so as to carry out deep characteristic extraction work. Repeating the three-time stepwise block merging and SWTR structure block operation to obtain three feature layers with different scales after feature extraction.
Step S4-2, focusing the SWTR captured different scale features through a neck network focusing feature pyramid network, and enhancing the expression of the different scale features, wherein the method specifically comprises the following steps:
in the small feature layer of 7×7 pixels, for C 3 Performing continuous two adaptive focusing blocks and up-sampling to obtain a characteristic layer C 2 '. For C 2 ' continue to use 1 adaptive focusing block, upsample to obtain feature layer P 1 '. For C 1 Channel adjustment using 1 adaptive focusing block and P 1 ' 1-time addition operation to obtain a feature layer P 1 ；
In the middle feature layer of 14×14 pixels, for C 2 Obtaining a feature layer P using consecutive 1 adaptive focusing block and 1 adaptive focusing residual block 2 ' P is to 2 ' and C 2 ' 1-time addition operation to obtain a feature layer P 2 ；
In the case of a large feature layer of 28 x 28 pixels, for an adjusted C 1 Downsampling of the CBS convolution module 1 time and using two consecutive adaptive focusing blocks for it, then for P 2 ' use 1 adaptive focusing residual block, use cascade operation to perform channel adjustment of CBS convolution module on feature layer, use 1 adaptive focusing residual block again and use C with C 3 Adding the adjusted characteristic layer to obtain a characteristic layer P 3 . Thus, recalibration of the feature in the neck network is completed.
S4-3, generating a strip steel surface defect detection model through YOLOXHEAD, wherein the method specifically comprises the following steps:
feature layer P using 3 Yoloxhead modules 1 、P 2 And P 3 Segmentation is performed to realize decoupling prediction, and a CBS convolution module with a 1-to-3×3 convolution kernel size is used on one side of decoupling and channels are adjusted to the number of defect types to predict segmentationClass results; the other side uses a base convolution module of 3 x 3 convolution kernel size and adapts the channels to predict the four points of the anchor frame and determine the existence of the target, respectively.
Step S5, training the network by using the strip steel surface defect data set to obtain a trained model, and storing a weight file in the training process, wherein the method specifically comprises the following steps:
step S5-1, dividing the strip steel surface defect data set obtained in the step S3 into a proportion of 8:1:1, a training set, a testing set and a verification set;
s5-2, training a network by using the divided strip steel surface defect data set to generate a strip steel surface defect detection model;
the Loss function Loss used in the training of the strip steel surface defect detection model is as follows:
Loss＝λLoss reg +Loss cls +Loss obj
wherein, loss reg Representing regression loss, calculated using IoU loss; loss (Low Density) cls Representing Loss of classification, loss obj Representing target losses, all calculated using BCEWithLogits losses; lambda is the weight and Adam is used by the optimizer.
The image input size was 224×224, the learning rate was 0.00005, the momentum was 0.9, and the weight decay was 0.0005,batch size 32.
And S6, detecting the strip steel surface defect image by using the trained model, and obtaining the type and position information of the defect in the image.
Referring to fig. 9, the embodiment also discloses a strip steel surface defect detection device based on deep learning, which comprises:
the original image acquisition module 901 is used for acquiring an original image of the surface of the strip steel on a strip steel production line;
the defect data set integrating module 902 is configured to label the collected original image with a defect type, and integrate the original image and the label data into a strip steel surface defect data set;
the defect data set dividing module 903 is configured to divide the surface defect data set of the strip steel in a certain proportion to obtain a training set, a verification set and a test set;
the model construction module 904 is used for constructing a strip steel surface defect detection model, and the model comprises a main network, a focusing characteristic pyramid network and a detection head;
the model training module 905 is configured to train the network to obtain a trained model by using the strip steel surface defect data set, and store a weight file in the training process;
and the strip steel surface detection module 906 is used for detecting the strip steel surface defect image by using the trained model to obtain the type and position information of the defect in the image.
It will be evident to those skilled in the art that the application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it will be obvious that the term "comprising" does not exclude other elements or that the singular does not exclude a plurality. A plurality of units or means recited in the system claims can also be implemented by means of software or hardware by means of one unit or means.
Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present application without departing from the spirit and scope of the technical solution of the present application.
Claims (7)
1. The strip steel surface defect detection method based on deep learning is characterized by comprising the following steps of:
step S1, collecting original images of the surface of strip steel on a strip steel production line;
s2, marking the defect types of the collected original image, and integrating the original image and marking data into a strip steel surface defect data set;
s3, dividing the strip steel surface defect data set in a certain proportion to obtain a training set, a verification set and a test set;
s4, constructing a strip steel surface defect detection model, wherein the model comprises a main network, a focusing characteristic pyramid network and a detection head;
s5, training a network by using the strip steel surface defect data set to obtain a trained model, and storing a weight file in the training process;
s6, detecting a strip steel surface defect image by using a trained model to obtain the type and position information of the defect in the image;
wherein, the backbone network extracts three different scale feature layers C 1 、C 2 And C 3 C is carried out by 1 、C 2 And C 3 Transmitting to a focusing type characteristic pyramid network; the focusing type characteristic pyramid network is used for completing recalibration of the characteristic in the neck network to obtain a characteristic layer P 1 、P 2 And P 3 The method comprises the steps of carrying out a first treatment on the surface of the In the detection head part, for the feature layer P 1 、P 2 And P 3 And respectively splitting to realize decoupling prediction.
2. The method for detecting the surface defects of the strip steel based on the deep learning according to claim 1, wherein the marking of the defect type is performed on the collected original image, and the original image and the marking data are integrated into a strip steel surface defect data set, specifically comprising:
carrying out data annotation by using labelImg software to obtain an annotation file, wherein the format of the annotation file is XML; and integrating the original image and the labeling data into a strip steel surface defect data set.
3. The deep learning-based strip steel surface defect detection method according to claim 1, wherein the backbone network comprises image segmentation, linear embedding, SWTR structure blocks and merging blocks, and the four-stage feature extraction is performed; in the first stage, the input image is segmented by using image segmentation and linear embedding, marks are linearly embedded into each image block, and the linear image blocks are extracted by using the attention mechanism of the SWTR structural blocks; in the second stage, the transmitted feature layer is downsampled by using a merging block, and feature extraction is realized on the feature layer by using an SWTR structure block to obtain a feature layer C1; repeating the operation of the second stage in the third stage and the fourth stage to obtain feature layers C2 and C3; and transmitting the three different-scale feature layers C1, C2 and C3 extracted by the backbone network to a focusing feature pyramid network.
4. The deep learning-based strip steel surface defect detection method of claim 3, wherein the focused feature pyramid network comprises a CBS convolution module, an adaptive focusing block, an adaptive focusing residual block and upsampling; in a focusing type characteristic pyramid network, three characteristic layers with different scales are utilized for carrying out enhanced characteristic extraction; in the small feature layer of 7×7 pixels, for C 3 Performing continuous two adaptive focusing blocks and up-sampling to obtain a characteristic layer C 2 'A'; for C 2 ' continue to use 1 adaptive focusing block, upsample to obtain feature layer P 1 'A'; for C 1 Channel adjustment using 1 adaptive focusing block and P 1 ' 1-time addition operation to obtain a feature layer P 1 The method comprises the steps of carrying out a first treatment on the surface of the In the middle feature layer of 14×14 pixels, for C 2 Obtaining a feature layer P using consecutive 1 adaptive focusing block and 1 adaptive focusing residual block 2 ' P is to 2 ' and C 2 ' 1-time addition operation to obtain a feature layer P 2 The method comprises the steps of carrying out a first treatment on the surface of the In the case of a large feature layer of 28 x 28 pixels, for an adjusted C 1 Downsampling of the CBS convolution module 1 time and using two consecutive adaptive focusing blocks for it, then for P 2 ' use 1 adaptive focusing residual block, use cascade operation to perform channel adjustment of CBS convolution module on feature layer, use 1 adaptive focusing residual block again and use C with C 3 Adjusted feature layerPerforming addition operation to obtain a feature layer P 3 。
5. The deep learning-based strip steel surface defect detection method of claim 4, wherein the characteristic layer P is detected by using 3 yoxhead modules at the detection head part 1 、P 2 And P 3 Dividing to realize decoupling prediction, wherein one side of decoupling uses a CBS convolution module with a 1-to-3 x 3 convolution kernel size and adjusts channels into the number of defect types to predict classification results; the other side uses a base convolution module of 3 x 3 convolution kernel size and adapts the channels to predict the four points of the anchor frame and determine the existence of the target, respectively.
6. The method for detecting the surface defects of the strip steel based on the deep learning according to claim 1, wherein a Loss function Loss used in training of the surface defect detection model of the strip steel is:
Loss＝λLoss reg +Loss cls +Loss obj
wherein, loss reg Representing regression loss, calculated using IoU loss; loss (Low Density) cls Representing Loss of classification, loss obj Representing target losses, all calculated using BCEWithLogits losses; lambda is the weight and Adam is used by the optimizer.
7. The utility model provides a belted steel surface defect detection device based on degree of depth study which characterized in that includes:
the original image acquisition module is used for acquiring original images of the surface of the strip steel on a strip steel production line;
the defect data set integration module is used for marking the defect types of the collected original images and integrating the original images and marking data into a strip steel surface defect data set;
the defect data set dividing module is used for dividing the strip steel surface defect data set in a certain proportion to obtain a training set, a verification set and a test set;
the model construction module is used for constructing a strip steel surface defect detection model, and the model comprises a main network, a focusing characteristic pyramid network and a detection head;
the model training module is used for training the network by using the strip steel surface defect data set to obtain a trained model, and storing a weight file in the training process;
and the strip steel surface detection module is used for detecting the strip steel surface defect image by using the trained model to obtain the type and position information of the defect in the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310678776.8A CN116721078A (en) | 2023-06-09 | 2023-06-09 | Strip steel surface defect detection method and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310678776.8A CN116721078A (en) | 2023-06-09 | 2023-06-09 | Strip steel surface defect detection method and device based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116721078A true CN116721078A (en) | 2023-09-08 |
Family
ID=87865525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310678776.8A Pending CN116721078A (en) | 2023-06-09 | 2023-06-09 | Strip steel surface defect detection method and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116721078A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117333491A (en) * | 2023-12-01 | 2024-01-02 | 北京航空航天大学杭州创新研究院 | Steel surface defect detection method and system |
-
2023
- 2023-06-09 CN CN202310678776.8A patent/CN116721078A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117333491A (en) * | 2023-12-01 | 2024-01-02 | 北京航空航天大学杭州创新研究院 | Steel surface defect detection method and system |
| CN117333491B (en) * | 2023-12-01 | 2024-03-15 | 北京航空航天大学杭州创新研究院 | Steel surface defect detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112465790A (en) | Surface defect detection method based on multi-scale convolution and trilinear global attention | |
| CN110009622B (en) | Display panel appearance defect detection network and defect detection method thereof | |
| CN113160123B (en) | Leather defect detection method, system and device based on YOLOv5 | |
| CN114120093B (en) | Coal gangue target detection method based on improved YOLOv algorithm | |
| Enshaei et al. | Automated detection of textured-surface defects using UNet-based semantic segmentation network | |
| CN116721078A (en) | Strip steel surface defect detection method and device based on deep learning | |
| CN111127454A (en) | Method and system for generating industrial defect sample based on deep learning | |
| CN110599458A (en) | Underground pipe network detection and evaluation cloud system based on convolutional neural network | |
| CN114155474A (en) | Damage identification technology based on video semantic segmentation algorithm | |
| CN113506239A (en) | Strip steel surface defect detection method based on cross-stage local network | |
| CN111754507A (en) | Light-weight industrial defect image classification method based on strong attention machine mechanism | |
| CN114092467A (en) | Scratch detection method and system based on lightweight convolutional neural network | |
| CN113901924A (en) | Document table detection method and device | |
| CN111612803B (en) | Vehicle image semantic segmentation method based on image definition | |
| Wang et al. | MeDERT: A metal surface defect detection model | |
| CN117475150A (en) | Efficient semantic segmentation method based on SAC-UNet | |
| Zhang et al. | An industrial interference-resistant gear defect detection method through improved YOLOv5 network using attention mechanism and feature fusion | |
| CN116092179A (en) | Improved Yolox fall detection system | |
| CN115601357A (en) | Stamping part surface defect detection method based on small sample | |
| CN114494236A (en) | Fabric defect detection method and system based on over-complete convolutional neural network | |
| CN112016403B (en) | Video abnormal event detection method | |
| CN114821224A (en) | Method and system for amplifying railway image style conversion data | |
| Runhu et al. | Semantic segmentation using DeepLabv3+ model for fabric defect detection | |
| CN112487911A (en) | Real-time pedestrian detection method and device based on improved yolov3 in intelligent monitoring environment | |
| CN112802026A (en) | Deep learning-based real-time traffic scene semantic segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |