CN113628178A - Method for detecting surface defects of steel products with balanced speed and precision - Google Patents
Method for detecting surface defects of steel products with balanced speed and precision Download PDFInfo
- Publication number
- CN113628178A CN113628178A CN202110872859.1A CN202110872859A CN113628178A CN 113628178 A CN113628178 A CN 113628178A CN 202110872859 A CN202110872859 A CN 202110872859A CN 113628178 A CN113628178 A CN 113628178A
- Authority
- CN
- China
- Prior art keywords
- module
- convolution
- target detection
- detection model
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 107
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 57
- 239000010959 steel Substances 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000001514 detection method Methods 0.000 claims abstract description 115
- 238000012360 testing method Methods 0.000 claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 31
- 230000004927 fusion Effects 0.000 claims abstract description 28
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000005070 sampling Methods 0.000 claims description 22
- 238000012545 processing Methods 0.000 claims description 5
- 238000011176 pooling Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 3
- 238000013519 translation Methods 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims 1
- 238000003199 nucleic acid amplification method Methods 0.000 claims 1
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 238000005266 casting Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000011160 research Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000009776 industrial production Methods 0.000 description 3
- 230000002950 deficient Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000013100 final test Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 238000003466 welding Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 238000005098 hot rolling Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention discloses a method for detecting surface defects of steel products with balanced speed and precision, which belongs to the technical field of steel surface defect detection and comprises the following steps: s1: obtaining a typical image sample of the surface defect of the hot-rolled strip steel from a database, and preprocessing the sample; s2: designing a hot-rolled strip steel surface defect target detection model comprising a jump layer connection and pyramid feature fusion module based on a CenterNet target detection model, and selecting a backbone network as a feature extractor of the target detection model; s3: initializing parameters of a backbone network structure part in the target detection model by using parameters trained on an ImageNet data set, and training the target detection model by using a training sample; s4: and testing the test sample by using the trained target detection model, and outputting a detection result. The method has high accuracy and high detection speed for detecting the target position of the surface defect of the hot-rolled strip steel, and can be effectively applied to the field real-time detection of the surface defect of the hot-rolled strip steel.
Description
Technical Field
The invention relates to the technical field of steel surface defect detection, in particular to a method for detecting surface defects of steel products with balanced speed and precision.
Background
In the production process of hot-rolled strip steel, various defects such as pressed oxide skin, scratches, pitted surface, impurities, patches, cracks and the like are easy to appear on the surface of the strip steel due to various physical and chemical factors and the complexity of a hot rolling process, and the performance and the attractiveness of a product are seriously influenced. The creation and flow of defective products to the market place can result in significant economic and commercial reputation losses for product manufacturers. Therefore, the real-time and correct detection and research on the surface defects of the hot-rolled strip steel are very important for the production and quality control of the strip steel.
At present, the inspection of the surface defects of steel is mainly divided into manual observation and automatic defect detection systems. The detection method of manual observation has high dependence on detection personnel, can consume a large amount of manpower, has strong subjectivity and is difficult to scientifically and accurately identify the surface defects of the products. The wide application of automatic defect detection technologies such as infrared, magnetic flux leakage and machine vision in actual industrial production gradually solves the problems brought by artificial defect detection methods. Especially, the accuracy and the speed of the defect classification technology are greatly improved due to the development of computer vision. However, these methods cannot obtain accurate defect location information in a defect inspection task. In recent years, with the development of deep learning, the convolutional neural network has unique advantages in feature extraction and achieves better performance in the recognition task. Therefore, applying the deep learning method to the industrial target detection is a research focus of product detection at present.
The Chinese patent with the application number of CN201611136821.3 discloses an accurate positioning method for transmitting surface defects of a casting blank to a rolled material. The patent of the invention mainly researches the corresponding relation between time and space generated by the surface defect of the large special steel core product, deduces the established procedure generated by the surface defect, and provides an important reference basis for improving the quality of the core product and the production efficiency of enterprises. According to the method for quickly and accurately positioning the surface defect of the casting blank corresponding to the defect of the rolled material, holes are formed in the two ends of the surface defect of the casting blank according to the required hole diameter and depth at a certain distance along the rolling direction, welding rods are used for welding and filling the holes smoothly, the defect on the rolled material corresponding to the holes can be quickly and accurately found according to the corresponding relation after rolling, the surface defect of the casting blank is quickly positioned according to the hole diameter and the depth, sampling detection analysis is carried out, the quick and accurate positioning of the surface defect of the casting blank corresponding to the defect of the rolled material is realized, and the timeliness and the accuracy of judging the defect generation reason and the process improvement can be effectively improved.
Currently in deep learning methods, it has been demonstrated that different levels of feature maps in convolutional neural networks contain different image information. Specifically, the shallow feature map can fully represent the position information of the target, and the high-level features contain rich target semantic information. An 10.1109/TIM.2019.2915404 scientific research paper with DOI discloses an end-to-end algorithm for detecting surface defects of hot-rolled strip steel by fusing multilayer features. The paper mainly researches on improving the detection capability of a detection model on the surface defects of the hot-rolled strip steel by fusing a plurality of grades of feature maps. The method firstly detects that the classification performance of the model to the defects is not obviously reduced by introducing the feature fusion method, and then the network structure is applied to the defect detection of the hot-rolled strip steel. The method mainly focuses on scaling the feature map size to a uniform size by using different down-sampling or up-sampling strategies for features of different levels, and then fusing the feature map size and the uniform size. The experimental result proves that the method can improve the defect detection capability of the detection model, but a special solution is not designed aiming at the surface defect images of the hot-rolled strip steel with different characteristics, and almost all the hierarchical characteristic graphs are integrated uniformly. In addition, the selected network backbone has more training parameters, and the detection speed requirement in the actual industrial production environment is not considered. Therefore, the method for detecting the surface defects of the steel products with balanced speed and precision is provided.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: how to detect the surface defects of the steel products with high precision and speed meeting the requirements of a real-time production line provides a method for detecting the surface defects of the steel products with balanced speed and precision.
The invention solves the technical problems through the following technical scheme, and the invention comprises the following steps:
s1: obtaining a typical image sample of the surface defect of the hot-rolled strip steel from an NEU-DET database, and preprocessing the typical image sample;
s2: designing a hot-rolled strip steel surface defect target detection model comprising a jump layer connection and pyramid feature fusion module based on a CenterNet target detection model, and selecting a backbone network with a small parameter number as a feature extractor of the model;
s3: initializing parameters of a backbone network structure part in the model by using parameters trained on an ImageNet data set, and training the model by using a training sample;
s4: and testing the test sample by using the trained model, and outputting a detection result.
Still further, the step S1 includes the steps of:
s1.1: dividing the obtained samples into a training set and a testing set according to a ratio of 7:3, wherein the training set and the testing set respectively comprise 6 types, and expanding the samples in the training set into pictures of 384 by 384;
s1.2: and carrying out a series of data enhancement processing such as turning, translation, brightness increase, cutting and the like on the training sample.
Further, the step S2 is to design a model for detecting the target of the surface defect of the hot-rolled strip steel, which includes a skip layer connection and pyramid feature fusion module, based on the centret target detection model, and select the backbone network with a small number of parameters as the feature extractor of the model, and the step S is to:
s2.1: based on the centrnet target detection model, a backbone network ResNet18-dcn with a small number of parameters and portability is selected as a feature extractor.
S2.2: in order to improve the detection capability of the detection model on the defect image with small difference between the target and the background, the model for detecting the surface defect target of the hot-rolled strip steel comprising the jump layer connection module is designed;
s2.3: in order to improve the detection capability of the detection model on the defect images with the characteristics of large intra-class difference and high inter-class similarity, a pyramid feature fusion module is added into the model added with the layer jump connection module to obtain a final detection model.
Further, the step S3 initializes the parameters of the backbone network structure part in the model with the parameters trained on the ImageNet data set, and the step of training the model with the training samples includes:
s3.1: in an improved model, initial parameters of a ResNet-18 backbone network structure in ResNet18-dcn adopt parameters trained on an ImageNet data set, an upsampling layer in a decoding network structure and pyramid feature fusion module in ResNet18-dcn adopts a linear interpolation mode to initialize parameters, and a deformation convolution layer adopts a xavier Gauss initialization mode to initialize parameters;
s3.2: and inputting the training set into an improved target detection model, and learning and updating parameters by adopting a BP algorithm.
Further, the step S4 of testing the test sample by using the trained model, and outputting the test result includes:
s4.1: amplifying the test sample by 1.6 times and turning over the test sample;
s4.2: the final test result is the average of the test results of the enhanced sample and the original sample.
Furthermore, the centret model in step S2 adopts three backbone network structures, namely ResNet-101, DLA-34 and Hourglass-104, as feature extractors in the target detection process. When ResNet is used as a backbone network, a coding and decoding mode is adopted to extract target features, wherein a ResNet network structure is called a coding network, three deformation convolution layers and an upsampling layer which are alternately connected are introduced behind an encoder and are used as a decoding network, the deformation convolution layers adopt convolution kernel sizes of 3 x 3 and step lengths of 1, the upsampling layer adopts convolution kernel sizes of 4 x 4, an initialization mode is linear interpolation, the step lengths of 2, and channels of each feature map after upsampling are 256, 128 and 64.
Further, the step S2.1 of designing the layer jump connection module includes:
s2.2.1: based on a ResNet-18 coding network and a decoding network consisting of a deformation convolution layer and an up-sampling layer, combining two characteristic graphs with the same channel number and size in the two networks, namely adding pixel values on the same position of the two characteristic graphs;
s2.1.2: inputting the combined characteristic diagram into a deformation convolution layer in a decoding network;
further, the step S2.3 of designing the pyramid feature fusion module includes:
s2.3.1: each feature graph passing through an up-sampling layer in the decoding network is up-sampled again, the sizes of convolution kernels of the up-sampling layer again are respectively 16 × 16, 8 × 8 and 4 × 4, the step lengths are respectively 8, 4 and 2, and the parameter initialization mode is linear interpolation;
s2.3.2: and combining the four feature maps after the up-sampling, wherein the combination mode is that the pixel values at the same position are added.
Compared with the prior art, the invention has the following advantages:
(1) based on a first-order detection CenterNet model with a higher detection speed, a lighter and modularized ResNet-18 network is adopted as a backbone structure, and meanwhile, a deformation convolution layer is added before an upper sampling layer in a decoding network to adapt to industrial data with various defect forms, so that the robustness of the model is enhanced; the design of the residual error structure in ResNet-18 can simplify the learning process of the network, accelerate the gradient propagation of the network and avoid the network degradation.
(2) On the basis of a portable feature extraction network structure, two modules of skip layer connection and pyramid feature fusion are designed; the design of the layer jump connection module realizes the deep supervision of the whole network structure on the basis of not increasing network parameters, and simultaneously fuses deep and shallow features of the network to realize the integration of local and global features, thereby improving the detection capability of the model on low-quality defect samples; the pyramid feature fusion module is designed to fuse four feature maps with rich semantic information into an output feature map with a larger size, so that the detection capability of the model on defect samples with the characteristics of large intra-class difference and high inter-class similarity is improved.
(3) Compared with the existing network model, the method has the advantages that the detection accuracy is improved, and the requirement for the lowest speed of detecting the defects of the sample in the actual production process is met.
Drawings
FIG. 1 is a flow chart of a method for detecting surface defects of steel products with balanced speed and precision according to an embodiment of the present invention;
FIG. 2 is a sample of 6 typical defect images in the NEU database in an embodiment of the present invention;
FIG. 3 is a diagram of a network architecture in an embodiment of the present invention;
FIG. 4 is a diagram illustrating the effect of detecting a defect picture according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of different layer jump connection modules.
Detailed Description
The following examples are given for the detailed implementation and specific operation of the present invention, but the scope of the present invention is not limited to the following examples.
The embodiment provides a technical scheme: a method for detecting surface defects of steel products with balanced speed and precision takes the identification and positioning of the surface defects of hot rolled strip as an example, as shown in figure 1, and comprises the following steps:
s1: and acquiring a typical image sample of the surface defect of the hot-rolled strip steel from the NEU-DET database, and preprocessing the sample.
The NEU database refers to a surface defect database (northeast University) In which images of six typical surface defects on the surface of a hot-rolled strip and information of position coordinates of the defects In each image are obtained, as shown In fig. 2, wherein the data images include 1800 typical hot-rolled strip surface defect images including cracks (Cr), scale inclusions (Rs), pits (Ps), patches (Pa), inclusions (In), and scratches (Sc).
The method comprises the following two steps:
s1.1: dividing the obtained samples into a training set and a testing set according to a ratio of 7:3, wherein the training set and the testing set respectively comprise 6 types, and enlarging the size of the samples in the training set into pictures of 384 by 384;
s1.2: and carrying out a series of data enhancement processing such as turning, translation, brightness increase, cutting and the like on the training sample.
S2: based on a CenterNet target detection model, a small-parameter ResNet18-dcn network structure is used as a feature extractor of the model, and the model for detecting the surface defect target of the hot-rolled strip steel, which comprises a jump layer connection and pyramid feature fusion module, is built.
The ResNet18-dcn model is a feature extractor combining an encoder and a decoder, and adopts a traditional ResNet-18 model as a backbone network (also called as an encoder in ResNet 18-dcn) and adopts 3 stacked deformation convolutional layers and upsampling layers as a decoding network structure of ResNet 18-dcn. ResNet-18 includes 17 convolutional layers, 1 max pooling layer. The structure of the network for detecting the defective target is shown in FIG. 3. Referring to fig. 3, the target detection network of the present invention is specifically described, first, an original defect image is input into a convolutional layer and a maximum pooling layer (named conv1) connected in sequence in an encoder, the size of an output feature map is 1/4 (the fractional value refers to the ratio of the size of the output image to the size of the original input image, which will be explained below), and then 4 identical convolution modules (conv2, conv3, conv4 and conv5) are connected in sequence, each module includes two residual error structures with two convolutional layers, the step size of the convolutional layer is 1 in conv2, the step size of the first convolutional layer in the first convolutional layer structure is 2 in conv3, conv4 and conv5, the step size of the remaining convolutional layer is 1, and the size of the output feature map of each convolution module is 1/4, 1/8, 2, and the maximum pooling layer size of the convolution module, 1/16, 1/32; then, three same up-sampling modules up-conv1, up-conv2 and up-conv3 in the decoding network are connected, each up-sampling module comprises one deformation convolution layer and one up-sampling layer, the deformation convolution layer adopts convolution kernel with the size of 3 x 3 and the step length of 1, the up-sampling layer adopts convolution kernel with the size of 4 x 4, the initialization mode is linear interpolation and the step length of 2, the channel of each feature map after up-sampling is 256, 128 and 64, and the sizes of the output feature maps of the three up-sampling modules are 1/16, 1/8 and 1/4. The cross-layer connection module (SCM) represents a connection between feature maps of the same size as the output feature map in the encoding network and the decoding network, and the actual combination method is the addition of pixel values between feature maps, and the output size of the fused feature map after the addition is not changed. The pyramid feature fusion module (PFM) is characterized in that four feature graphs with different sizes are firstly adjusted to be uniform in size through an upper sampling layer, and then are combined in an adding mode to be fused into a feature graph with a larger size; then, three output modules consisting of two convolutional layers are connected, wherein the first module (cls) is used for defect classification, the second module (loc _ offset) is used for predicting the width and height of the defect target detection frame, and the third module (loc _ wh) is used for predicting the offset amount of the center point coordinate of the defect target in the x and y directions.
S3: and initializing parameters of a backbone network structure part in the model by adopting the parameters trained on the ImageNet data set, and training the model by using the training sample.
The process of training the target detection model includes the steps of:
s3.1: in an improved model, the initial parameters of the ResNet-18 backbone network structure in ResNet18-dcn adopt the parameters trained on the ImageNet data set, the parameters are initialized by the decoding network structure in ResNet18-dcn and an upsampling layer in a pyramid feature fusion module in a linear interpolation mode, and the parameters are initialized by the deformation convolution layer in a xavier Gauss initialization mode;
s3.2: and inputting the training set into an improved target detection model for parameter learning and updating.
The improved target detection model is trained by adopting a BP algorithm, the improved detection network is called as a steel product surface defect detector because the invention mainly aims at the detection of a steel product surface defect target, network parameters are updated according to the network output and the errors of sample defect types and defect positions, 24 pictures are adopted each time to calculate the network errors for one batch and update the weight, an Adam optimization algorithm is adopted during the training, the initial learning rate is set to be 1.25e-4, when the training is carried out to 60 and 120 rounds, the learning rate is sequentially reduced by 10 times, and the training is finished when the iteration is carried out to 160.
S4: and testing the test sample by using the trained model, and outputting a detection result.
The process of testing the target detection model includes the steps of:
s4.1: amplifying the test sample by 1.6 times and turning over the test sample;
s4.2: the final test result is the average of the test results of the enhanced sample and the original sample.
The method is adopted to detect the defects of a group of steel product surface samples, and the steel product surface defects are identified by adopting several detection methods which are latest in detection speed or precision in the existing first-order and second-order target detection technologies, and the detection precision and the detection speed are compared. The specific first-order detection models are the fast RCNN, the cascade RCNN and the DDN models, and the second-order detection models are the M2Det, the SSD, the FCOS, the ATSS, the YOLOv3 and the CenterNet models. In order to obtain a more competitive detection result, the fast RCNN, DDN, M2Det and CneterNet detection models are trained by adopting a plurality of characteristic backbone networks. Through training, the detection results of each model for different types of defects are obtained, and are shown in table 1. The effect of the detection of a part of the image therein is shown in fig. 4.
TABLE 1 evaluation of surface Defect Properties of Steel products based on different models
As shown in Table 1, the second-order detector has a high average precision average value for detecting the surface defects of the steel products, and the detection precisions of the three models are 77.9%, 73.3% and 82.3%, respectively, but the detection speeds of the three models for each sample are very slow. For example, when the DDN detector selects ResNet50 as the feature extraction network, its detection speed is only 11 FPS. In contrast, the first-order detector can achieve a faster detection speed, but the detection precision difference is large, especially the detection result of the M2Det model. Compared with other detection models, the detector for the surface defects of the steel products (Our work) has high average precision and high speed. Compared with a CenterNet model taking resnet18-dcn as a backbone, the layer jump connection and pyramid feature fusion module designed by the invention improves the average precision mean value by 6.1% under the condition of not obviously reducing the detection speed. In general, the defect target detection model in the invention realizes the detection precision of 80.0% and the detection speed of 64FPS, and realizes the optimal speed and precision balance.
Design and discussion of model structures
Jumping layer connection module
Shallow features in the convolutional neural network may provide more target location information. In the above section we have briefly described that a layer jump connection module (as shown in fig. 5 a) can improve detection performance by fusing features with the same resolution. However, if other combinations of these layers would yield better performance? For this reason, the present embodiment designs two other combination patterns to integrate different size feature maps based on ResNet18-dcn, as shown in FIG. 5b and FIG. 5 c.
The different combinations are applied to the surface defect detection of the hot-rolled strip steel, and compared with a network taking ResNet18-dcn as a main body, the detection performance is improved. Specifically, the low-layer fusion mode SCM _ L combining the large feature mAP in the coding network and the small feature mAP in the decoding network obtains the detection speed of the mAP and 61FPS of 77.2%. The high-level feature fusion mode SCM _ H combining the small feature mAP in the coding network and the large feature mAP in the decoding network obtains the optimal mAP of 80.2%. However, the increased deconvolution operation in this manner reduces the detection speed of the detection model to 59FPS, which is slower than the same level feature map fused manner SCM _ S. For comparison, the mAP of the SCM _ S combined with the same level is 80.0%, the speed is 64FPS, and the minimum speed requirements of different industrial production scenes can be met.
More specifically, the detection performances of the detection models with different combinations on different defect types are shown in table 3, and it can be seen that the detection performances of the three combination modes on scratches, patches, pressed scale and inclusions are relatively stable, the average detection accuracy rates on crack defects are 53.7%, 58.6% and 45.0%, respectively, and the detection accuracy rates on pit surfaces are 87%, 84.5% and 80.1%, respectively, which indicates that the performance influences of different feature fusion modes on the two defects are obviously different. The potential reason is that cracks and pitted surface defects are very similar to the background, and the SCM-L combination method adopts a maximum pool method at a low layer to fuse features of different layers, so that more target position information is lost, and the detection result is poor. In contrast, the SCM _ H utilizes a deconvolution technology to enlarge the size of the shallow feature and further extract more semantic features, so that a better detection effect is obtained. However, convolution operations also reduce the speed of model detection. Finally, SCM _ S achieves the best performance trade-off.
TABLE 3 model test Performance in different combinations
Pyramid feature fusion module
Experiments prove that the pyramid feature fusion module can improve the detection performance by combining four depth features. In order to search which level of feature fusion can achieve the best precision without reducing the resolution of the output feature map, the embodiment compares the detection performances of different levels of fusion features in the encoding network, such as the conv5 module, the upsampling conv1, the up-conv2 and the up-conv3 module. As previously described, we also used ResNet18-dcn as the baseline. The results of the different integration modes are shown in table 4. By introducing SCM, the maps of the model were 77.6%. Along with the increase of the number of the introduced characteristic graphs, the detection precision is improved to different degrees, and the detection speed is not obviously reduced. The detection accuracy of the two-layer or three-layer feature map fusion is 77.8%, 78.6% and 78.5%, respectively, and the speed only deviates by 1 FPS. The method has the advantages that the characteristic maps of different levels can provide unique defect information, and the fusion of more characteristic information is an effective way for improving the detection precision of the steel surface defects.
TABLE 4 detection Performance of the model by combining the feature maps of different levels
| conv5 | up-conv1 | up-conv2 | up-conv3 | mAP(％) | FPS |
| √ | 77.6 | 70 | |||
| √ | √ | 77.8 | 69 | ||
| √ | √ | √ | 78.6 | 68 | |
| √ | √ | √ | 78.5 | 64 | |
| √ | √ | √ | √ | 80.0 | 64 |
In summary, the method for detecting the surface defects of the steel products with balanced speed and precision has the advantages of high accuracy and high detection speed for detecting the target position of the surface defects of the hot-rolled strip steel, can be effectively applied to the field real-time detection of the surface defects of the hot-rolled strip steel, and is worthy of popularization and application.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A method for detecting surface defects of steel products with balanced speed and precision is characterized by comprising the following steps:
s1: obtaining a typical image sample of the surface defect of the hot-rolled strip steel from a database, and preprocessing the sample;
s2: designing a hot-rolled strip steel surface defect target detection model comprising a jump layer connection and pyramid feature fusion module based on a CenterNet target detection model, and selecting a backbone network as a feature extractor of the target detection model;
s3: initializing parameters of a backbone network structure part in the target detection model by using parameters trained on an ImageNet data set, and training the target detection model by using a training sample;
s4: and testing the test sample by using the trained target detection model, and outputting a detection result.
2. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 1, wherein the method comprises the following steps: in step S1, the database is an NEU surface defect database, and images of six typical surface defects on the surface of the hot-rolled steel strip and position coordinate information of the defect in each image are obtained in the database, where the six typical surface defects are cracks, pressed scale, pitted surface, patches, inclusions, and scratches, respectively.
3. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 1, wherein the method comprises the following steps: the step S1 includes the steps of:
s11: dividing the obtained samples into a training set and a testing set according to a set proportion, wherein the training set and the testing set respectively comprise 6 typical surface defect categories, and expanding the samples in the training set into pictures of 384X 384;
s12: and carrying out data enhancement processing on the training samples, wherein the data enhancement processing mode comprises turning, translation, brightness increase, cutting and amplification.
4. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 1, wherein the method comprises the following steps: the step S2 includes the steps of:
s21: selecting a backbone network ResNet18-dcn as a feature extractor based on a CenterNet target detection model;
s22: designing a hot-rolled strip steel surface defect target detection model comprising a jump layer connection module;
s23: and adding a pyramid feature fusion module in the target detection model added with the layer jump connection module to obtain a final target detection model.
5. The method of claim 4 for detecting surface defects of steel products with balanced speed and precision, wherein the method comprises the following steps: in step S23, the final target detection model includes a coding network, a decoding network, a layer jump connection module, a pyramid feature fusion module, and an output module; the coding network comprises a first convolution module, a second convolution module, a third convolution module, a fourth convolution module and a fifth convolution module which are connected in sequence, the first convolution module comprises a convolution layer and a pooling layer which are connected in sequence, the second convolution module, the third convolution module, the fourth convolution module and the fifth convolution module are identical in structure and respectively comprise two residual error structures which are connected in sequence and are provided with two convolution layers, the decoding network comprises a first up-sampling module, a second up-sampling module and a third up-sampling module which are connected in sequence, the first up-sampling module, the second up-sampling module and the third up-sampling module are identical in structure and respectively comprise a deformation convolution layer and an up-sampling layer which are connected in sequence; the layer jump connection module is used for realizing the connection between the feature graphs with the same size as the output feature graph in the coding network and the decoding network, and the connection combination mode is the addition of pixel values between the feature graphs, and the output size of the added fusion feature graph is not changed; the pyramid feature fusion module is used for adjusting feature maps with different sizes into a uniform size through an upsampling layer, and then combining the feature maps in an adding mode to fuse the feature maps into a feature map with a larger size.
6. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 5, wherein the method comprises the following steps: in the coding network, the size of the feature map output by the first convolution module is 1/4 of the size of the original image, the step size of the convolution layer in the second convolution module is 1, the step size of the first convolution layer in the first convolution residual structure in the third convolution module, the fourth convolution module and the fifth convolution module is 2, the step sizes of the rest convolution layers are 1, and except for the first convolution module, the size of the feature map output by each convolution module is 1/4, 1/8, 1/16 and 1/32 in sequence.
7. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 5, wherein the method comprises the following steps: in the decoding network, the size of a convolution kernel adopted by the deformation convolution layer is 3 × 3, the step size is 1, the size of a convolution kernel adopted by the upsampling layer is 4 × 4, the step size is 2, and the channel of each feature map after upsampling is 256, 128 and 64.
8. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 5, wherein the method comprises the following steps: in step S23, the final target detection model further includes a first output module, a second output module, and a third output module connected to the pyramid feature fusion module, where the first output module is used for defect classification, the second output module is used for predicting width and height of a defect target detection frame, and the third output module is used for predicting offsets of the coordinates of the center point of the defect target in the x and y directions.
9. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 5, wherein the method comprises the following steps: the step S3 includes the steps of:
s31: the initial parameters of the ResNet-18 backbone network structure in ResNet18-dcn adopt the parameters trained on the ImageNet data set, the backbone network structure is a coding network structure, the parameters are initialized by adopting a linear interpolation mode in the ResNet18-dcn decoding network structure and an upsampling layer in a pyramid feature fusion module, and the parameters are initialized by adopting a xavier Gaussian initialization mode in the deformation convolution layer;
s3.2: and inputting the training set into an improved target detection model, and learning and updating parameters by adopting a BP algorithm.
10. The method for detecting surface defects of steel products with balanced speed and precision as claimed in claim 1, wherein the method comprises the following steps: the step S4 includes the steps of:
s41: amplifying and turning over two kinds of data enhancement processing are carried out on the test sample to obtain an enhanced test sample, and the enhanced test sample and the original test sample are input into a trained target detection model;
s42: and respectively obtaining the detection results of the enhanced test sample and the original test sample, and taking the average value of the detection results of the enhanced test sample and the original test sample as the final detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110872859.1A CN113628178B (en) | 2021-07-30 | 2021-07-30 | Steel product surface defect detection method with balanced speed and precision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110872859.1A CN113628178B (en) | 2021-07-30 | 2021-07-30 | Steel product surface defect detection method with balanced speed and precision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113628178A true CN113628178A (en) | 2021-11-09 |
| CN113628178B CN113628178B (en) | 2024-03-15 |
Family
ID=78381876
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110872859.1A Active CN113628178B (en) | 2021-07-30 | 2021-07-30 | Steel product surface defect detection method with balanced speed and precision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113628178B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907560A (en) * | 2021-03-16 | 2021-06-04 | 中科海拓(无锡)科技有限公司 | Notebook appearance flaw segmentation method based on deep learning |
| CN114998695A (en) * | 2022-07-18 | 2022-09-02 | 深圳市前海泽金产融科技有限公司 | Method and system for improving image recognition speed |
| CN115496752A (en) * | 2022-11-16 | 2022-12-20 | 齐鲁工业大学 | Steel surface defect detection method based on one-stage target detection algorithm |
| CN116051541A (en) * | 2023-03-06 | 2023-05-02 | 杭州深度视觉科技有限公司 | Bearing end face gentle abrasion detection method and device based on stroboscopic light source |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154504A (en) * | 2017-12-25 | 2018-06-12 | 浙江工业大学 | A kind of detection method of the Surface Defects in Steel Plate based on convolutional neural networks |
| CN110363253A (en) * | 2019-07-25 | 2019-10-22 | 安徽工业大学 | A kind of Surfaces of Hot Rolled Strip defect classification method based on convolutional neural networks |
| CN111161260A (en) * | 2020-01-02 | 2020-05-15 | 中冶赛迪重庆信息技术有限公司 | Hot-rolled strip steel surface defect detection method and device based on deep learning |
| CN112381787A (en) * | 2020-11-12 | 2021-02-19 | 福州大学 | Steel plate surface defect classification method based on transfer learning |
| CN113160141A (en) * | 2021-03-24 | 2021-07-23 | 华南理工大学 | Steel sheet surface defect detecting system |
-
2021
- 2021-07-30 CN CN202110872859.1A patent/CN113628178B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108154504A (en) * | 2017-12-25 | 2018-06-12 | 浙江工业大学 | A kind of detection method of the Surface Defects in Steel Plate based on convolutional neural networks |
| CN110363253A (en) * | 2019-07-25 | 2019-10-22 | 安徽工业大学 | A kind of Surfaces of Hot Rolled Strip defect classification method based on convolutional neural networks |
| CN111161260A (en) * | 2020-01-02 | 2020-05-15 | 中冶赛迪重庆信息技术有限公司 | Hot-rolled strip steel surface defect detection method and device based on deep learning |
| CN112381787A (en) * | 2020-11-12 | 2021-02-19 | 福州大学 | Steel plate surface defect classification method based on transfer learning |
| CN113160141A (en) * | 2021-03-24 | 2021-07-23 | 华南理工大学 | Steel sheet surface defect detecting system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907560A (en) * | 2021-03-16 | 2021-06-04 | 中科海拓(无锡)科技有限公司 | Notebook appearance flaw segmentation method based on deep learning |
| CN114998695A (en) * | 2022-07-18 | 2022-09-02 | 深圳市前海泽金产融科技有限公司 | Method and system for improving image recognition speed |
| CN114998695B (en) * | 2022-07-18 | 2022-11-15 | 深圳市前海泽金产融科技有限公司 | Method and system for improving image recognition speed |
| CN115496752A (en) * | 2022-11-16 | 2022-12-20 | 齐鲁工业大学 | Steel surface defect detection method based on one-stage target detection algorithm |
| CN116051541A (en) * | 2023-03-06 | 2023-05-02 | 杭州深度视觉科技有限公司 | Bearing end face gentle abrasion detection method and device based on stroboscopic light source |
| CN116051541B (en) * | 2023-03-06 | 2023-07-18 | 杭州深度视觉科技有限公司 | Bearing end face gentle abrasion detection method and device based on stroboscopic light source |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113628178B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113628178B (en) | Steel product surface defect detection method with balanced speed and precision | |
| CN110533084B (en) | Multi-scale target detection method based on self-attention mechanism | |
| CN107909564B (en) | Full convolution network image crack detection method based on deep learning | |
| CN111768388B (en) | Product surface defect detection method and system based on positive sample reference | |
| CN112037219B (en) | Metal surface defect detection method based on two-stage convolutional neural network | |
| CN113807355B (en) | Image semantic segmentation method based on coding and decoding structure | |
| CN112232214A (en) | Real-time target detection method based on depth feature fusion and attention mechanism | |
| CN113643268B (en) | Industrial product defect quality inspection method and device based on deep learning and storage medium | |
| CN113239930A (en) | Method, system and device for identifying defects of cellophane and storage medium | |
| CN116883393B (en) | Metal surface defect detection method based on anchor frame-free target detection algorithm | |
| CN114742799B (en) | Industrial scene unknown type defect segmentation method based on self-supervision heterogeneous network | |
| CN111833310B (en) | Surface defect classification method based on neural network architecture search | |
| CN113012153A (en) | Aluminum profile flaw detection method | |
| CN114627383A (en) | Small sample defect detection method based on metric learning | |
| CN112991271A (en) | Aluminum profile surface defect visual detection method based on improved yolov3 | |
| CN112819748A (en) | Training method and device for strip steel surface defect recognition model | |
| Qian et al. | LFF-YOLO: A YOLO algorithm with lightweight feature fusion network for multi-scale defect detection | |
| CN114897802A (en) | Metal surface defect detection method based on improved fast RCNN algorithm | |
| CN116012310A (en) | Cross-sea bridge pier surface crack detection method based on linear residual error attention | |
| CN113763364B (en) | Image defect detection method based on convolutional neural network | |
| CN115908793A (en) | Coding and decoding structure semantic segmentation model based on position attention mechanism | |
| CN116342542A (en) | Lightweight neural network-based steel product surface defect detection method | |
| CN114332047A (en) | Construction method and application of surface defect detection model | |
| Chen et al. | Research on anti-interference detection of 3D-printed ceramics surface defects based on deep learning | |
| Dahai et al. | A lightweight convolutional neural network for recognition and classification for Si3N4 chip substrate surface defects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |