CN111739001A - Product surface defect detection model and detection method based on deformable convolution - Google Patents

Product surface defect detection model and detection method based on deformable convolution Download PDF

Info

Publication number
CN111739001A
CN111739001A CN202010552242.7A CN202010552242A CN111739001A CN 111739001 A CN111739001 A CN 111739001A CN 202010552242 A CN202010552242 A CN 202010552242A CN 111739001 A CN111739001 A CN 111739001A
Authority
CN
China
Prior art keywords
module
layer
defect
detection
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010552242.7A
Other languages
Chinese (zh)
Inventor
张洁
寇恩溥
汪俊亮
杨振良
徐楚桥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Donghua University
National Dong Hwa University
Original Assignee
Donghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Donghua University filed Critical Donghua University
Priority to CN202010552242.7A priority Critical patent/CN111739001A/en
Publication of CN111739001A publication Critical patent/CN111739001A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention designs a product surface defect detection model and a detection method based on deformable convolution by adopting a deep learning technology, firstly designs a model structure and parameters based on a deformable convolution neural network, decomposes mixed defect mode identification into single type defect identification by adopting a one-hot coding mode according to a model output result, carries out network model training by adopting deep learning, labels data of the mixed defect mode by adopting the one-hot coding method, and then detects image data of a product of the mixed defect mode. The method can better meet the detection task, is efficient and accurate, improves the detection precision of the product in the mixed defect mode, reduces the false detection rate and the missed detection rate, has very high application value and economic benefit, and can be better applied to the actual product surface defect detection task as proved by actual verification.

Description

Product surface defect detection model and detection method based on deformable convolution
Technical Field
The invention relates to a product surface defect detection model and a detection method based on deformable convolution, and belongs to the technical field of product defect detection.
Background
The product surface defects not only affect the subsequent processing and assembly of the product, but also affect the product recall rate and the reputation of manufacturing enterprises, and directly determine the survival of the enterprises. Along with the improvement of the automation level of modern manufacturing enterprises, the product yield is improved, and the requirements on the precision and the efficiency of the detection of the surface defects of the products are also improved. The product surface defects are various in types and sizes, and in addition, not only a single type of defects but also a plurality of types of mixed defects exist on the surface of one product, so that the detection and identification difficulty of the product surface defects is increased.
The existing product surface defect identification method mainly comprises a manual detection method, a statistical-based method, a machine learning-based method and a deep learning-based method. The manual detection method is eliminated gradually due to low efficiency and high false detection rate, and the method based on statistics and machine learning cannot be applied due to low identification capability and long calculation time. The current method based on deep learning has low accuracy for detecting the product defects in the mixed defect mode. In addition, the product surface defect mode has multi-angle characteristics, the defect modes at different angles can be mistaken for different defect modes during recognition, the recognition difficulty of the mixed defect mode is increased, and the method does not consider the influence of the multi-angle characteristics on the mixed mode recognition. Therefore, the current product surface defect detection method needs to be broken through urgently, the automation intelligence level of a manufacturing enterprise is improved, and the product surface defect detection method capable of accurately identifying mixed defects is provided for the modern manufacturing enterprise to improve the efficiency and accuracy of product surface defect detection.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: how to accurately and efficiently detect and identify the surface defects of the products on line.
In order to solve the above problems, the technical solution of the present invention is to provide a product surface defect detection model based on deformable convolution, which is characterized in that: the surface defect detection system comprises a first module, a plurality of second modules and a third module, wherein surface defect image data are processed by the first module and then serve as input of the second module, and detection results are output by the third module after being processed by the second module;
the module I comprises a normal convolution layer and a batch-normalization layer, the surface defect image data are processed, the normal convolution layer is used for preliminarily extracting the characteristics of the product surface defect image data, and then the batch-normalization layer is used for normalizing the characteristic data to prevent network overfitting;
the second module comprises a normal convolution layer, a deformable convolution layer and a batch-normalization layer, the second module firstly utilizes the normal convolution layer to extract the characteristics of the input data of the second module, then utilizes the deformable convolution layer to extract the multi-angle defect characteristics in the data, and finally utilizes the batch-normalization layer to normalize the characteristic data to prevent over-fitting of the network;
and the third module comprises two fully-connected layers, the first fully-connected layer is used for processing the input data of the third module, the second fully-connected layer is used as an output layer for processing the data, the output results of all nodes of the output layer are detection results, the activation function of the output layer adopts a sigmoid activation function, the output result adopts a one-hot coding mode, the detection result of each defect corresponds to one output node, and the number of the output nodes is equal to the number of defect types.
Another technical solution of the present invention is to provide a method for detecting surface defects of products based on deformable convolution, which is characterized in that the method for detecting surface defects of products based on deformable convolution is applied, and comprises the following steps:
step 1, establishing a model based on a deformable convolution neural network, wherein the model comprises a module I, a plurality of modules II and a module III;
the module comprises a normal convolution layer and a batch-normalization layer;
the module two comprises a normal convolution layer, a deformable convolution layer and a batch-normalization layer;
the third module comprises two fully-connected layers;
step 2, marking a surface defect image data set for training;
step 3, taking the surface defect image data marked in the step 2 as the input of the model in the step 1;
step 4, processing the input product defect image by the module of the model, preliminarily extracting the characteristics of the product defect image by using a normal convolution layer in the module I, and then normalizing the extracted characteristic data by using a batch-normalization layer to prevent over-fitting of the network;
step 5, the result processed by the module I in the step 4 is used as input to enter a module II, the module II firstly utilizes the normal convolution layer to extract the characteristics of input data, then utilizes the deformable convolution layer to extract the multi-angle defect characteristics in the data, and finally utilizes the batch-normalization layer to normalize the characteristic data so as to prevent over-fitting of the network;
step 6: taking the result processed by the module II in the step 5 as an input to enter a module III, firstly, processing input data by using a first full connection layer of the module III, then, processing the data by using a second full connection layer as an output layer, wherein the number of nodes of the output layer corresponds to the number of types of defects, each node of the output layer gives a detection result belonging to each single defect mode through a sigmoid activation function, and finally, the output results of all nodes of the output layer are detection results;
and 7: calculating the cross entropy between the output result and the real defect label by using the output result of the step 6, training the model by back propagation, and detecting the mixed defect mode by using the trained model;
and 8: and (4) taking the model trained in the step (7) as a detection model, taking the image data of the detected product defect as the input of the detection model, and sequentially carrying out the steps (4), (5) and (6) in the detection process, so that the detection result is obtained, and finally, the accurate identification of the product surface defect is realized.
Preferably, the model established in step 1 includes three modules two, the result processed by the module two in step 5 is input into the next module two for repeated processing, the result processed by the next module two is input into the module three through the third module two for repeated processing, and the result processed by the third module two is input into the module three.
Preferably, the labeling method in step 2 is one-hot encoding, the dimension of the encoding is the number of types of defects, and for each piece of defect image data, if there is a certain defect, the value of the corresponding encoding position is 1, otherwise, it is 0.
Preferably, the output result is 1 when the defect is detected in the step 6, otherwise 0 is output.
Preferably, step 8 considers the defect coded as 1 in the detection result as the corresponding defect.
Compared with the prior art, the invention has the beneficial effects that:
the invention designs a product surface defect detection model and a detection method based on deformable convolution by adopting a deep learning technology, firstly designs a model structure and parameters based on a deformable convolution neural network, decomposes mixed defect mode identification into single type defect identification by adopting a one-hot coding mode according to a model output result, carries out network model training by adopting deep learning, labels data of the mixed defect mode by adopting the one-hot coding method, and then detects image data of a product of the mixed defect mode. The method can better meet the detection task, is efficient and accurate, improves the detection accuracy of the product in the mixed defect mode, reduces the false detection rate and the omission factor, has very high application value and economic benefit, and can be better applied to the actual product surface defect detection task as proved by actual verification.
Drawings
FIG. 1 is a diagram of a model for detecting surface defects of a product based on deformable convolution according to the present invention;
FIG. 2 is a network model parameter diagram of the present invention;
FIG. 3 is a specific example single defect mode data image;
FIG. 4 is a specific example hybrid defect mode data image;
FIG. 5 is a concrete example data annotation result;
FIG. 6 is a schematic diagram of the implementation of normal convolution;
FIG. 7 is a schematic diagram of an implementation of a deformable convolution;
fig. 8 is a diagram of a full connection layer structure.
Detailed Description
In order to make the invention more comprehensible, preferred embodiments are described in detail below with reference to the accompanying drawings.
The invention relates to a product surface defect detection model based on deformable convolution, which comprises a first module, a second module and a third module, wherein surface defect image data are used as the input of the whole detection model and are processed by the first module to be used as the input of the second module, the output of the second module is used as the input of the second module, and the output of the second module is processed by the third module to output a detection result;
the module I comprises a normal convolution layer and a batch-normalization layer, the surface defect image data are processed, the normal convolution layer is used for preliminarily extracting the characteristics of the product surface defect image data, and then the batch-normalization layer is used for normalizing the characteristic data to prevent network overfitting;
the second module comprises a normal convolution layer, a deformable convolution layer and a batch-normalization layer, wherein in the second module, the normal convolution layer is used for extracting the characteristics of the input data of the second module, the deformable convolution layer is used for extracting the multi-angle defect characteristics in the data, and the batch-normalization layer is used for normalizing the characteristic data to prevent over-fitting of the network;
and the third module comprises two fully-connected layers, the first fully-connected layer is used for processing the input data of the third module, the second fully-connected layer is used as an output layer for processing the data, the output results of all nodes of the output layer are detection results, the activation function of the output layer adopts a sigmoid activation function, the output result adopts a one-hot coding mode, the detection result of each defect corresponds to one output node, the number of the output nodes is equal to the number of defect types, the method can be used for decomposing the defects of the mixed defect mode and decomposing the identification of the mixed defect mode into the identification of a single defect in the model, and the method can be used for identifying the product defect image of the mixed defect mode.
The following describes a method for detecting surface defects of a product based on deformable convolution by taking wafer map detection in industrial defect detection as an example and combining fig. 1 and fig. 2, and specifically includes the following steps:
step 1, establishing a model based on a deformable convolution neural network, wherein the overall network structure diagram of the model is shown in fig. 1, the model network structure comprises a module I, a module II and a module III, the module I comprises a normal convolution layer and a batch-normalization layer, the module II comprises a normal convolution layer, a deformable convolution layer and a batch-normalization layer, and the module III comprises two fully-connected layers.
The normal convolutional layer is used for initially extracting features of the wafer map, convolution kernels in the normal convolutional layer are similar to a filter when convolution calculation is performed, and as shown in fig. 6, sliding sampling calculation is performed on a feature map of a previous layer by a certain step length to obtain a numerical value of each point of a feature map of a next layer. In normal convolution, each position p of the next layer signature0The output result of (1) is:
Figure RE-GDA0002597946390000051
wherein p isnThe pixel point represented in the convolution kernel, w (p)n) Is a pixel point pnX represents the input feature map of the previous layer.
The basic idea of the deformable convolution is to replace the original fixed position sampling by adding offset to each sampling point on the convolution kernel in the X-axis and Y-axis directions respectively, and the added offset in the two directions is part of the network structureEnd-to-end learning can be performed by gradient backpropagation. After the offset is added for learning, the positions of sampling points of the deformable convolution kernel can be dynamically adjusted according to the angle of the defect mode of the current wafer map, and adaptive change according to multiple angles is realized. In a deformable convolution kernel, an offset vector delta p is introducedn. In the deformable convolution, each position p of the next layer feature map0The output result of (1) is:
Figure RE-GDA0002597946390000052
wherein, Δ pn(a, b). a and b are respectively pixel points pnOffset in the x-axis and y-axis. In calculating the offset of the convolution kernel sample point, the calculated offset result Δ pnUsually, the image coordinate index data is an integer, and therefore, it is necessary to perform bilinear interpolation on the offset result to make the offset result an integer.
FIG. 7 demonstrates the implementation of the deformable convolution. The original normal convolution is divided into two paths, wherein the former path is learned by a convolution layer with an offset delta pnAnd obtaining an overall offset domain such as H W2N, wherein H and W are the length and width of the input layer, N is the size of the convolution kernel R, and 2N indicates that the offset has two directions of an x axis and a y axis. After obtaining the offset of the whole convolution kernel to the sampling points of the input layer, for each convolution window of the original normal convolution, the window is not a normal rectangular sliding window (such as the convolution layer area in the figure) any more, but an irregular window (such as the offset area in the figure) of each sampling point subjected to offset calculation. Respectively adding offset vectors to original fixed sampling points in an input layer, then rounding the offset vectors through a bilinear interpolation algorithm, enabling an output characteristic diagram with the offset vectors, an input characteristic diagram and an input characteristic diagram to have consistent spatial resolution, and finally transmitting the spatial resolution to a next network layer to realize the extraction of multi-angle characteristics.
The Batch-normalization layer (BN) is used for normalizing data, reducing the influence of parameter initialization in the model, preventing gradient disappearance during back propagation and improving the generalization capability of the recognition model to a certain extent.
Assume the inputs to the BN layer in the convolutional network are:
X＝{x1,x2,...,xn}
first, the mean value of the input elements needs to be found:
Figure BDA0002542987040000061
the variance of the input elements is then found:
Figure BDA0002542987040000062
each element is then normalized:
Figure BDA0002542987040000063
and finally, carrying out scale scaling and offset operation on the data in the BN layer, namely converting the data into the original distribution to realize identity conversion. The scaling and shifting operations are as follows:
yi＝γi*xi'+β
wherein gamma isiAnd β are the self-learning parameters of the model, which can be learned in the training process.
The structure of the full link layer is shown in fig. 8, after defect feature information is extracted by performing operations such as convolution on the wafer map, the full link layer is required to further identify the defect mode. Each neuron of the full connection layer is connected with all neurons of the previous layer and is used for integrating the learned characteristics of the previous layer for classification, and the mathematical expression of the classification is as follows:
yj＝wij*xi+bi
in the above formula wijAnd biThe weights and offsets of the fully connected layer are shown, i is the number of neurons in the current layer, and j is the number of neurons in the next layer.
And 2, marking the wafer map data set for training, wherein the marking method is one-hot coding, and the dimension of the coding is the number of the types of the defects.
The single defect labeling data types are shown in fig. 3, and include a center defect, a circular ring defect, a scratch, a local area defect, an edge ring defect, an edge local ring defect, a global defect, and a random defect; the type of the labeled data of the mixed defect is shown in FIG. 4, and includes (a) a defect of a central plus edge local ring, (b) a defect of a central plus edge ring, (c) a defect of a ring plus local area plus scratch, (d) a defect of a central plus scratch, (e) a defect of a central plus local area plus edge local area plus scratch, (f) a defect of a ring plus local area, (g) a defect of a ring plus scratch, (h) a defect of a central plus edge ring plus scratch, (i) the defect of the local area plus the edge local ring, (j) the defect of the central plus the local area plus the edge ring plus the scratch, (k) the defect of the ring plus the local area plus the edge local ring, (l) the defect of the central plus the local area plus the edge ring, (m) the defect of the scratch plus the edge ring, (n) the defect of the local area plus the edge ring, (o) the defect of the ring plus the local area plus the edge local ring plus the scratch. For each piece of defective image data, if there is a certain defect, the value of the corresponding coding position is 1, otherwise, it is 0, and the labeling result is shown in fig. 5;
step 3, using the wafer map defect image data of the product marked in the step 2 as the input of a detection model;
step 4, processing the input images by the model module, preliminarily extracting the characteristics of the product defect images by using a normal convolution layer (C) in the module I, wherein the input characteristic diagram size of the normal convolution layer (C) is [ n,32,32,1], n represents the number of the input images of each batch, the output characteristic diagram size is [ n,32,32,32], the convolution kernel is 5 x 5, and the activation function adopts a Relu function; normalizing the feature data by using a batch-normalization layer (BN), so as to prevent overfitting of the network, wherein the input feature diagram size of the batch-normalization layer is [ n,32,32,32], and the output feature diagram size is [ n,32,32,32 ];
step 5, the result processed by the module I in the step 4 is used as input and enters a first module II, the first module II firstly extracts the characteristics of input data by using a normal convolution layer (C), then extracts multi-angle defect characteristics in the data by using a deformable convolution layer (DC), and finally normalizes the characteristic data by using a batch-normalization layer (BN) to prevent network overfitting; and inputting the result processed by the first module II into the next module II for repeated processing, and inputting the result processed by the next module II into the third module II for repeated processing, wherein the result processed by the third module II is used as input to enter the third module.
The normal convolution layer (C) of the first module II inputs the characteristic diagram size [ n,32,32,32], outputs the characteristic diagram size [ n,16,16,64], the convolution kernel is 3 x 3, and the activation function adopts a Relu function; the variable-shape convolutional layer (DC) has the input characteristic diagram size of [ n,16,16,64], the output characteristic diagram size of [ n,16,16,64], the convolution kernel of 3 x 3, and the activation function adopts a Relu function; the BN layer input feature diagram size is [ n,16,16,64], and the output feature diagram size is [ n,16,16,64 ];
the normal convolution layer (C) of the second module II inputs the characteristic diagram size of [ n,16,16,64], outputs the characteristic diagram size of [ n,8, 128], the convolution kernel is 3 x 3, and the activation function adopts a Relu function; the input characteristic diagram size of the deformable convolution layer (DC) is [ n,8, 128], the output characteristic diagram size is [ n,8, 128], the convolution kernel is 3 x 3, and the activation function adopts a Relu function; the BN layer input feature diagram size is [ n,8, 128], and the output feature diagram size is [ n,8, 128 ];
the input feature size of the normal convolution layer (C) of the second module is [ n,8, 128], the output feature size is [ n,4, 256], the convolution kernel is 3 x 3, and the activation function adopts a Relu function; the input characteristic diagram size of the deformable convolution layer (DC) is [ n,4, 256], the output characteristic diagram size is [ n,4, 256], the convolution kernel is 3 x 3, and the Relu function is adopted as the activation function; the input feature size of the BN layer is [ n,4, 256], and the output feature size is [ n,4, 256 ].
And 6, taking the result processed by the third module II in the step 5 as input to enter a third module, wherein the third module consists of two fully-connected layers, the first fully-connected layer is used for processing data, the second fully-connected layer is used as an output layer for processing data, and the number of nodes of the output layer corresponds to the number of types of defects. And (4) through the sigmoid activation function, each node of the output layer gives a detection result belonging to each single defect mode, and the detected output is 1, otherwise, the detected output is 0. And finally, outputting the output results of all nodes of the layer as detection results.
The input characteristic diagram size of the first full-connection layer is [ n,4 x 256], the output characteristic diagram size is [ n,128], and the activation function adopts a Relu function; and the second full-link layer inputs a characteristic diagram size of [ n,128], outputs a characteristic diagram size of [ n,8], and gives detection results belonging to each single defect mode through a sigmoid activation function.
Step 7, calculating the cross entropy between the output result and the real defect label by using the output result of the step 6, training the model by back propagation, and detecting the defect image of the wafer map by using the trained model;
and 8, taking the model trained in the step 7 as a detection model, taking wafer map defect image detection data as input of the detection model, and sequentially carrying out the steps 4, 5 and 6 in the detection process to obtain a detection result, and taking the defect coded 1 in the detection result as a corresponding defect, so as to finally realize accurate identification of the surface defect of the product, wherein the final detection overall precision is 93.20%.

Claims (6)

1. A product surface defect detection model based on deformable convolution is characterized in that: the surface defect image data are processed by the first module and then serve as the input of the second module, and the detection result is output by the third module after being processed by the second module;
the module I comprises a normal convolution layer and a batch-normalization layer, the surface defect image data are processed, the normal convolution layer is used for preliminarily extracting the characteristics of the product surface defect image data, and then the batch-normalization layer is used for normalizing the characteristic data to prevent network overfitting;
the second module comprises a normal convolution layer, a deformable convolution layer and a batch-normalization layer, the second module firstly utilizes the normal convolution layer to extract the characteristics of the input data of the second module, then utilizes the deformable convolution layer to extract the multi-angle defect characteristics in the data, and finally utilizes the batch-normalization layer to normalize the characteristic data to prevent over-fitting of the network;
and the third module comprises two fully-connected layers, the first fully-connected layer is used for processing input data of the third module, the second fully-connected layer is used as an output layer for processing the data, the output results of all nodes of the output layer are detection results, the activation function of the output layer adopts a sigmoid activation function, the output result adopts a one-hot coding mode, the detection result of each defect corresponds to one output node, and the number of the output nodes is equal to the number of defect types.
2. A method for detecting surface defects of products based on deformable convolution, which is characterized by applying the product surface defect detection model based on deformable convolution according to claim 1, and comprises the following steps:
step 1, establishing a model based on a deformable convolution neural network, wherein the model comprises a module I, a plurality of modules II and a module III;
the module comprises a normal convolution layer and a batch-normalization layer;
the module two comprises a normal convolution layer, a deformable convolution layer and a batch-normalization layer;
the third module comprises two fully-connected layers;
step 2, marking a surface defect image data set for training;
step 3, taking the surface defect image data marked in the step 2 as the input of the model in the step 1;
step 4, processing the input product defect image by the module of the model, preliminarily extracting the characteristics of the product defect image by using a normal convolution layer in the module I, and then normalizing the extracted characteristic data by using a batch-normalization layer to prevent over-fitting of the network;
step 5, the result processed by the module I in the step 4 is used as input to enter a module II, the module II firstly utilizes the normal convolution layer to extract the characteristics of input data, then utilizes the deformable convolution layer to extract the multi-angle defect characteristics in the data, and finally utilizes the batch-normalization layer to normalize the characteristic data so as to prevent over-fitting of the network;
step 6: taking the result processed by the module II in the step 5 as an input to enter a module III, firstly, processing input data by using a first full connection layer of the module III, then, processing the data by using a second full connection layer as an output layer, wherein the number of nodes of the output layer corresponds to the number of types of defects, each node of the output layer gives a detection result belonging to each single defect mode through a sigmoid activation function, and finally, the output results of all nodes of the output layer are detection results;
and 7: calculating the cross entropy between the output result and the real defect label by using the output result of the step 6, training the model by back propagation, and detecting the mixed defect mode by using the trained model;
and 8: and (4) taking the model trained in the step (7) as a detection model, taking the image data of the detected product defect as the input of the detection model, and sequentially carrying out the steps (4), (5) and (6) in the detection process, so as to obtain a detection result and finally realize the accurate identification of the product surface defect.
3. The method for detecting the surface defects of the product based on the deformable convolution as claimed in claim 2, characterized in that: the model established in the step 1 comprises three modules II, the result processed by the module II in the step 5 is input into the next module II for repeated processing, the result processed by the next module II is input into the module III through the third module II for repeated processing, and the result processed by the third module II is input into the module III.
4. The method for detecting the surface defects of the product based on the deformable convolution as claimed in claim 2, characterized in that: the marking method in the step 2 is one-hot coding, the number of dimensions of the coding is the number of types of defects, and for each piece of defect image data, if a certain defect exists, the value of the corresponding coding position is 1, otherwise, the value is 0.
5. The method for detecting the surface defects of the product based on the deformable convolution as claimed in claim 2, characterized in that: and (6) outputting a result of 1 when the defect is detected in the step 6, otherwise, outputting 0.
6. The method of claim 5, wherein the method comprises: step 8 regards the coded 1 in the detection result as a corresponding defect.
CN202010552242.7A 2020-06-17 2020-06-17 Product surface defect detection model and detection method based on deformable convolution Pending CN111739001A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010552242.7A CN111739001A (en) 2020-06-17 2020-06-17 Product surface defect detection model and detection method based on deformable convolution

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010552242.7A CN111739001A (en) 2020-06-17 2020-06-17 Product surface defect detection model and detection method based on deformable convolution

Publications (1)

Publication Number Publication Date
CN111739001A true CN111739001A (en) 2020-10-02

Family

ID=72649600

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010552242.7A Pending CN111739001A (en) 2020-06-17 2020-06-17 Product surface defect detection model and detection method based on deformable convolution

Country Status (1)

Country Link
CN (1) CN111739001A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116309587A (en) * 2023-05-22 2023-06-23 杭州百子尖科技股份有限公司 Cloth flaw detection method and device, electronic equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108510467A (en) * 2018-03-28 2018-09-07 西安电子科技大学 SAR image target recognition method based on variable depth shape convolutional neural networks
CN109977808A (en) * 2019-03-11 2019-07-05 北京工业大学 A kind of wafer surface defects mode detection and analysis method
CN110705457A (en) * 2019-09-29 2020-01-17 核工业北京地质研究院 Remote sensing image building change detection method
CN110717368A (en) * 2018-07-13 2020-01-21 北京服装学院 Qualitative classification method for textiles
CN110717881A (en) * 2018-07-12 2020-01-21 长鑫存储技术有限公司 Wafer defect identification method and device, storage medium and terminal equipment
CN110826596A (en) * 2019-10-09 2020-02-21 天津大学 Semantic segmentation method based on multi-scale deformable convolution
CN111160273A (en) * 2019-12-31 2020-05-15 北京云智空间科技有限公司 Hyperspectral image space spectrum combined classification method and device
CN111199543A (en) * 2020-01-07 2020-05-26 南京航空航天大学 Refrigerator-freezer surface defect detects based on convolutional neural network

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108510467A (en) * 2018-03-28 2018-09-07 西安电子科技大学 SAR image target recognition method based on variable depth shape convolutional neural networks
CN110717881A (en) * 2018-07-12 2020-01-21 长鑫存储技术有限公司 Wafer defect identification method and device, storage medium and terminal equipment
CN110717368A (en) * 2018-07-13 2020-01-21 北京服装学院 Qualitative classification method for textiles
CN109977808A (en) * 2019-03-11 2019-07-05 北京工业大学 A kind of wafer surface defects mode detection and analysis method
CN110705457A (en) * 2019-09-29 2020-01-17 核工业北京地质研究院 Remote sensing image building change detection method
CN110826596A (en) * 2019-10-09 2020-02-21 天津大学 Semantic segmentation method based on multi-scale deformable convolution
CN111160273A (en) * 2019-12-31 2020-05-15 北京云智空间科技有限公司 Hyperspectral image space spectrum combined classification method and device
CN111199543A (en) * 2020-01-07 2020-05-26 南京航空航天大学 Refrigerator-freezer surface defect detects based on convolutional neural network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JIAN ZHU ET AL.: "Deformable Convolutional Neural Networks for Hyperspectral Image Classification", 《IEEE GEOSCIENCE AND REMOTE SENSING LETTERS》 *
S. Y. GUAN ET AL.: "A SURFACE DEFECT DETECTION METHOD OF THE MAGNESIUM ALLOY SHEET BASED ON DEFORMABLE CONVOLUTION NEURAL NETWORK", 《METALURGIJA》 *
TAKESHI NAKAZAWA ET AL.: "Wafer Map Defect Pattern Classification and Image Retrieval Using Convolutional Neural Network", 《IEEE TRANSACTIONS ON SEMICONDUCTOR MANUFACTURING》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116309587A (en) * 2023-05-22 2023-06-23 杭州百子尖科技股份有限公司 Cloth flaw detection method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
CN110570396B (en) Industrial product defect detection method based on deep learning
CN112085735B (en) Aluminum material image defect detection method based on self-adaptive anchor frame
Su et al. Deep learning-based solar-cell manufacturing defect detection with complementary attention network
CN110033000B (en) Text detection and identification method for bill image
CN108171209B (en) Face age estimation method for metric learning based on convolutional neural network
CN111563473B (en) Remote sensing ship identification method based on dense feature fusion and pixel level attention
CN109840521B (en) Integrated license plate recognition method based on deep learning
Al-Haija et al. Multi-class weather classification using ResNet-18 CNN for autonomous IoT and CPS applications
CN111008632B (en) License plate character segmentation method based on deep learning
CN110163286B (en) Hybrid pooling-based domain adaptive image classification method
CN107545263B (en) Object detection method and device
CN112561910A (en) Industrial surface defect detection method based on multi-scale feature fusion
CN112381788A (en) Part surface defect increment detection method based on double-branch matching network
CN110648310A (en) Weak supervision casting defect identification method based on attention mechanism
CN111062381B (en) License plate position detection method based on deep learning
CN115035371B (en) Well wall crack identification method based on multi-scale feature fusion neural network
CN114581782A (en) Fine defect detection method based on coarse-to-fine detection strategy
Pham et al. Skeletal movement to color map: A novel representation for 3D action recognition with inception residual networks
CN115527072A (en) Chip surface defect detection method based on sparse space perception and meta-learning
CN111739001A (en) Product surface defect detection model and detection method based on deformable convolution
CN113421223B (en) Industrial product surface defect detection method based on deep learning and Gaussian mixture
Singh et al. CNN based approach for traffic sign recognition system
CN111126173A (en) High-precision face detection method
CN114972871A (en) Image registration-based few-sample image anomaly detection method and system
CN111325270B (en) Dongba text recognition method based on template matching and BP neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20201002

WD01 Invention patent application deemed withdrawn after publication