CN113808079B - Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet - Google Patents

Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet Download PDF

Info

Publication number
CN113808079B
CN113808079B CN202110918998.3A CN202110918998A CN113808079B CN 113808079 B CN113808079 B CN 113808079B CN 202110918998 A CN202110918998 A CN 202110918998A CN 113808079 B CN113808079 B CN 113808079B
Authority
CN
China
Prior art keywords
defect
aglnet
image
deep learning
learning model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110918998.3A
Other languages
Chinese (zh)
Other versions
CN113808079A (en
Inventor
余建波
王延舒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongji University
Original Assignee
Tongji University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongji University filed Critical Tongji University
Priority to CN202110918998.3A priority Critical patent/CN113808079B/en
Publication of CN113808079A publication Critical patent/CN113808079A/en
Application granted granted Critical
Publication of CN113808079B publication Critical patent/CN113808079B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The invention provides a self-adaptive detection method for surface defects of industrial products based on an deep learning model AGLNet, which is characterized by comprising the following steps: step S1, carrying out image acquisition on surface defects of industrial products on a production line to obtain defect images; s2, manually marking the defect image to obtain tag files of different categories; s3, performing image enhancement operation on the defect image to obtain an enhanced image; s4, taking the tag file and the corresponding enhanced image as a surface defect data set; step S5, a defect detector model based on a deep learning network is constructed, a surface defect data set is taken as input, and defect data in the manufacturing process is trained based on the deep learning model AGLNet; s6, carrying out real-time defect detection on products on an industrial production line through a trained AGLNet model, and obtaining the type and position information of the defects; and S7, carrying out arrangement statistics on the type and the position information of the defects, and analyzing the reasons for generating the defects.

Description

Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet
Technical Field
The invention belongs to the technical field of product defect detection, and relates to an industrial product surface defect self-adaptive detection method based on a deep learning model AGLNet.
Background
The development direction of the industrial manufacturing industry in China is towards high reliability, high precision, zero defects and intelligent high-speed development. The high precision, high speed and high stability are important references in the industrial product defect detection and identification process, and whether a product meets the industrial quality requirement is judged when the surface of a product part has defects. The identification and detection of the surface defects of the product parts in the traditional industry are mainly finished in a manual quality inspection mode, but the manual detection method consumes a large amount of manpower, has the problems of low efficiency, low precision, high manpower cost and the like, and can not meet the requirement of long-time real-time detection. Therefore, there is an urgent need in the actual industrial defect detection to automatically detect defects on the surface of a product using a machine by using computer technology.
The machine vision detection technology based on the artificial features overcomes the defects of the artificial quality inspection to a certain extent, realizes automatic defect detection, has the advantages of strong classification capacity, high detection precision, low detection cost and the like, and improves the yield of industrial production. However, the detection method still has the following defects that the actual requirements of industrial production are difficult to meet: firstly, the manual feature extraction process is complex, and the feature information hardly contains all defect features; secondly, the problem of excessive human intervention still exists in the feature extraction, and the feature extraction is difficult to have good portability depending on the manually designed features; meanwhile, in a complex detection environment, complex defect space aggregation or multi-target detection environment, the detection accuracy based on machine vision is relatively poor, and the generalization capability is poor; finally, all algorithms and parameters based on machine vision need to be redesigned and developed when the type of product being tested changes.
Disclosure of Invention
In order to solve the problems, the invention provides a defect detection method with high efficiency, high precision and good applicability, which adopts the following technical scheme:
the invention provides a self-adaptive detection method for surface defects of industrial products based on an deep learning model AGLNet, which is characterized by comprising the following steps: step S1, carrying out image acquisition on surface defects of industrial products on a production line to obtain defect images; s2, manually labeling the defect image based on an image labeling module to obtain label files of different categories; s3, performing image enhancement operation on the defect image to obtain an enhanced image; s4, taking the tag file and the corresponding enhanced image as a surface defect data set; step S5, a defect detector model based on a deep learning network is constructed, a surface defect data set is taken as input, and defect data in the manufacturing process is trained based on the deep learning model AGLNet; s6, carrying out real-time defect detection on products on an industrial production line through a trained AGLNet model, and obtaining the type and position information of the defects; and S7, sorting and counting the types and the position information of the defects, and analyzing the reasons for generating the defects, wherein the deep learning model AGLNet is obtained by combining a ResNet_FPN feature extraction network, an AT-RPN candidate frame extraction network and a Global local regression algorithm module.
The self-adaptive detection method for the surface defects of the industrial product based on the deep learning model AGLNet can also have the technical characteristics that the image enhancement operation is to perform image inversion, image translation, image brightness change and noise enhancement on the defective image.
The self-adaptive detection method for the surface defects of the industrial product based on the deep learning model AGLNet can also have the technical characteristics that the tag file contains defect type information and defect position information, the tag file is an xml file, and each defect image corresponds to one xml file.
The adaptive detection method for the surface defects of the industrial product based on the deep learning model AGLNet provided by the invention can also have the technical characteristics that the ResNet_FPN characteristic extraction network is constructed by a convolutional neural network taking a residual network and a characteristic pyramid network as backbones, and the working process of the ResNet_FPN characteristic extraction network comprises the following steps: performing feature extraction on the defect images according to the sequence from low latitude to high latitude based on a residual error network, and generating a feature map of each stage; and carrying out up-sampling and transverse connection operation from high latitude to low latitude on the feature graphs of each stage based on the feature pyramid network, and carrying out network convolution on the feature graphs to finally obtain feature graphs with different scales from high latitude abstract features to low latitude bottom features.
The method for adaptively detecting the surface defects of the industrial product based on the deep learning model AGLNet provided by the invention can also have the technical characteristics that the AT-RPN candidate frame extraction network is used for adaptively extracting the rectangular labels in the defect data set, and the adaptive extraction comprises the following steps: a1, sequentially inputting feature images with different scales into an AT-RPN candidate frame extraction network, automatically generating a plurality of anchor frames with different scales based on initialized parameter setting, and sequentially sliding on the feature images of each layer; step A2, based on the step A1 and a preset parameter optimization algorithm, generating optimized anchor frame parameters and anchor frames along with iteration, and acquiring a plurality of candidate areas where a defect target exists; step A3, selecting a candidate region with defects based on an anchor frame according to the principle of IoU, performing IoU calculation on the candidate region in each anchored frame, adding a subsequent operation considered as a positive sample and having IoU of more than 0.7, adding a subsequent operation considered as a negative sample and having IoU of less than 0.3, and taking other regions as useless samples and not adding the subsequent operation; and A4, extracting the category and information position information of the defects in the candidate area screened in the step A3, comparing the category and information position information with the real information of the defect target, respectively calculating the difference value between the position information predicted by the AT-RPN and the real position information and the difference value between the category information predicted by the AT-RPN and the real category information, inputting the difference value into the calculation of a loss function, and continuously optimizing and reducing the loss.
The adaptive detection method for the industrial product surface defects based on the deep learning model AGLNet can also have the technical characteristics that the Global local regression algorithm module carries out regression calculation on the screened candidate areas to obtain the position information of the defect targets, and the AT-RPN candidate frame extraction network classifies the screened candidate areas to obtain the category information of the defect targets.
The actions and effects of the invention
According to the method for adaptively detecting the surface defects of the industrial product based on the deep learning model AGLNet, the trained deep learning model AGLNet is used for extracting characteristics of a defect target in a defect image and carrying out classification regression detection to obtain defect information in the defect image. The deep learning model is obtained by combining a ResNet_FPN feature extraction network, an AT-RPN candidate frame extraction network and a Global local regression algorithm module, after the ResNet_FPN feature extraction network carries out network convolution on a defect image to obtain a feature map, the AT-RPN candidate frame extraction network carries out feature extraction on the feature map to obtain candidate areas, after each candidate area is screened, the Global local regression algorithm module carries out regression calculation on the screened candidate areas to obtain information of a defect target.
Therefore, the self-adaptive detection method for the surface defects of the industrial product based on the deep learning model AGLNet does not need the test accumulation of priori knowledge, discards the mode of artificial parameter adjustment, reduces the memory requirement of a computer, relieves the problem of low detection precision caused by large defect shape difference and dense space positions, greatly improves the efficiency of defect detection in the manufacturing industry, avoids the false detection results caused by complex operation environment, subjective judgment errors and the like, realizes better and more accurate defect detection, realizes real-time online detection, and has good applicability in complex environment and multi-target scenes.
Drawings
FIG. 1 is a flow chart of an adaptive detection method for surface defects of an industrial product based on a deep learning model AGLNet in an embodiment of the invention;
FIG. 2 is a schematic structural diagram of a convolutional neural network of an adaptive detection method for surface defects of an industrial product based on a deep learning model AGLNet in an embodiment of the invention;
fig. 3 is a schematic diagram of an AT-RPN candidate block extraction network according to an embodiment of the present invention;
FIG. 4 is a flow chart of a TPE adaptive anchor frame parameter adjustment module in an embodiment of the invention.
Detailed Description
In order to make the technical means, creation characteristics, achievement purposes and effects achieved by the invention easy to understand, the method for adaptively detecting the surface defects of the industrial product based on the deep learning model AGLNet is specifically described below with reference to the embodiments and the accompanying drawings.
< example >
The adaptive detection method for the surface defects of the industrial product based on the deep learning model AGLNet is realized based on a computer, hardware adopted by the computer is configured as an Intel (R) Core (TM) i7-8700K processor, a GTX 1080Ti graphic card, a software environment is CUDA10.0 and cuDNN7.6, and a development environment is Ubuntu18.04.
Fig. 1 is a flowchart of an adaptive detection method for surface defects of an industrial product based on a deep learning model AGLNet in an embodiment of the present invention.
As shown in fig. 1, the adaptive detection method for the surface defects of the industrial product based on the deep learning model AGLNet comprises the following steps:
step S1, carrying out image acquisition on the surface defects of industrial products on a production line to obtain defect images.
And S2, manually labeling defects in the enhanced image based on the image labeling module to obtain different types of label files.
In this embodiment, the tag file contains defect type information and defect location information, and the tag file is an xml file, and each defective image corresponds to one xml file.
And S3, performing image enhancement operation on the defect image to obtain an enhanced image.
In this embodiment, the image enhancement operation is to perform image inversion, image translation, image brightness change, and noise enhancement on the defective image.
And S4, taking the label file and the corresponding enhanced image as a surface defect data set.
And S5, constructing a defect detector model based on a deep learning network, taking a surface defect data set as input, and training defect data in the manufacturing process based on the deep learning model AGLNet.
And S6, carrying out real-time defect detection on products on the industrial production line through the trained AGLNet model, and obtaining the type and position information of the defects.
And S7, carrying out arrangement statistics on the type and the position information of the defects, and analyzing the reasons for generating the defects.
In this embodiment, the deep learning model AGLNet is completed through Pycharm and an open-source deep learning framework pytorch1.1.0, and the SGD optimizer is used to optimize network parameters in training, where the AGLNet model is obtained by combining a resnet_fpn feature extraction network, an AT-RPN candidate frame extraction network and a Global local regression algorithm module. Specifically:
the ResNet_FPN feature extraction network is constructed by a convolutional neural network taking a residual network and a feature pyramid network as backbones.
Fig. 2 is a schematic structural diagram of a convolutional neural network of an adaptive detection method for surface defects of an industrial product based on a deep learning model AGLNet in an embodiment of the present invention.
As shown in fig. 2, the res net FPN feature extraction network operates as follows:
firstly, carrying out feature extraction on a defect image according to the sequence from low latitude to high latitude based on a residual network, and generating a feature map { C2, C3, C4, C5} of each stage;
and then, carrying out up-sampling and transverse connection operation from high latitude to low latitude on the feature graphs { C2, C3, C4 and C5} of each stage based on the feature pyramid network, and carrying out network convolution on the feature graphs to finally obtain feature graphs with different scales from high latitude abstract features to low latitude bottom features. Specifically:
the feature map C5 is subjected to 256 1×1 convolutions to obtain T5, and the result obtained by the feature map C4 through 256 1×1 convolutions is tensor added with the result obtained by the up-sampling of T5 to obtain T4.
In this embodiment, the acquisition procedures of T3 and T2 are identical to the acquisition procedure of T4, and will not be described here again.
And finally, carrying out convolution on { T5, T4, T3 and T2} obtained based on tensor addition respectively by 256 pieces of 3×3 to obtain feature graphs of different scales from the high-dimensional abstract features to the low-dimensional bottom features.
Fig. 3 is a schematic diagram of an AT-RPN candidate block extraction network according to an embodiment of the present invention.
As shown in fig. 3, the AT-RPN candidate frame extraction network is configured to adaptively extract a label of a rectangular shape in the surface defect data set, that is, obtain a candidate region with a defect target based on an optimal anchor frame generated and screened by parameter optimization, and then predict a position and a category of the defect target in the candidate region. Specifically:
the adaptive extraction comprises the following steps:
and A1, sequentially inputting feature graphs with different scales into an AT-RPN candidate frame extraction network, automatically generating a plurality of anchor frames with different scales based on initialized parameter setting, and sequentially sliding on the feature graphs of each layer.
In this embodiment, 256 3×3 anchor frames are used to slide sequentially on the feature map of each layer.
And A2, generating optimized anchor frame parameters and anchor frames along with iteration based on the step A1 and a preset parameter optimization algorithm, and acquiring a plurality of candidate areas where the defect targets exist.
In this embodiment, the predetermined parameter optimization algorithm is a TPE optimization algorithm, a self-adaptive anchor frame parameter adjustment module is formed based on the TPE optimization algorithm, and an optimized anchor frame parameter is generated according to the anchor frame parameter adjustment module, and an anchor frame with a fixed size and different height-width ratios is mapped at each position where the window slides.
And A3, selecting a candidate region with defects based on an anchor frame according to the principle of IoU, performing IoU calculation on the candidate region in each anchored frame, adding a positive sample regarded as a subsequent operation with IoU being more than 0.7, adding a negative sample regarded as a subsequent operation with IoU being less than 0.3, and taking other regions as useless samples not to be added in the subsequent operation.
In the training process of the deep learning model AGLNet in this embodiment, a binary label is allocated to each anchor frame, and a IoU intersection ratio mode is adopted to determine whether a candidate region belongs to a target or a background, where the intersection ratio IoU represents the ratio of the intersection area and the union area between the anchor frame and the target real calibration frame, and is defined as:
Figure GDA0003307465590000091
wherein L is an allocated binary label, when IoU of a candidate region is more than or equal to 0.7, marking the candidate region as a positive sample, and adding the candidate region into a post operation; when IoU of the candidate region is less than 0.3, the candidate region is marked as a negative sample and no post-operation is added.
And A4, extracting the category information and the position information of the defects in the candidate area screened in the step A3, comparing the category information and the position information with the real information of the defect target, respectively calculating the difference value between the position information predicted by the AT-RPN and the real position information and the difference value between the category information predicted by the AT-RPN and the real category information, inputting the difference value into the calculation of a loss function, and continuously optimizing and reducing the loss.
In this embodiment, the surface defect dataset is continuously trained as input, and more optimized anchor frame generation parameters are iteratively obtained, so that candidate region information is obtained more quickly and better.
FIG. 4 is a flow chart of a TPE adaptive anchor frame parameter adjustment module in an embodiment of the invention.
As shown in fig. 4, in the AT-RPN candidate block extraction network, for a hyper-parametric model to be optimized:
Figure GDA0003307465590000101
wherein x is the hyper-parameter in the hyper-parameter model to be optimized, and χ is the range of the hyper-parameter.
Converting the algorithm to be optimized into an objective function:
x ξ+1 ＝arg x maxΓ(x；Ψ ξ ) (2)
wherein arg x maxΓ(x；Ψ ξ ) To collect the function (acquisition function), the function is to determine the input hyper-parameters among the existing plurality of hyper-parameters in each iteration.
At each iteration of the optimization process { ζ=1, 2,3. }, selecting an input hyper-parameter { x } ξ E χ }, into the original model Γ (x), the result is:
y t ＝Γ(x；Ψ ξ )+∈ (3)
where E is zero, is the mean gaussian distribution
Figure GDA0003307465590000102
Wherein σ is the variance of the noise.
This set of values { x } is observed so far ξ ,y ξ And added to existing observed data:
Ψ ξ+1 ＝{Ψ ξ ,(x ξ+1 ,y ξ+1 )} (4)
wherein, ψ is ξ For existing observed data, the next iteration is performed after new data is added.
In this embodiment, the desired functions employed are:
Figure GDA0003307465590000103
wherein y is * Is the threshold of the objective function, x is the proposed hyper-parameter, y is the actual value of the objective function using the hyper-parameter x, and p (y|x) is the probability representing y after x is given.
TPE optimization algorithm, on the basis of following Bayesian optimization, sets up:
Figure GDA0003307465590000104
from equation (6), when the value of the objective function is below the threshold { y }<y * When adopting density function
Figure GDA0003307465590000105
When the objective function value is greater than the threshold { y }>y * At } a density function g (x) is used.
The EI function for constructing the TPE optimization algorithm is as follows:
Figure GDA0003307465590000111
in the present embodiment, γ=p (y<y * ) Gamma is the value of the objective function below the thresholdProbability of value, and:
Figure GDA0003307465590000112
according to the above, we obtain:
Figure GDA0003307465590000113
the original EI function may be converted into:
Figure GDA0003307465590000114
in each iteration, the TPE optimization algorithm can determine x, which gives the optimal EI * Adding ψ as candidate superparameter point ξ And (3) performing the next iteration until the maximum iteration times or time are reached.
In this embodiment, the loss function of the AT-RPN network is composed of a classification loss function and a regression function, defined as:
Figure GDA0003307465590000115
wherein p is i Representing the prediction probability of the ith anchor frame, p i * Representing the probability of IOU calibration, namely:
when the calibration frame contains the target, p i * ＝1；
When the calibration frame does not contain a target, p i * ＝0。
In formula (12), N cls ，N reg Sum mu is a fixed value, L cls (. Cndot.) shows whether the log-loss function is the target:
L cls (p i ,p i * )＝-log[p i p i * +(1-p i )(1-p i * )] (13)
wherein L is reg (. Cndot.) the regression loss function after correction of the parameters is defined as:
L reg (r i ,r i * )＝∑ i∈{x,y,w,h} smooth L (r i -r i * ) 。(14)
and S5, inputting the defect image to be detected into a defect detector model for feature extraction and classification regression to obtain defect information in the defect image to be detected.
In this embodiment, the Global local regression algorithm module performs regression calculation on the candidate region after screening to obtain the location information of the defect target, and the AT-RPN candidate frame extraction network classifies the candidate region after screening to obtain the category information of the defect target. Specifically:
fig. 2 is a schematic structural diagram of a convolutional neural network of an adaptive detection method for surface defects of an industrial product based on a deep learning model AGLNet in an embodiment of the present invention.
As shown in fig. 2, first, a feature map divided into k×k cells is obtained based on ROIAlign, feature information of each cell is input into a full convolution network, and feature u of each cell is calculated i In the respective positions (x i ，y i ) The distances to the upper left and lower right corners of the real calibration frame G, thereby making predictions of the positional offsets (box offsets) to determine the precise location of the defect target. Specifically:
the position offset of each cell of the feature map is defined as follows:
Figure GDA0003307465590000121
Figure GDA0003307465590000122
wherein, I i ，t i ，r i ，b i Representing the distances of the cells to the left, upper, right and lower boundaries of the real calibration frame, respectively, (x) i ，y i ) Representation unitPosition coordinates of the grid, (x) l ，y t ) And (x) r ，y b ) And the coordinates of the upper left corner and the lower right corner of the real calibration frame are represented.
Then labeling each cell based on dilute vulcanization operation, judging the characteristics of each cell based on a two-term classification prediction algorithm, namely judging whether the cell belongs to a foreground characteristic or a background characteristic, and only allowing the cell with foreground characteristic information to be added into operation, wherein the definition is as follows:
Figure GDA0003307465590000131
wherein, c i The output quantity used as the cell classification judgment is added into the position offset calculation to form regression { l ] of five conditions i ，t i ，r i ，b i ，c i During training, two-term classification predicts output value c i The binary cross entropy loss is calculated by the incoming sigmoid activation function.
Finally, the location information of the defect is obtained by applying to five activated condition parameters { l }, based on the AGLNet model i ，t i ，r i ，b i ，c i And performing regression loss calculation prediction to obtain inactive condition parameters which do not participate in regression operation, wherein the classification information of the defects is a candidate region characteristic diagram obtained in the previous step, each candidate region is calculated through a full-connection layer and a softmax layer and is input into a certain defect category, and the classification probability of the defects is predicted by an output model to finally obtain the types of the defects.
In this embodiment, after the obtained information such as the defect type and the position is statistically processed, an expert is invited to analyze the cause of the defect based on the defect information after being statistically processed, so as to achieve the purpose of improving the chip yield.
Example operation and Effect
According to the industrial product surface defect self-adaptive detection method based on the deep learning model AGLNet, the trained deep learning model AGLNet is used for carrying out feature extraction and classification regression detection on a defect target in a defect image to obtain defect information in the defect image. The deep learning model is obtained by combining a ResNet_FPN feature extraction network, an AT-RPN candidate frame extraction network and a Global local regression algorithm module, after the ResNet_FPN feature extraction network carries out network convolution on a defect image to obtain a feature map, the AT-RPN candidate frame extraction network carries out feature extraction on the feature map to obtain candidate areas, after each candidate area is screened, the Global local regression algorithm module carries out regression calculation on the screened candidate areas to obtain information of a defect target.
According to the method for adaptively detecting the surface defects of the industrial product based on the deep learning model AGLNet, the test accumulation of priori knowledge is not needed, the mode of artificial parameter adjustment is abandoned, the memory requirement of a computer is reduced, the problems of high defect shape difference and low detection precision caused by dense space positions are solved, the defect detection efficiency in the manufacturing industry is greatly improved, the error detection results caused by complex operation environment, subjective judgment errors and the like are avoided, the defect detection can be better and more accurately performed, the real-time online detection is realized, and the method has good applicability in complex environments and multi-target scenes.
In the embodiment, compared with the prior art, the AT-RPN candidate frame extraction network in the AGLNet model discards the mode of artificial parameter adjustment, can autonomously learn super-parameter setting based on defect data aiming AT different surface defects, freely switch application scenes, is suitable for detecting various defects, reduces the memory requirement of a computer, and has higher generalization capability. Meanwhile, the AT-RPN candidate frame extraction network is a feature extraction structure formed by combining a residual network and a feature pyramid network, so that semantic loss caused by transmission among layers is reduced, the defect detection precision of the traditional deep learning model is improved, and the problems of large defect shape difference and dense space positions are effectively solved.
In the embodiment, the Global local regression algorithm module in the AGLNet model adopts a multipoint supervision positioning method to realize more accurate positioning of the defect target in the defect image, and achieves a better detection effect compared with the traditional deep learning model defect detection method.
The above examples are only for illustrating the specific embodiments of the present invention, and the present invention is not limited to the description scope of the above examples.

Claims (6)

1. An industrial product surface defect self-adaptive detection method based on an deep learning model AGLNet is characterized by comprising the following steps:
step S1, carrying out image acquisition on surface defects of industrial products on a production line to obtain defect images;
s2, manually labeling the defect image based on an image labeling module to obtain different types of label files;
s3, performing image enhancement operation on the defect image to obtain an enhanced image;
step S4, taking the tag file and the corresponding enhanced image as a surface defect data set;
step S5, a defect detector model based on a deep learning network is constructed, the surface defect data set is used as input, and defect data in the manufacturing process is trained based on the deep learning model AGLNet;
s6, carrying out real-time defect detection on products on an industrial production line through a trained AGLNet model, and obtaining the type and position information of the defects;
step S7, the defect type and the position information are processed and counted, the cause of the defect is analyzed,
the deep learning model AGLNet is obtained by combining a ResNet_FPN feature extraction network, an AT-RPN candidate frame extraction network and a Global local regression algorithm module.
2. The adaptive detection method for surface defects of industrial products based on deep learning model AGLNet according to claim 1, wherein the method comprises the following steps:
the image enhancement operation is to perform image inversion, image translation, image brightness change and noise enhancement on the defect image.
3. The adaptive detection method for surface defects of industrial products based on deep learning model AGLNet according to claim 1, wherein the method comprises the following steps:
wherein the tag file contains defect type information and defect location information,
the tag file is an xml file, and each defective image corresponds to one xml file.
4. The adaptive detection method for surface defects of industrial products based on deep learning model AGLNet according to claim 1, wherein the method comprises the following steps:
wherein the ResNet_FPN feature extraction network is constructed by a convolutional neural network taking a residual network and a feature pyramid network as backbones,
the ResNet_FPN feature extraction network working process comprises the following steps:
performing feature extraction on the defect image according to the sequence from low latitude to high latitude based on the residual error network, and generating a feature map of each stage;
and carrying out up-sampling and transverse connection operation from high latitude to low latitude on the feature map of each stage based on the feature pyramid network, and carrying out network convolution on the feature map to finally obtain feature maps with different scales from high latitude abstract features to low latitude bottom features.
5. The adaptive detection method for surface defects of industrial products based on deep learning model AGLNet as set forth in claim 4, wherein:
wherein the AT-RPN candidate box extraction network is used for adaptively extracting the labels of the rectangular shapes in the defect data set,
the adaptive extraction comprises the following steps:
a1, sequentially inputting the feature images with different scales into the AT-RPN candidate frame extraction network, automatically generating a plurality of anchor frames with different scales based on initialized parameter setting, and sequentially sliding on the feature images of each layer;
step A2, generating optimized anchor frame parameters and anchor frames along with iteration based on the step A1 and a preset parameter optimization algorithm, and acquiring a plurality of candidate areas where a defect target exists;
step A3, selecting a candidate region with defects based on an anchor frame according to the principle of IoU, performing IoU calculation on the candidate region in each anchored frame, adding a positive sample regarded as a subsequent operation with IoU being more than 0.7, adding a negative sample regarded as a subsequent operation with IoU being less than 0.3, and taking other regions regarded as useless samples not to be added in the subsequent operation;
and A4, extracting the category and information position information of the defects in the candidate area screened in the step A3, comparing the category and information position information with the real information of the defect target, respectively calculating the difference value between the position information predicted by the AT-RPN and the real position information and the difference value between the category information predicted by the AT-RPN and the real category information, inputting the difference value into the calculation of a loss function, and continuously optimizing and reducing the loss.
6. The adaptive detection method for surface defects of industrial products based on deep learning model AGLNet according to claim 5, wherein the method comprises the following steps:
wherein the Globallocalregression algorithm module carries out regression calculation on the candidate region after screening to obtain the position information of the defect target,
and the AT-RPN candidate frame extraction network classifies the screened candidate region to obtain the category information of the defect target.
CN202110918998.3A 2021-08-11 2021-08-11 Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet Active CN113808079B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110918998.3A CN113808079B (en) 2021-08-11 2021-08-11 Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110918998.3A CN113808079B (en) 2021-08-11 2021-08-11 Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet

Publications (2)

Publication Number Publication Date
CN113808079A CN113808079A (en) 2021-12-17
CN113808079B true CN113808079B (en) 2023-06-27

Family

ID=78893534

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110918998.3A Active CN113808079B (en) 2021-08-11 2021-08-11 Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet

Country Status (1)

Country Link
CN (1) CN113808079B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018108129A1 (en) * 2016-12-16 2018-06-21 北京市商汤科技开发有限公司 Method and apparatus for use in identifying object type, and electronic device
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
CN110555842A (en) * 2019-09-10 2019-12-10 太原科技大学 Silicon wafer image defect detection method based on anchor point set optimization
WO2020173036A1 (en) * 2019-02-26 2020-09-03 博众精工科技股份有限公司 Localization method and system based on deep learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018108129A1 (en) * 2016-12-16 2018-06-21 北京市商汤科技开发有限公司 Method and apparatus for use in identifying object type, and electronic device
CN109711474A (en) * 2018-12-24 2019-05-03 中山大学 A kind of aluminium material surface defects detection algorithm based on deep learning
WO2020173036A1 (en) * 2019-02-26 2020-09-03 博众精工科技股份有限公司 Localization method and system based on deep learning
CN110555842A (en) * 2019-09-10 2019-12-10 太原科技大学 Silicon wafer image defect detection method based on anchor point set optimization

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于Faster R-CNN的刨花板表面缺陷检测研究;彭煜;肖书浩;阮金华;汤勃;;组合机床与自动化加工技术(03);全文 *
基于深度学习的不规则特征识别检测技术;赵欣洋;蔡超鹏;王思;刘志远;;轻工机械(03);全文 *

Also Published As

Publication number Publication date
CN113808079A (en) 2021-12-17

Similar Documents

Publication Publication Date Title
CN109977808B (en) Wafer surface defect mode detection and analysis method
CN115351598A (en) Numerical control machine tool bearing detection method
CN108711148B (en) Tire defect intelligent detection method based on deep learning
CN111754498A (en) Conveyor belt carrier roller detection method based on YOLOv3
CN113724231B (en) Industrial defect detection method based on semantic segmentation and target detection fusion model
CN109284779A (en) Object detecting method based on the full convolutional network of depth
CN111444939A (en) Small-scale equipment component detection method based on weak supervision cooperative learning in open scene of power field
US11686689B2 (en) Automatic optimization of an examination recipe
CN111815564B (en) Method and device for detecting silk ingots and silk ingot sorting system
CN110648310A (en) Weak supervision casting defect identification method based on attention mechanism
CN113439276A (en) Defect classification in semiconductor samples based on machine learning
CN109543585A (en) Underwater optics object detection and recognition method based on convolutional neural networks
CN112365497A (en) High-speed target detection method and system based on Trident Net and Cascade-RCNN structures
CN111488920A (en) Bag opening position detection method based on deep learning target detection and recognition
JP2022027473A (en) Generation of training data usable for inspection of semiconductor sample
CN115147363A (en) Image defect detection and classification method and system based on deep learning algorithm
CN113591948A (en) Defect pattern recognition method and device, electronic equipment and storage medium
CN112819821A (en) Cell nucleus image detection method
TW202403603A (en) Computer implemented method for the detection of anomalies in an imaging dataset of a wafer, and systems making use of such methods
CN115984543A (en) Target detection algorithm based on infrared and visible light images
CN113221956A (en) Target identification method and device based on improved multi-scale depth model
CN115393265A (en) Cross-cloth flaw accurate detection method based on visual field adaptive learning
CN115830302B (en) Multi-scale feature extraction fusion power distribution network equipment positioning identification method
CN113808079B (en) Industrial product surface defect self-adaptive detection method based on deep learning model AGLNet
CN116912625A (en) Data enhancement method based on priori defect characteristics and SSPCAB attention mechanism

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant