CN115049627B - Steel surface defect detection method and system based on domain self-adaptive depth migration network - Google Patents
Steel surface defect detection method and system based on domain self-adaptive depth migration network Download PDFInfo
- Publication number
- CN115049627B CN115049627B CN202210739856.5A CN202210739856A CN115049627B CN 115049627 B CN115049627 B CN 115049627B CN 202210739856 A CN202210739856 A CN 202210739856A CN 115049627 B CN115049627 B CN 115049627B
- Authority
- CN
- China
- Prior art keywords
- domain
- encoder
- sample
- separation
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 52
- 229910000831 Steel Inorganic materials 0.000 title claims abstract description 47
- 239000010959 steel Substances 0.000 title claims abstract description 47
- 238000001514 detection method Methods 0.000 title claims abstract description 39
- 238000013508 migration Methods 0.000 title claims abstract description 31
- 230000005012 migration Effects 0.000 title claims abstract description 31
- 238000000926 separation method Methods 0.000 claims abstract description 99
- 230000006978 adaptation Effects 0.000 claims abstract description 64
- 238000000034 method Methods 0.000 claims abstract description 39
- 238000007781 pre-processing Methods 0.000 claims abstract description 18
- 238000012360 testing method Methods 0.000 claims abstract description 10
- 238000012549 training Methods 0.000 claims description 25
- 230000003044 adaptive effect Effects 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 238000005065 mining Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013526 transfer learning Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000009412 basement excavation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002994 raw material Substances 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30136—Metal
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention provides a steel surface defect detection method and system based on a domain self-adaptive deep migration network, wherein the method comprises the following steps: obtaining a typical defect image sample of the surface of the strip steel, and preprocessing the sample; constructing a reactance domain separation and self-adaptive network model; embedding the new sample characteristics into the shared characteristics of the source domain image samples, and calculating task classification loss and embedding classification loss; dynamically optimizing dynamic classification loss and dynamic adaptation loss by adding weights to the plurality of losses, and updating model parameters; and when the iteration times reach the optimal value, storing the model parameters, and inputting the target field test set to obtain the accuracy of steel surface defect detection in the target field. According to the invention, the self-adaptive mining sample hiding information and the dynamic weight optimizing loss algorithm are introduced on the basis of the anti-domain separation and self-adaptive deep migration network, so that the generalization capability of a network model is improved, and more accurate steel surface defect detection is finally realized.
Description
Technical Field
The invention belongs to the technical field of image detection, and particularly relates to a steel surface defect detection method and system based on a domain self-adaptive depth migration network.
Background
The steel is widely applied to fields of automobile manufacture, aerospace, daily necessities and the like, and is an indispensable important raw material in national economy development. With the rapid development of society, the quality requirements are more and more strict. During the production and processing, various defects such as holes, scratches and the like can be generated on the surface of the steel due to the influence of the factors such as environment, equipment performance, processing technology and the like. These observable defects can cause steel properties to change, greatly reducing product quality, and thus causing significant negative impact and economic loss to the manufacturing enterprise. Therefore, steel surface defect detection is widely focused as an important link for steel quality monitoring.
The primary aim of steel surface detection is to accurately predict the defect type, and the defects of high cost, low efficiency, subjective judgment and the like exist because manual detection is mainly adopted in the early stage. Therefore, the machine learning technology provides an efficient and objective image detection method, and the accuracy and efficiency of defect identification are improved to a certain extent. However, such conventional machine learning requires a professional to have a lot of experience and domain knowledge to extract more suitable features, and the detection performance is largely dependent on the selection of features. Unlike traditional machine learning, deep learning does not need to manually extract features, can avoid trial and error experiments of combining a large number of features with a classifier, and deeply and multi-angularly realizes the characterization of image features. However, in practical application, deep learning still has many problems to be solved, such as huge calculation amount, difficult obtaining of labeling samples, time and labor consuming training samples, and the like.
With the development of artificial intelligence technology, transfer learning becomes one of important research contents of the current surface defect detection method based on machine vision, and the goal of transfer learning is to transfer knowledge obtained by a source task to a target task to assist the target task to learn, so that the problems of insufficient sample size, low training efficiency and the like in deep learning are solved. However, direct migration will degrade model performance due to domain differences (feature distribution differences). The domain self-adaption can align the characteristic information of the source domain and the target domain through characteristic transformation, and solve the problem caused by domain difference. The depth domain self-adaptive learning breaks through the limitation of sample co-distribution in the traditional deep learning by transferring characteristic information, and accelerates the convergence rate of the model, but in practical application, the depth domain self-adaptive network still has the problems of weak image characteristic extraction capability, poor model stability, difficult convergence and the like. Therefore, there is a need to optimize the depth domain adaptive network to further improve the training performance of the model.
Disclosure of Invention
The embodiment of the invention provides a steel surface defect detection method and system based on a domain self-adaptive depth migration network, which are used for solving the problems of poor generalization capability of a detection model and even technical defects with lower recognition rate in the prior art.
The embodiment of the invention provides a steel surface defect detection method based on a domain self-adaptive deep migration network, which comprises the following steps:
s1: obtaining a typical defect image sample of the surface of the strip steel, and preprocessing the sample;
s2: constructing a reactance domain separation and self-adaptive network model according to the preprocessed sample;
s3: embedding new sample characteristics into shared characteristics of the source domain image samples obtained after preprocessing, inputting the shared characteristics into the opposite domain separation and self-adaptive network model, and calculating task classification loss and embedding classification loss;
s4: dynamically optimizing dynamic classification losses and dynamic adaptation losses by adding weights to the plurality of losses, wherein the dynamic classification losses include task classification losses and embedded classification losses, the dynamic adaptation losses include domain adaptation losses and domain separation losses, and updating parameters of the counterdomain separation and adaptation network model;
s5: judging whether the iteration number in the updating reaches the optimal iteration number, if so, executing the step S6, otherwise, returning to execute the step S3;
s6: and (3) storing the parameters, obtaining an optimized opposite domain separation and self-adaptive network model, and detecting a target domain sample test set to obtain the steel surface defect detection precision.
Preferably, the method for preprocessing the sample in step S1 is as follows:
firstly, dividing all image samples and unifying the sizes, and selecting N source domain image samples and N target domain image samples, wherein the source domain image samples and the target domain image samples comprise qualified image samples and defect image samples, and N is a positive integer;
then, dividing the source domain image sample and the target domain image sample into a training set and a testing set according to the same proportion;
and finally, inputting the source domain image sample into a depth extraction network model, and training the depth extraction network model to obtain trained model parameters.
Preferably, in the step S2, the method for constructing the anti-domain separation and adaptive network model according to the preprocessed sample includes:
firstly, inputting a training set of source domain image samples and target domain image samples into a plurality of encoder network models based on a depth convolutional neural network, and separating private parts of a source domain and a target domain and a shared part between the source domain and the target domain based on the encoder network models to realize domain information separation, wherein the encoder network models comprise a shared encoder, a source domain private encoder and a target domain private encoder network model;
then initializing the plurality of encoder network models by using the trained model parameters of the source domain image samples;
finally, the output of the initialized multiple encoder network models is input into a task classifier, a domain adaptation discriminator and a domain separation discriminator through a multi-layer full-connection network.
Preferably, the method of embedding the new sample feature into the shared feature of the source domain image sample obtained after the preprocessing in the step S3 is as follows:
the method comprises the steps of adaptively adjusting the inter-class distance of new sample characteristics according to the training state of the contrast domain separation and self-adaptive network model, and realizing the embedding of the new sample by adopting a space linear interpolation method, wherein the training state of the contrast domain separation and self-adaptive model is measured by the classification loss of a task classifier in the training process;
wherein the new sample characteristics are represented as follows:
wherein ,
for embedded new sample characteristics, the labels of the new sample characteristics correspond to corresponding heterogeneous sample labels, X is the same kind of sample characteristics, X - For heterogeneous sample characteristics, L task For task classification loss, lambda is a parameter for adjusting the distance between embedded new sample feature classes;
the new features are optimized, and the expression is as follows:
D E (X，X + )＝‖X，X + ‖ 2
D E (X，X + )<D E (X，X - )
wherein ,
for embedded new sample characteristics, the labels of the new sample characteristics correspond to corresponding heterogeneous sample labels, X is the same kind of sample characteristics, X - X is a heterogeneous sample feature + L is the original sample feature task For task classification loss, λ is a parameter that adjusts the distance between embedded new sample feature classes, D E (X，X + ) D is the distance between the same kind of samples E (X，X - ) Is the distance between the homogeneous sample and the heterogeneous sample.
Preferably, in the step S4, the dynamic classification loss and the dynamic adaptation loss are dynamically optimized by adding weights to a plurality of losses, and specifically includes:
the dynamic classification loss is a result of dynamically adjusting the weights of the task classification loss and the embedded classification loss, and is expressed as follows:
wherein ,Ldynamic-class For dynamic classification loss, L task For task classification loss, L embedded Classifying the loss for embedding;
the dynamic adaptation loss is a result of dynamically adjusting the weights of the domain adaptation loss and the domain separation loss, expressed as follows:
wherein ,Ldynamic-ad To dynamically adapt to losses, L adapt To accommodate loss of domain, L sep Loss for domain separation.
Preferably, the task classification loss L task Calculated from the cross entropy, the representation is as follows:
wherein ,Ctask For task classifier, en j In order to share the encoder with the encoder,
as a weight parameter of the task classifier,
to share the weight parameters of the encoder, x s Is a source domain image sample.
Preferably, the domain adaptation loss L adapt Obfuscating domain feature generation from a domain adaptation discriminator is represented as follows:
wherein ,Enj To share the encoder, D adapt For the domain adaptation discriminator,
in order to share the weight parameters of the encoder,
for adapting the weight parameters of the discriminator to the domain, x s For source domain image samples, x t For target domain image samples, E x Mathematical expectations of the image samples.
Preferably, the domain separation loss L sep The separation domain feature generation from the domain separation discriminator is represented as follows:
wherein ,Enj In order to share the encoder with the encoder,
for a source domain private encoder,>
private encoder for target domain, D sep For domain separation discriminator,>
for sharing the weight parameters of the encoder, +.>
The weight parameters for the source domain private encoder,
weight parameter for private encoder of target domain, < +.>
For the weight parameter of the domain separation discriminator, x s For source domain image samples, x t For target domain image samplesThe root, E x Mathematical expectations of the image samples.
Preferably, the method for updating the reactive domain separation and adaptive network model in step S4 is as follows:
the model parameters of the plurality of encoders, the domain adaptation discriminator and the domain separation discriminator are iteratively updated by back propagation through dynamic classification loss, domain adaptation loss and domain separation loss, comprising:
initializing a parameter theta:
wherein ,
adapting the weight parameter of the discriminator for the domain, < +.>
For the weight parameters of the domain separation discriminator,
weight parameters for task classifier, +.>
For sharing the weight parameters of the encoder, +.>
Weight parameter for private encoder of source domain, < ->
The weight parameters of the private encoder are the target domain;
the dynamic classification loss updates the network parameters as follows:
wherein ,Enj To share the encoder, C task In order to be a task classifier, the task classifier,
in order to share the weight parameters of the encoder,
for the weight parameters of the task classifier, L dynamic-class For dynamic classification loss, η is learning rate, ++>
Is a differential operator;
domain adaptation loss updates the network model parameters as follows:
wherein ,Dadapt For domain adaptation discriminator, en j In order to share the encoder with the encoder,
adapting the weight parameter of the discriminator for the domain, < +.>
To share the weight parameters of the encoder, L adapt For domain adaptation loss, η is learning rate, +.>
Is a differential operator;
the domain separation loss updates the network model parameters as follows:
wherein ,Enj In order to share the encoder with the encoder,
for a source domain private encoder,>
private encoder for target domain, D sep For domain separation discriminator,>
for sharing the weight parameters of the encoder, +.>
The weight parameters for the source domain private encoder,
weight parameter for private encoder of target domain, < +.>
For the weight parameter of the domain separation discriminator, L adapt To accommodate loss of domain, L sep For domain separation loss, η is learning rate, ++>
Is a differential operator.
The embodiment of the invention provides a steel surface defect detection system based on a domain self-adaptive deep migration network, which comprises the following components:
the sample pretreatment module is used for obtaining a typical defect image sample of the surface of the strip steel and carrying out pretreatment on the sample;
the network model building module is used for building a reactance domain separation and self-adaptive network model according to the preprocessed sample;
the optimized network model module is used for embedding new sample characteristics into the shared characteristics of the source domain image samples obtained after preprocessing, inputting the characteristics into the opposite domain separation and self-adaptive network model, and calculating task classification loss and embedding classification loss; dynamically optimizing dynamic classification losses and dynamic adaptation losses by adding weights to the plurality of losses, wherein the dynamic classification losses include task classification losses and embedded classification losses, the dynamic adaptation losses include domain adaptation losses and domain separation losses, and updating parameters of the counterdomain separation and adaptation network model; judging whether the iteration times in updating reach the optimal iteration times, if so, inputting an optimization result into a sample detection module, otherwise, continuing to perform iterative computation;
and the sample detection module is used for storing the parameters, obtaining an optimized opposite domain separation and self-adaptive network model, and detecting a sample test set in the target field to obtain the steel surface defect detection precision.
The system is used for realizing the steel surface defect detection method based on the domain self-adaptive depth migration network.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a steel surface defect detection method and system based on a domain self-adaptive depth migration network. The self-adaptive excavation of the sample hidden information can improve the convergence speed and the recognition precision in the network model training process, and the addition of the dynamic weight optimizing loss can enable the network model trained on the source field to perform well on the target field, so that the change of the network model in the self-adaptive field can be improved, the generalization capability of the network model can be improved, and finally, the more accurate steel surface defect detection can be realized.
Drawings
For a clearer description of embodiments of the invention or of solutions in the prior art, reference will be made below to the accompanying drawings, which are used in the embodiments and which are intended to illustrate, but not to limit, the invention, and from which other drawings can be obtained without inventive effort for a person skilled in the art. Wherein:
FIG. 1 is a flow chart of a method for detecting defects on a steel surface based on a domain-adaptive depth migration network according to an embodiment of the present invention;
FIG. 2 is a block diagram of an architecture based on a reactive domain separation and adaptive deep migration network model in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of the embedded new sample feature principle according to an embodiment of the present invention;
FIG. 4 is a block diagram of a steel surface defect detection system based on a domain adaptive depth migration network in accordance with an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
The embodiment of the invention provides a steel surface defect detection method based on a domain self-adaptive deep migration network, as shown in fig. 1, the method of the embodiment comprises the following steps:
s101: obtaining a typical defect image sample of the surface of the strip steel, and preprocessing the sample;
s102: constructing a reactance domain separation and self-adaptive network model according to the preprocessed sample;
s103: embedding new sample characteristics into shared characteristics of the source domain image samples obtained after preprocessing, inputting the shared characteristics into the opposite domain separation and self-adaptive network model, and calculating task classification loss and embedding classification loss;
s104: dynamically optimizing dynamic classification losses and dynamic adaptation losses by adding weights to the plurality of losses, wherein the dynamic classification losses include task classification losses and embedded classification losses, the dynamic adaptation losses include domain adaptation losses and domain separation losses, and updating parameters of the counterdomain separation and adaptation network model;
s105: judging whether the iteration number in the updating reaches the optimal iteration number, if so, executing the step S106, otherwise, returning to execute the step S103;
s106: and (3) storing the parameters, obtaining an optimized opposite domain separation and self-adaptive network model, and detecting a target domain sample test set to obtain the steel surface defect detection precision.
The invention provides a steel surface defect detection method based on a domain self-adaptive deep migration network, which is characterized in that a mechanism for separating a countering domain and evaluating the performance of a self-adaptive model based on classification loss is established; meanwhile, sample information hidden in a migration space is self-adaptively mined by using a space linear interpolation method, a classification result of a new mined sample is used as a main measurement index for measuring the contribution performance of the new mined sample to a network, and the contribution performance is used as a weight to be applied to classification loss, so that the influence of noise samples on a model is eliminated; in the countermeasure training process, the countermeasure loss and the smoothing network parameters are optimized by adding dynamic weights, so that the distinguishing performance of the model is improved. The invention can improve the training performance of the network model and greatly improve the accuracy of steel surface defect detection under the field deviation environment.
Further, the method for preprocessing the sample in step S101 is as follows:
firstly, all image samples are segmented and unified in size, and N source domain image samples { X } are selected s And N target domain image samples { X } t The source domain image sample and the target domain image sample comprise a qualified image sample and a defect image sample, and N is a positive integer;
then, dividing the source domain image sample and the target domain image sample into a training set and a testing set according to the same proportion;
and finally, inputting the source domain image sample into a depth extraction network model, and training the depth extraction network model to obtain trained model parameters.
Fig. 2 is a block diagram of a network model based on reactive domain separation and adaptive deep migration according to an embodiment of the present invention, and as shown in fig. 2, the network model is composed of three parts: a feature extraction section, a feature migration section, and a task classification section. Wherein the feature extraction section includes a shared encoder En j Source domain private encoder
And target domain private encoder->
The feature migration section includes a domain adaptation discriminator D adapt Sum domain separation discriminator D sep The task classification section includes a task classifier C task 。
The shared encoder En j Source domain private encoder
And target domain private encoder->
Feature extraction architecture (i.e., 5 convolutional layers) employing AlexNet networkAnd 3 full connection layers), extracting the shared features of the source domain and the target domain +.>
Respective private features->
The task classifier C task The method comprises the steps of forming a full connection layer for predicting task labels; the domain adaptation discriminator D adapt Consists of two fully connected layers for predicting the sharing feature +.>
Domain label of (D), domain separation discriminator D sep Also consisting of two fully connected layers for predictive feature labeling.
Further, the method for constructing the reactive domain separation and adaptive network model in step S102 is as follows:
first, the source domain image sample { X } s Sample { X } and target domain image t The training set is input into a plurality of encoder network models based on a depth convolutional neural network, the private parts of the source domain and the target domain and the shared part between the source domain and the target domain are separated based on the encoder network models, so as to realize domain information separation, and the encoder network models comprise a shared encoder En j Source domain private encoder
And target domain private encoder->
A network model;
then initializing the plurality of encoder network models by using the trained model parameters of the source domain image samples;
finally, the output of the initialized multiple encoder network models is input into a task classifier C through a multi-layer full-connection network task Domain adaptation discriminator D adapt Sum domain separation authenticationDevice D sep Is a kind of medium.
FIG. 3 is a schematic diagram of the feature principle of the embedded new sample according to the embodiment of the present invention, as shown in FIG. 3, X + Representing original sample characteristics, X representing similar sample characteristics
Representing embedded new sample features->
X - Representing heterogeneous sample characteristics.
Further, the method of embedding the new sample feature into the shared feature of the source domain image sample obtained after the preprocessing in step S103 is as follows:
the inter-class distance of the new sample characteristics is adaptively adjusted according to the training state of the contrast domain separation and self-adaptive network model, and the new sample is embedded by adopting a space linear interpolation method, wherein the training state of the contrast domain separation and self-adaptive model is obtained through a task classifier C in the training process task Is a measure of the classification loss of (2);
wherein the new sample characteristics are represented as follows:
wherein ,
to embed new sample features, the labels thereof correspond to corresponding heterogeneous sample labels, X is the same kind of sample features, X - For heterogeneous sample characteristics, L task For task classification loss, lambda is a parameter for adjusting the distance between embedded new sample feature classes;
to avoid embedded new sample features
The inter-class distance from the sample feature of the same class X is close to zero, for the new sample feature +.>
Optimization is performed, and the expression is as follows:
D E (X，X + )＝‖X，X + ‖ 2
D E (X，X + )<D E (X，X - )
wherein ,
for embedded new sample characteristics, the labels of the new sample characteristics correspond to corresponding heterogeneous sample labels, X is the same kind of sample characteristics, X - X is a heterogeneous sample feature + L is the original sample feature task For task classification loss, λ is a parameter that adjusts the distance between embedded new sample feature classes, D E (X，X + ) D is the distance between the same kind of samples E (X，X - ) Is the distance between the homogeneous sample and the heterogeneous sample.
Further, in step S104, by adding weights to the plurality of losses, dynamic classification losses and dynamic adaptation losses are dynamically optimized, specifically including:
the dynamic classification loss is a result of dynamically adjusting the weights of the task classification loss and the embedded classification loss, and is expressed as follows:
wherein ,Ldynamic-class For dynamic classification loss, L task For task classification loss, L embedded Classifying the loss for embedding;
the dynamic adaptation loss is a result of dynamically adjusting the weights of the domain adaptation loss and the domain separation loss, expressed as follows:
wherein ,Ldynamic-ad To dynamically adapt to losses, L adapt To accommodate loss of domain, L sep Loss for domain separation.
Further, the task classification loss L task Calculated from the cross entropy, the representation is as follows:
wherein ,Ctask For task classifier, en j In order to share the encoder with the encoder,
as a weight parameter of the task classifier,
to share the weight parameters of the encoder, x s Is a source domain image sample.
Further, the domain adaptation loss L adapt Obfuscating domain feature generation from a domain adaptation discriminator is represented as follows:
wherein ,Enj To share the encoder, D adapt For the domain adaptation discriminator,
in order to share the weight parameters of the encoder,
for adapting the weight parameters of the discriminator to the domain, x s For source domain image samples, x t For target domain image samples, E x Mathematical expectations of the image samples.
Further, the domain separation loss L sep The separation domain feature generation from the domain separation discriminator is represented as follows:
wherein ,Enj In order to share the encoder with the encoder,
for a source domain private encoder,>
private encoder for target domain, D sep For domain separation discriminator,>
for sharing the weight parameters of the encoder, +.>
The weight parameters for the source domain private encoder,
weight parameter for private encoder of target domain, < +.>
For the weight parameter of the domain separation discriminator, x s For source domain image samples, x t For target domain image samples, E x Mathematical expectations of the image samples.
Still further, the method for updating the parameters of the reactive domain separation and adaptive network model comprises the following steps:
the model parameters of the plurality of encoders, the domain adaptation discriminator and the domain separation discriminator are iteratively updated by back propagation through dynamic classification loss, domain adaptation loss and domain separation loss, comprising:
initializing a parameter theta:
wherein ,
adapting the weight parameter of the discriminator for the domain, < +.>
For the weight parameters of the domain separation discriminator,
weight parameters for task classifier, +.>
For sharing the weight parameters of the encoder, +.>
Weight parameter for private encoder of source domain, < ->
The weight parameters of the private encoder are the target domain;
the dynamic classification loss updates the network parameters as follows:
wherein ,Enj To share the encoder, C task In order to be a task classifier, the task classifier,
in order to share the weight parameters of the encoder,
for the weight parameters of the task classifier, L dynamic-class For dynamic classification loss, η is learning rate, ++>
Is a differential operator;
domain adaptation loss updates the network model parameters as follows:
wherein ,Dadapt For domain adaptation discriminator, en j In order to share the encoder with the encoder,
adapting the weight parameter of the discriminator for the domain, < +.>
To share the weight parameters of the encoder, L adapt For domain adaptation loss, η is learning rate, +.>
Is a differential operator;
the domain separation loss updates the network model parameters as follows:
wherein ,Enj In order to share the encoder with the encoder,
for a source domain private encoder,>
private encoder for target domain, D sep For domain separation discriminator,>
for sharing the weight parameters of the encoder, +.>
The weight parameters for the source domain private encoder,
weight parameter for private encoder of target domain, < +.>
For the weight parameter of the domain separation discriminator, L adapt To accommodate loss of domain, L sep For domain separation loss, η is learning rate, ++>
Is a differential operator.
Example two
The embodiment of the invention provides a steel surface defect detection system based on a domain self-adaptive deep migration network, which is shown in fig. 4 and comprises the following steps:
the sample preprocessing module 401 is used for acquiring a typical defect image sample of the surface of the strip steel and preprocessing the sample;
a construction network model module 402, configured to construct a reactive domain separation and adaptive network model according to the preprocessed sample;
the optimized network model module 403 is configured to embed new sample features into the shared features of the source domain image samples obtained after preprocessing, input the new sample features into the opposite domain separation and adaptive network model, and calculate task classification loss and embedding classification loss; dynamically optimizing dynamic classification losses and dynamic adaptation losses by adding weights to the plurality of losses, wherein the dynamic classification losses include task classification losses and embedded classification losses, the dynamic adaptation losses include domain adaptation losses and domain separation losses, and updating parameters of the counterdomain separation and adaptation network model; judging whether the iteration times in updating reach the optimal iteration times, if so, inputting an optimization result into a sample detection module, otherwise, continuing to perform iterative computation;
the sample detection module 404 is configured to store the parameters, obtain an optimized separation and adaptive network model of the reactance domain, and detect a sample test set in the target domain, thereby obtaining the detection accuracy of the steel surface defect.
The system is used for implementing the steel surface defect detection method based on the domain adaptive depth migration network in the first embodiment, and is not described herein.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It is apparent that the above examples are given by way of illustration only and are not limiting of the embodiments. Other variations and modifications will be apparent to persons skilled in the art from the foregoing description. It is not necessary here nor is it exhaustive of all embodiments. And obvious variations or modifications thereof are contemplated as falling within the scope of the present invention.
Claims (9)
1. A method for detecting defects on a steel surface based on a domain-adaptive deep migration network, comprising the steps of:
s1: obtaining a typical defect image sample of the surface of the strip steel, and preprocessing the sample;
s2: constructing a reactance domain separation and self-adaptive network model according to the preprocessed sample;
s3: embedding new sample characteristics into shared characteristics of the source domain image samples obtained after preprocessing, inputting the shared characteristics into the opposite domain separation and self-adaptive network model, and calculating task classification loss and embedding classification loss;
s4: dynamically optimizing dynamic classification losses and dynamic adaptation losses by adding weights to the plurality of losses, wherein the dynamic classification losses include task classification losses and embedded classification losses, the dynamic adaptation losses include domain adaptation losses and domain separation losses, and updating parameters of the counterdomain separation and adaptation network model;
s5: judging whether the iteration number in the updating reaches the optimal iteration number, if so, executing the step S6, otherwise, returning to execute the step S3;
s6: the parameters are stored, an optimized opposite domain separation and self-adaptive network model is obtained, a target domain sample test set is detected, and steel surface defect detection accuracy is obtained;
in the step S2, the method for constructing the reactance domain separation and self-adaptive network model according to the preprocessed sample comprises the following steps:
firstly, inputting a training set of source domain image samples and target domain image samples into a plurality of encoder network models based on a depth convolutional neural network, and separating private parts of a source domain and a target domain and a shared part between the source domain and the target domain based on the encoder network models to realize domain information separation, wherein the encoder network models comprise a shared encoder, a source domain private encoder and a target domain private encoder network model;
then initializing the plurality of encoder network models by using the trained model parameters of the source domain image samples;
finally, the output of the initialized multiple encoder network models is input into a task classifier, a domain adaptation discriminator and a domain separation discriminator through a multi-layer full-connection network.
2. The method for detecting the steel surface defects based on the domain-adaptive deep migration network according to claim 1, wherein the method for preprocessing the sample in the step S1 is as follows:
firstly, dividing all image samples and unifying the sizes, and selecting N source domain image samples and N target domain image samples, wherein the source domain image samples and the target domain image samples comprise qualified image samples and defect image samples, and N is a positive integer;
then, dividing the source domain image sample and the target domain image sample into a training set and a testing set according to the same proportion;
and finally, inputting the source domain image sample into a depth extraction network model, and training the depth extraction network model to obtain trained model parameters.
3. The method for detecting the defects on the steel surface based on the domain adaptive depth migration network according to claim 1, wherein the method for embedding the new sample features into the shared features of the source domain image samples obtained after the preprocessing in the step S3 is as follows:
the method comprises the steps of adaptively adjusting the inter-class distance of new sample characteristics according to the training state of the contrast domain separation and self-adaptive network model, and realizing the embedding of the new sample by adopting a space linear interpolation method, wherein the training state of the contrast domain separation and self-adaptive model is measured by the classification loss of a task classifier in the training process;
wherein the new sample characteristics are represented as follows:
wherein ,
for embedded newSample characteristics, the labels of which correspond to corresponding heterogeneous sample labels, X is the same kind of sample characteristics, X - For heterogeneous sample characteristics, L task For task classification loss, lambda is a parameter for adjusting the distance between embedded new sample feature classes;
optimizing the new sample characteristics, wherein the expression is as follows:
D E (X，X + )＝‖X，X + ‖ 2
D E (X，X + )<D E (X，X - )
wherein ,
for embedded new sample characteristics, the labels of the new sample characteristics correspond to corresponding heterogeneous sample labels, X is the same kind of sample characteristics, X - X is a heterogeneous sample feature + L is the original sample feature task For task classification loss, λ is a parameter that adjusts the distance between embedded new sample feature classes, D E (X，X + ) D is the distance between the same kind of samples E (X，X - ) Is the distance between the homogeneous sample and the heterogeneous sample.
4. The method for detecting the steel surface defects based on the domain adaptive depth migration network according to claim 1, wherein the step S4 dynamically optimizes the dynamic classification loss and the dynamic adaptation loss by adding weights to a plurality of losses, specifically comprises:
the dynamic classification loss is a result of dynamically adjusting the weights of the task classification loss and the embedded classification loss, and is expressed as follows:
wherein ,Ldynamic-class For dynamic classification loss, L task For task classification loss, L embedded Classifying the loss for embedding;
the dynamic adaptation loss is a result of dynamically adjusting the weights of the domain adaptation loss and the domain separation loss, expressed as follows:
wherein ,Ldynamic-ad To dynamically adapt to losses, L adapt To accommodate loss of domain, L sep Loss for domain separation.
5. The method for detecting steel surface defects based on domain-adaptive deep migration network of claim 4, wherein the task classification loss L task Calculated from the cross entropy, the representation is as follows:
wherein ,Ctask For task classifier, en j In order to share the encoder with the encoder,
weight parameters for task classifier, +.>
To share the weight parameters of the encoder, x s Is a source domain image sample.
6. The method for detecting steel surface defects based on domain-adaptive depth migration network of claim 4, wherein the domain adaptation loss L adapt Obfuscating domain feature generation from a domain adaptation discriminator is represented as follows:
wherein ,Enj To share the encoder, D adapt For the domain adaptation discriminator,
in order to share the weight parameters of the encoder,
for adapting the weight parameters of the discriminator to the domain, x s For source domain image samples, x t For target domain image samples, E x Mathematical expectations of the image samples.
7. The method for detecting steel surface defects based on domain-adaptive deep migration network of claim 4, wherein the domain separation loss L sep The separation domain feature generation from the domain separation discriminator is represented as follows:
wherein ,Enj In order to share the encoder with the encoder,
for a source domain private encoder,>
private encoder for target domain, D sep For domain separation discriminator,>
for sharing the weight parameters of the encoder, +.>
Weight parameter for private encoder of source domain, < ->
Weight parameter for private encoder of target domain, < +.>
For the weight parameter of the domain separation discriminator, x s For source domain image samples, x t For target domain image samples, E x Mathematical expectations of the image samples.
8. The method for detecting steel surface defects based on domain adaptive deep migration network according to claim 1 or 4, wherein the method for updating the parameters of the contrast domain separation and adaptive network model in step S4 is as follows:
the model parameters of the plurality of encoders, the domain adaptation discriminator and the domain separation discriminator are iteratively updated by back propagation through dynamic classification loss, domain adaptation loss and domain separation loss, comprising:
initializing a parameter theta:
wherein ,
adapting the weight parameter of the discriminator for the domain, < +.>
Weight parameter for domain separation discriminator, < ->
Weight parameters for task classifier, +.>
In order to share the weight parameters of the encoder,/>
weight parameter for private encoder of source domain, < ->
The weight parameters of the private encoder are the target domain;
the dynamic classification loss updates the network parameters as follows:
wherein ,Enj To share the encoder, C task In order to be a task classifier, the task classifier,
for sharing the weight parameters of the encoder, +.>
For the weight parameters of the task classifier, L dynamic-class For dynamic classification loss, η is learning rate, ++>
Is a differential operator;
domain adaptation loss updates the network model parameters as follows:
wherein ,Dadapt Adapting to domainDiscriminator, en j In order to share the encoder with the encoder,
for the domain adaptation discriminator weight parameters,
to share the weight parameters of the encoder, L adapt For domain adaptation loss, η is learning rate, +.>
Is a differential operator;
the domain separation loss updates the network model parameters as follows:
wherein ,Enj In order to share the encoder with the encoder,
for a source domain private encoder,>
private encoder for target domain, D sep For domain separation discriminator,>
encoding for sharingWeight parameter of the device, < >>
Weight parameter for private encoder of source domain, < ->
Weight parameter for private encoder of target domain, < +.>
For the weight parameter of the domain separation discriminator, L adapt To accommodate loss of domain, L sep For domain separation loss, η is learning rate, ++>
Is a differential operator.
9. A steel surface defect detection system based on a domain adaptive depth migration network, comprising:
the sample pretreatment module is used for obtaining a typical defect image sample of the surface of the strip steel and carrying out pretreatment on the sample;
the network model building module is used for building a reactance domain separation and self-adaptive network model according to the preprocessed sample;
the optimized network model module is used for embedding new sample characteristics into the shared characteristics of the source domain image samples obtained after preprocessing, inputting the characteristics into the opposite domain separation and self-adaptive network model, and calculating task classification loss and embedding classification loss; dynamically optimizing dynamic classification losses and dynamic adaptation losses by adding weights to the plurality of losses, wherein the dynamic classification losses include task classification losses and embedded classification losses, the dynamic adaptation losses include domain adaptation losses and domain separation losses, and updating parameters of the counterdomain separation and adaptation network model; judging whether the iteration times in updating reach the optimal iteration times, if so, inputting an optimization result into a sample detection module, otherwise, continuing to perform iterative computation;
the sample detection module is used for storing the parameters, obtaining an optimized opposite domain separation and self-adaptive network model, and detecting a sample test set in the target field to obtain the steel surface defect detection precision;
the method for constructing the opposite domain separation and self-adaptive network model according to the preprocessed sample in the network model constructing module comprises the following steps:
firstly, inputting a training set of source domain image samples and target domain image samples into a plurality of encoder network models based on a depth convolutional neural network, and separating private parts of a source domain and a target domain and a shared part between the source domain and the target domain based on the encoder network models to realize domain information separation, wherein the encoder network models comprise a shared encoder, a source domain private encoder and a target domain private encoder network model;
then initializing the plurality of encoder network models by using the trained model parameters of the source domain image samples;
finally, the output of the initialized multiple encoder network models is input into a task classifier, a domain adaptation discriminator and a domain separation discriminator through a multi-layer full-connection network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210739856.5A CN115049627B (en) | 2022-06-21 | 2022-06-21 | Steel surface defect detection method and system based on domain self-adaptive depth migration network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210739856.5A CN115049627B (en) | 2022-06-21 | 2022-06-21 | Steel surface defect detection method and system based on domain self-adaptive depth migration network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115049627A CN115049627A (en) | 2022-09-13 |
| CN115049627B true CN115049627B (en) | 2023-06-20 |
Family
ID=83164015
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210739856.5A Active CN115049627B (en) | 2022-06-21 | 2022-06-21 | Steel surface defect detection method and system based on domain self-adaptive depth migration network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115049627B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116883735B (en) * | 2023-07-05 | 2024-03-08 | 江南大学 | Domain self-adaptive wheat seed classification method based on public features and private features |
| CN117892203A (en) * | 2024-03-14 | 2024-04-16 | 江南大学 | Defective gear classification method, device and computer readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110880175A (en) * | 2019-11-15 | 2020-03-13 | 广东工业大学 | Welding spot defect detection method, system and equipment |
| CN113065581A (en) * | 2021-03-18 | 2021-07-02 | 重庆大学 | Vibration fault migration diagnosis method for reactance domain adaptive network based on parameter sharing |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902798A (en) * | 2018-05-31 | 2019-06-18 | 华为技术有限公司 | The training method and device of deep neural network |
| CN111739076B (en) * | 2020-06-15 | 2022-09-30 | 大连理工大学 | Unsupervised content protection domain adaptation method for multiple CT lung texture recognition |
-
2022
- 2022-06-21 CN CN202210739856.5A patent/CN115049627B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110880175A (en) * | 2019-11-15 | 2020-03-13 | 广东工业大学 | Welding spot defect detection method, system and equipment |
| CN113065581A (en) * | 2021-03-18 | 2021-07-02 | 重庆大学 | Vibration fault migration diagnosis method for reactance domain adaptive network based on parameter sharing |
Non-Patent Citations (2)
| Title |
|---|
| 基于鉴别模型和对抗损失的无监督域自适应方法;赵文仓;袁立镇;徐长凯;;高技术通讯(第07期);全文 * |
| 采用机器视觉与自适应卷积神经网络检测花生仁品质;张思雨;张秋菊;李可;;农业工程学报(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115049627A (en) | 2022-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115049627B (en) | Steel surface defect detection method and system based on domain self-adaptive depth migration network | |
| CN107169956B (en) | Color woven fabric defect detection method based on convolutional neural network | |
| CN106528417A (en) | Intelligent detection method and system of software defects | |
| CN110751186B (en) | Cross-project software defect prediction method based on supervised expression learning | |
| CN108549817A (en) | A kind of software security flaw prediction technique based on text deep learning | |
| CN116448419A (en) | Zero sample bearing fault diagnosis method based on depth model high-dimensional parameter multi-target efficient optimization | |
| CN108334943A (en) | The semi-supervised soft-measuring modeling method of industrial process based on Active Learning neural network model | |
| CN103226728A (en) | Intelligent detection and yield optimization method for HDPE (high density polyethylene) cascade polymerization reaction course | |
| CN116934687B (en) | Injection molding product surface defect detection method based on semi-supervised learning semantic segmentation | |
| CN117076921A (en) | Prediction method of logging-while-drilling resistivity curve based on residual fully-connected network | |
| CN112241808A (en) | Road surface technical condition prediction method, device, electronic equipment and storage medium | |
| CN116680639A (en) | Deep-learning-based anomaly detection method for sensor data of deep-sea submersible | |
| CN111460367A (en) | Algorithm for solving unbalanced data leakage of halogen conveying pipeline based on S transformation/WGAN | |
| CN115619028A (en) | Clustering algorithm fusion-based power load accurate prediction method | |
| CN115358473A (en) | Power load prediction method and prediction system based on deep learning | |
| CN114970813A (en) | Dissolved oxygen concentration data restoration and prediction method | |
| CN114492174A (en) | Full life cycle shield tunneling parameter prediction method and device | |
| CN114266337A (en) | Intelligent tobacco leaf grading model based on residual error network and grading method using model | |
| CN113592028A (en) | Method and system for identifying logging fluid by using multi-expert classification committee machine | |
| Mikheev et al. | Problems of using neural networks | |
| CN112348275A (en) | Regional ecological environment change prediction method based on online incremental learning | |
| CN111046612B (en) | Blast furnace molten iron silicon content prediction method based on multi-target integrated learning | |
| CN115293462B (en) | Method for predicting size range of leakage channel based on deep learning | |
| CN115982660A (en) | Double-source checking broadband transaction identification system and detection method | |
| Chen | Brain Tumor Prediction with LSTM Method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |