CN117314892A - Incremental learning-based continuous optimization method for solar cell defect detection - Google Patents
Incremental learning-based continuous optimization method for solar cell defect detection Download PDFInfo
- Publication number
- CN117314892A CN117314892A CN202311584711.3A CN202311584711A CN117314892A CN 117314892 A CN117314892 A CN 117314892A CN 202311584711 A CN202311584711 A CN 202311584711A CN 117314892 A CN117314892 A CN 117314892A
- Authority
- CN
- China
- Prior art keywords
- defect
- representing
- unknown
- follows
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000007547 defect Effects 0.000 title claims abstract description 136
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 238000005457 optimization Methods 0.000 title claims abstract description 32
- 238000012549 training Methods 0.000 claims abstract description 44
- 230000008569 process Effects 0.000 claims abstract description 29
- 238000013138 pruning Methods 0.000 claims abstract description 24
- 230000004927 fusion Effects 0.000 claims abstract description 18
- 238000004364 calculation method Methods 0.000 claims abstract description 12
- 238000011478 gradient descent method Methods 0.000 claims abstract description 10
- 238000010276 construction Methods 0.000 claims description 12
- 238000007670 refining Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 6
- 238000012804 iterative process Methods 0.000 claims description 6
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 238000010200 validation analysis Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 4
- 238000003062 neural network model Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000011895 specific detection Methods 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims description 2
- 238000007906 compression Methods 0.000 claims description 2
- 238000003860 storage Methods 0.000 abstract description 5
- 206010027175 memory impairment Diseases 0.000 abstract 1
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02E—REDUCTION OF GREENHOUSE GAS [GHG] EMISSIONS, RELATED TO ENERGY GENERATION, TRANSMISSION OR DISTRIBUTION
- Y02E10/00—Energy generation through renewable energy sources
- Y02E10/50—Photovoltaic [PV] energy
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Photovoltaic Devices (AREA)
Abstract
The invention relates to the technical field of computer vision, in particular to a solar cell defect detection continuous optimization method based on incremental learning, which constructs a defect detection model and continuously optimizes and updates the defect detection model, and specifically comprises the following steps: continuously inputting defect data, updating a feature extractor, an auxiliary classifier, a feature fusion device and a classifier, performing incremental training by using a gradient descent method, constructing an unknown defect type example set, adjusting a known defect type example set, pruning the feature extractor based on the geometric center of the feature extractor, and realizing defect detection which greatly reduces disastrous forgetfulness and continuously optimizes new and old task detection. The method has the advantages that the defect detection process can be started by a small amount of class data sets, and the defect detection model is ensured to be continuously and optimally updated only by storing a small-scale sample set and an input training set of an old data set at each stage, so that the storage and calculation load is reduced.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a solar cell defect detection continuous optimization method based on incremental learning.
Background
Under the promotion of new energy and energy saving policies, the solar cell is applied and produced in a large scale. However, when the manufacturing process fails, the produced battery has a certain defect, the defect occurring in the production process is detected and analyzed, and the corresponding failure process is adjusted in time, so that the loss can be reduced, and therefore, the defect type of the battery piece is accurately detected and found in real time, and the corresponding failure process is very necessary to be adjusted.
In the process of mass production of solar cells, defect data are collected, the defect data are distributed in long tails, the number of the categories of partial categories (head defect categories) is small, but the sample size of each category is large, the number of the samples of each category is large, the tail defect categories are opposite, so that an equilibrium data set containing all defect categories is difficult to obtain at one time, only a small number of defect category data sets can be obtained, and the rest defect categories cannot be learned at one time temporarily; the defect expression form may change due to the influence of actual production factors, so that the known defect category learned by the model has a new characteristic value; adjustments to the manufacturing process may also lead to new defect categories. Therefore, a small amount of defect type data sets can be used as a basic type training set to train a basic neural network model, and continuously and stage-by-stage train learning of continuously arriving unknown defect type data and known defect types containing new characteristic values on the basis of retaining old knowledge so as to continuously optimize the detection capability.
The current intelligent defect detection of solar cells still adopts a traditional offline learning mode, which is necessary to rely on a complete data set in a training stage, and only one model can be trained from scratch, and the training is long. The offline learning mode cannot adapt to data change in time when learning continuously arriving new data and continuously optimizing a model, so that the offline learning mode is applied to the current large-scale, high-dimensionality and continuously changing data set learning scene, and the problems of update lag, high calculation resource consumption and the like are faced.
In summary, in order to solve the problems of large consumption of computational resources and update lag in model continuous optimization performed by the current offline learning method, a continuous optimization method for detecting defects of a solar cell based on incremental learning is urgently needed.
Disclosure of Invention
The invention aims to provide a continuous optimization method for detecting defects of a solar cell based on incremental learning, which comprises the following steps of:
a solar cell defect detection continuous optimization method based on incremental learning comprises the following steps:
step S1, defect data is continuously input:
inputting an example set and defect data, wherein the example set is a subset of an old data set, the defect data comprises unknown class data and known class data containing new characteristic values, and a union set of the example set and the defect data is a training set;
step S2, expanding a feature extractor:
constructing a new feature extractor, and initializing the new feature extractor by using parameters of the feature extractor in the old model to obtain a new feature extractor;
step S3, constructing an auxiliary classifier:
constructing an auxiliary classifier, treating all known class data as a class, and calculating auxiliary loss of related new features;
step S4, constructing a feature fusion device:
extracting features of the solar cell image from a plurality of scales through a newly added feature extractor, connecting all the extracted features to obtain an intermediate feature map, refining the intermediate feature map based on a channel and a spatial attention mechanism, inputting the intermediate feature map, sequentially deducing a one-dimensional attention force map and a two-dimensional attention force map, and obtaining a refined feature map through a feature fusion device;
step S5, updating a classifier:
copying parameters for old features from old model classifiers to construct a current stage classifier, adding corresponding full-connection layer nodes according to the number of unknown defect categories in the current stage, randomly initializing new parameters, sending a refined feature map to the current stage classifier to obtain prediction probability, and calculating cross entropy loss;
step S6, performing incremental training by using a random gradient descent method:
performing incremental training on the old model based on a training set, learning an unknown class sample and a known class sample containing a new characteristic value by using a random gradient descent method according to the global loss containing classification loss and auxiliary loss, and converting the unknown class sample into the known class sample to obtain an updated new model;
step S7, the unknown defect class example set construction and the known defect class example set adjustment:
training unknown defect type data and known defect type data based on a new model to obtain an unknown type example set and a known type example set with new characteristics, wherein the unknown type example set, the known type example set and the example set in the step S1 form a new example set for incremental training of the next stage;
step S8, pruning by the feature extractor:
calculating the geometric center of each layer of the newly added feature extractor in the new model, pruning the convolution kernel of the layer based on the geometric center;
step S9, realizing continuous optimization updating of the defect detection model:
continuously inputting unknown class defect data and known class defect data, repeating the steps S1 to S8, continuously updating the old model to obtain a defect detection network which is globally unified for the unknown class and the known class, and realizing continuous optimization of defect detection.
Preferably, in step S1, for continuously inputted defect dataPerforming optimized detection, wherein->When (when)When the model to be updated is the basic neural network model +.>，/>Training set ∈>Training to obtain the product.
Preferably, in step S2, the feature extractor is built based on a multi-scale residual network.
Preferably, in step S3, the auxiliary classifier uses probabilitiesThe prediction classification is performed with the following expression:
；
wherein,representation->A function; />Representing an auxiliary classifier whose label space is，/>A set of tags representing unknown categories; />Representing the new feature;
auxiliary loss on new featuresThe expression of (2) is as follows:
；
wherein,representing a training set; />Representing an image; />Representing a corresponding category label.
Preferably, in step S4, the channel and spatial attention mechanisms are specifically a channel attention sub-network and a spatial attention sub-network, and the specific process of obtaining the refined feature map through the feature fusion device is as follows:
channel attention subnetwork pass-throughAnd->Compression intermediate feature map->Is>Is +.>Two eigenvectors are generated->And->The dimensions are->Connecting the two eigenvectors to get +.>Two pairs of full convolution layers are used>The process is performed as follows:
；
；
wherein,representing the weight of each channel; />Representing an activation function; first full convolution layerThe number of output channels of (a) is +.>，/>Representing a reduction ratio; second full convolution layer->Is of convolution kernel size +.>The number of output channels is +.>；/>Representing channel characteristics;
the spatial attention sub-network takes as input the channel characteristics, applies the average and maximum operations along the channel axis to generate two feature maps, expressed as follows:
；
；
wherein,and->Representing the element average and maximum operations in the channel axis, respectively;
based onAnd->Obtain dimension +.>Is striving for->The expression is as follows:
；
the spatial attention map is multiplied by the channel characteristics to obtain a refined characteristic map, and the expression is as follows:
；
wherein,representing spatially refined features, i.e. refining feature map +.>。
Preferably, in step S5, the probability is predictedThe expression of (2) is as follows:
；
wherein,representing a classifier;
cross entropy lossThe expression of (2) is as follows:
；
wherein,representing a training set; />Representing an image; />Representing a corresponding category label.
Preferably, in step S6, global lossThe expression of (2) is as follows:
；
wherein,super parameters representing the effect of controlling the auxiliary classifier, for the initial model, i.e. +.>In the time of (1),。
preferably, in step S7, the exemplary set updating manner is specifically as follows:
building an unknown class exemplar setFor unknown class data set->Random sampling to obtain an initial unknown class exemplar set +.>Use of the model->Initializing intermediate model->And at->Upper trainingPerforming several iterations by random gradient descent to obtain +.>The iterative process is as follows:
；
wherein,representing fine tuning->Is->Representing cross entropy loss, < >>Representation->Is a gradient of (2);
calculation ofUpper->And back-propagating the validation loss to optimize +.>Obtaining a final sample set of unknown classes +.>The propagation process is as follows:
；
wherein,learning rate representing unknown class exemplar set propagation process, < >>Representing cross entropy loss, < >>Representation->Is a gradient of (2);
adjusting a set of known class exemplarsFor the known class dataset +.>Randomly sampling to obtain->The method comprises the steps of carrying out a first treatment on the surface of the Second, use +.>Initializing intermediate model->And at->Upper training->Performing several iterations by random gradient descent to obtain +.>The iterative process is as follows:
；
calculation ofUpper->And back-propagating the validation loss to optimize +.>Obtaining the final new characteristic valueIs->The propagation process is as follows:
；
wherein,representing the learning rate of the known sample set propagation process.
Preferably, in step S8, the geometric centerIs represented as follows:
；
；
wherein,representation->First->First of all convolution layers>A convolution kernel with dimension +>，/>And->Respectively represent +.>The number of input channels and the number of output channels of the convolution layers;
when (when)First->Geometric center of the convolutional layers->Is located at->In the convolution kernels of the convolution layers, the following formula is satisfied:
；
wherein,indicate->The convolution layers have geometric centers in the layer +.>Convolution kernels of the same value, i.e. requiring pruning, +.>Representation->And->A second norm of the difference;
the first is calculated by a formulaThe convolution kernel in each convolution layer requiring pruning is thus realPruning of the current model is carried out, and the expression is as follows:
；
wherein,indicate->The convolution layers have geometric centers in the layer +.>Convolution kernels of the same or similar value, i.e. requiring pruning, +.>Representation->And->And the two norms of the difference.
Preferably, in step S9, unknown and known defect class data are continuously input, and based on the old model, feature extractor expansion, auxiliary classifier construction, feature fusion construction, classifier updating, incremental training by using a random gradient descent method, unknown defect class example set construction and known defect class example set adjustment, and feature extractor pruning steps are sequentially performed to obtain a defect detection network globally unified for the current unknown and known classesThe continuous optimization of defect detection is realized;
the specific detection process is as follows:
defect detection networkFor all received solar cell images to be detected +.>Performing defect detection; first, based on feature extractor->The extracted features are connected to obtain intermediate feature map +.>，
；
Then, according to step S3, the intermediate feature map is passed through a feature fusion deviceTransformation into a refining profile->，
；
；
Will refine the feature mapSend to classifier->The defect type label of the solar cell is predicted to be +.>And according to the defect category, the defect category is obtained, and the expression is as follows:
；
。
the technical scheme of the invention has the following beneficial effects:
according to the incremental learning-based solar cell defect detection continuous optimization method disclosed by the invention, the network learning process can be started by adopting the initial data set, and the storage and calculation loads are effectively reduced by only storing the old data set minimum-scale example set and the input training set at each stage. In the network learning process, unknown class samples are continuously learned, and accurate detection of all defect classes is realized. When learning unknown defect categories, the network effectively reduces the catastrophic forgetting of the known defect categories. And the updates required for the detection of known defect categories are implemented for new manifestations in which known defect categories may occur.
According to the invention, for the occurrence of an unknown class, in order to balance the stability and plasticity of a model, a new feature extractor is created when the unknown class occurs, and the parameters of the old feature extractor are used for initialization, so that the knowledge of the known class is reserved, and the characteristics of the unknown class can be effectively learned. The auxiliary classifier construction is then used to train the newly constructed feature extractor. The feature fusion device effectively solves the problem of confusion possibly generated between the known category and the unknown category in the incremental learning.
In addition, the present invention not only builds the sample set for the occurrence of the unknown class, but also considers the known class with new feature values, dynamically builds the sample set of the unknown class and adjusts the sample set of the known class when the occurrence of the unknown class and the change of the known class. The invention also provides a feature extractor pruning, which can realize great parameter reduction while maintaining the performance of the model as much as possible, reduce the storage and calculation requirements of the model and improve the practicability of the model.
In addition to the objects, features and advantages described above, the present invention has other objects, features and advantages. The present invention will be described in further detail with reference to the drawings.
Drawings
For a clearer description of embodiments of the invention or of the prior art, the drawings that are used in the description of the embodiments or of the prior art will be briefly described, it being apparent that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from them without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of steps of a method for continuous optimization of defect detection of a solar cell in a preferred embodiment of the invention;
FIG. 2 is a flow chart of updating of an unknown class example set and a known class example set in a preferred embodiment of the present invention.
Detailed Description
The invention provides a continuous optimization method for detecting defects of a solar cell based on incremental learning. In the prior art, intelligent defect detection of solar cells still adopts a traditional offline learning mode, which is necessary to rely on a complete data set in a training stage, and only one model can be trained from scratch, and the training is time-consuming. Therefore, the offline learning mode cannot adapt to data change in time when learning continuously arriving new data and performing model continuous optimization, and therefore the offline learning mode is applied to the current large-scale, high-dimensionality and continuously changing data set processing scene, and the problems of update lag, high calculation resource consumption and the like are faced.
According to the incremental learning-based solar cell defect detection continuous optimization method disclosed by the invention, the network learning process can be started by adopting the initial data set, and the storage and calculation loads are effectively reduced by only storing the old data set minimum-scale example set and the input training set at each stage. In the network learning process, unknown class samples are continuously learned, and accurate detection of all defect classes is realized. When learning unknown defect categories, the network effectively reduces the catastrophic forgetting of the known defect categories. And the updates required for the detection of known defect categories are implemented for new manifestations in which known defect categories may occur.
In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Examples:
referring to fig. 1, fig. 1 is a flowchart illustrating steps of a method for continuously optimizing defect detection of a solar cell according to a preferred embodiment of the present invention, wherein steps S1 to S8 are updating models based on features in old models and known class data, and step S9 is continuously updating the old models, so as to implement a model continuously optimizing updated defect detection network of the solar cell.
Referring to fig. 1, in an embodiment of the present invention, a method for continuously optimizing defect detection of a solar cell includes the steps of:
step S1, defect data is continuously input:
inputting an example set of solar cell data and defect data, wherein the example set is a subset of an old data set (the old data set is defect data used for training in an old model), the defect data comprises unknown class data and known class data containing new characteristic values, and a union set of the example set and the defect data is a training set; for continuously input defect dataPerforming a staged optimization test, wherein->When->When the model to be updated is the basic neural network model +.>，/>Training set ∈>Training to obtain the product.
Step S2, expanding a feature extractor:
constructing a new feature extractor, and initializing the new feature extractor by using parameters of all the existing feature extractors to obtain a new feature extractor; the feature extractor is built based on a multi-scale residual network (acceptance-ResNet), which is state of the art and is not described here in detail. In this embodiment, to avoid catastrophic forgetting of old knowledge and to preserve knowledge that has been learned before, all feature extractors are frozen currentlyIs a parameter of (a). Then, in order to more effectively adapt to the feature distribution of the unknown class and thus more accurately detect the unknown defect class, a new feature extractor is constructed>And copy->To initialize +.>。
Step S3, constructing an auxiliary classifier:
constructing an auxiliary classifier, treating all known class data as a class, and calculating auxiliary loss of related new features; the auxiliary classifier uses probabilityThe prediction classification is performed with the following expression:
；
wherein,representation->A function; />Representing an auxiliary classifier whose label space is，/>A set of tags representing unknown categories; />Representing the new feature;
auxiliary loss on new featuresThe expression of (2) is as follows:
；
wherein,representing a training set; />Representing an image; />Representing a corresponding category label.
Step S4, constructing a feature fusion device:
extracting features of the solar cell image from a plurality of scales through a newly added feature extractor, connecting all the extracted features to obtain an intermediate feature map, inputting the intermediate feature map based on a channel and a spatial attention mechanism, sequentially deducing a one-dimensional attention force map and a two-dimensional attention force map by a feature fusion device, and obtaining a refined feature map through the feature fusion device;
in this embodiment, the channel and spatial attention mechanism is specifically a channel attention sub-network and a spatial attention sub-network, and the specific process of obtaining the refined feature map through the feature fusion device is as follows:
channel attention subnetwork pass-through(Global average pooling) and +.>(Global maximization) compressed intermediate feature map +.>Is>Is +.>(C represents the number of channels of the feature map; H represents the height of the feature map; W represents the width of the feature map), two feature vectors are generated +.>And->The dimensions are allConnecting the two eigenvectors to get +.>Two pairs of full convolution layers are used>The process is performed as follows:
；
；
wherein,representing the weight of each channel; />Representing an activation function; first full convolution layerThe number of output channels of (a) is +.>，/>Representing a reduction ratio; second full convolution layer->Is of convolution kernel size +.>The number of output channels is +.>；/>Representing channel characteristics;
the spatial attention sub-network takes as input the channel characteristics, applies the average and maximum operations along the channel axis to generate two feature maps, expressed as follows:
；
；
wherein,and->Representing the element average and maximum operations in the channel axis, respectively;
based onAnd->Obtain dimension +.>Is striving for->The expression is as follows:
；
the spatial attention map is multiplied by the channel characteristics to obtain a refined characteristic map, and the expression is as follows:
；
wherein,representing spatially refined features, i.e. refining feature map +.>。
Step S5, updating a classifier:
copying parameters for old features from old model classifiers to construct new models, adding corresponding full-connection layer nodes according to the number of unknown defect categories in the old models, randomly initializing the new parameters, sending a refined feature map to the classifiers to obtain prediction probability, and calculating cross entropy loss;
specifically, the probability of predictionThe expression of (2) is as follows:
；
wherein,representing a classifier;
cross entropy lossThe expression of (2) is as follows:
。
step S6, performing incremental training by using a random gradient descent method:
performing incremental training on the old model based on a training set, learning an unknown class sample and a known class sample containing a new characteristic value by using a random gradient descent method according to the global loss containing classification loss and auxiliary loss, and converting the unknown class sample into the known class sample to obtain an updated new model;
specifically, in step S6, global lossThe expression of (2) is as follows:
；
wherein,super parameters representing the effect of controlling the auxiliary classifier, for the initial model, i.e. +.>In the time of (1),。
step S7, the unknown defect class example set construction and the known defect class example set adjustment:
training unknown defect type data and known defect type data based on the new model to obtain an unknown type example set and a known type example set with new characteristics, wherein the unknown type example set, the known type example set and the example set in the step S1 form a new example set for incremental training of the new model;
specifically, as shown in fig. 2, the exemplary set update method is specifically as follows:
building an unknown class exemplar setFor unknown class data set->Random sampling to obtain unknown class sample set +.>Use of the model->Initializing intermediate model->And at->Upper training->Performing several iterations by random gradient descent to obtain +.>The iterative process is as follows:
；
wherein,representing fine tuning->Is a learning rate of (a);
calculation ofUpper->And back-propagating the verification loss to optimize +.>Obtaining a final sample set of unknown classes +.>The propagation process is as follows:
；
wherein,representing a learning rate of the unknown class example set propagation process;
adjusting a set of known class exemplarsKnown class data set for new model +.>Randomly sampling to obtain->. Second, use +.>Initializing to obtain an intermediate model->And at->Upper training->Performing several iterations by random gradient descent to obtain +.>The iterative process is as follows:
；
calculation ofUpper->And back-propagating the validation loss to optimize +.>Obtaining the final set of known class exemplars with new feature values +.>The propagation process is as follows:
；
wherein,representing the learning rate of the known sample set propagation process.
Step S8, pruning by the feature extractor:
calculating the geometric center of each layer of the newly added feature extractor in the new model, pruning the convolution kernel of the layer based on the geometric center;
in particular, the geometric centerIs represented as follows:
；
；
wherein,representation->First->First of all convolution layers>A convolution kernel with dimension +>，/>And->Respectively represent +.>The number of input channels and the number of output channels of the convolution layers;
when (when)First->Geometric center of the convolutional layers->Is located at->In the convolution kernels of the convolution layers, the following formula is satisfied:
；
wherein,indicate->The convolution layers have geometric centers in the layer +.>Convolution kernels of the same value, i.e. requiring pruning, +.>Representation->And->A second norm of the difference;
the first is calculated by a formulaThe convolution kernels of pruning are needed in the convolution layers, so that model pruning is realized, and the expression is as follows:
；
wherein,indicate->The convolution layers have geometric centers in the layer +.>Convolution kernels of the same or similar value, i.e. requiring pruning, +.>Representation->And->And the two norms of the difference.
It should be noted that if the values of certain convolution kernels are close to or equal to the geometric center of the layer in the same layer, then these convolution kernels may be represented by other convolution kernels of the layer, pruning these convolution kernels has little effect on the performance of the network, optimizing the storage requirements of the model and speeding up training and prediction while maintaining the performance of the network by pruning these convolution kernels.
Step S9, realizing continuous optimization updating of the defect detection model:
continuously inputting unknown class defect data and known class defect data, repeating the steps S1 to S8, continuously updating the network to obtain a defect detection network which is globally unified for the unknown class and the known class, and realizing continuous optimization of defect detection.
Specifically, unknown and known type defect data are continuously input, based on an old model, the steps of feature extractor expansion, auxiliary classifier construction, feature fusion device construction, classifier updating, incremental training by using a random gradient descent method, unknown defect type example set construction and known defect type example set adjustment, and feature extractor pruning are sequentially performed, so that a defect detection network which is globally unified to the current unknown and known types is obtainedThereby realizing continuous optimization of defect detection;
the specific detection process is as follows:
defect detection networkFor all received solar cell images to be detected +.>And performing defect detection. First, based on feature extractor->The extracted features are connected to obtain intermediate feature map +.>，
；
Then, according to step S3, the intermediate feature map is passed through a feature fusion deviceTransformation into a refining profile->，
；
；
Will refine the feature mapSend to classifier->The defect type label of the solar cell is predicted to be +.>And according to the defect category, the defect category is obtained, and the expression is as follows:
；
。
the above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. The continuous optimization method for detecting the defects of the solar cell based on incremental learning is characterized by comprising the following steps of:
step S1, defect data is continuously input:
inputting an example set and defect data, wherein the example set is a subset of an old data set, the defect data comprises unknown class data and known class data containing new characteristic values, and a union set of the example set and the defect data is a training set;
step S2, expanding a feature extractor:
constructing a new feature extractor, and initializing the new feature extractor by using parameters of the feature extractor in the old model to obtain a new feature extractor;
step S3, constructing an auxiliary classifier:
constructing an auxiliary classifier, treating all known class data as a class, and calculating auxiliary loss of related new features;
step S4, constructing a feature fusion device:
extracting features of the solar cell image from a plurality of scales through a newly added feature extractor, connecting all the extracted features to obtain an intermediate feature map, refining the intermediate feature map based on a channel and a spatial attention mechanism, inputting the intermediate feature map, sequentially deducing a one-dimensional attention force map and a two-dimensional attention force map, and obtaining a refined feature map through a feature fusion device;
step S5, updating a classifier:
copying parameters for old features from old model classifiers to construct a current stage classifier, adding corresponding full-connection layer nodes according to the number of unknown defect categories in the current stage, randomly initializing new parameters, sending a refined feature map to the current stage classifier to obtain prediction probability, and calculating cross entropy loss;
step S6, performing incremental training by using a random gradient descent method:
performing incremental training on the old model based on a training set, learning an unknown class sample and a known class sample containing a new characteristic value by using a random gradient descent method according to the global loss containing classification loss and auxiliary loss, and converting the unknown class sample into the known class sample to obtain an updated new model;
step S7, the unknown defect class example set construction and the known defect class example set adjustment:
training unknown defect type data and known defect type data based on a new model to obtain an unknown type example set and a known type example set with new characteristics, wherein the unknown type example set, the known type example set and the example set in the step S1 form a new example set for incremental training of the next stage;
step S8, pruning by the feature extractor:
calculating the geometric center of each layer of the newly added feature extractor in the new model, pruning the convolution kernel of the layer based on the geometric center;
step S9, realizing continuous optimization updating of the defect detection model:
continuously inputting unknown class defect data and known class defect data, repeating the steps S1 to S8, continuously updating the old model to obtain a defect detection network which is globally unified for the unknown class and the known class, and realizing continuous optimization of defect detection.
2. The continuous optimization method for detecting defects of a solar cell according to claim 1, wherein in step S1, for continuously inputted defect dataPerforming optimized detection, wherein->When->When the model to be updated is the basic neural network model +.>，/>Training set ∈>Training to obtain the product.
3. The continuous optimization method for solar cell defect detection according to claim 2, wherein in step S2, the feature extractor is built based on a multi-scale residual network.
4. The continuous optimization method for solar cell defect detection according to claim 3, wherein in step S3, the auxiliary classifier uses probabilityThe prediction classification is performed with the following expression:
；
wherein,representation->A function; />Representing an auxiliary classifier whose label space is，/>A set of tags representing unknown categories; />Representing the new feature;
auxiliary loss on new featuresThe expression of (2) is as follows:
；
wherein,representing a training set; />Representing an image; />Representing a corresponding category label.
5. The continuous optimization method for solar cell defect detection according to claim 4, wherein in step S4, the channel and spatial attention mechanisms are specifically a channel attention sub-network and a spatial attention sub-network, and the specific process of obtaining the refined feature map by the feature fusion device is as follows:
channel attention subnetwork pass-throughAnd->Compression intermediate feature map->Is>Is +.>Two eigenvectors are generated->And->The dimensions are->Connecting the two eigenvectors to get +.>Two pairs of full convolution layers are used>The process is performed as follows:
；
；
wherein,representing the weight of each channel; />Representing an activation function; first full convolution layerThe number of output channels of (a) is +.>，/>Representing a reduction ratio; second full convolution layer->Is of convolution kernel size +.>The number of output channels is +.>；/>Representing channel characteristics;
the spatial attention sub-network takes as input the channel characteristics, applies the average and maximum operations along the channel axis to generate two feature maps, expressed as follows:
；
；
wherein,and->Representing the element average and maximum operations in the channel axis, respectively;
based onAnd->Obtain dimension +.>Is striving for->The expression is as follows:
；
the spatial attention map is multiplied by the channel characteristics to obtain a refined characteristic map, and the expression is as follows:
；
wherein,representing spatially refined features, i.e. refining feature map +.>。
6. The continuous optimization method for solar cell defect detection according to claim 5, wherein in step S5, the probability is predictedThe expression of (2) is as follows:
；
wherein,representing a classifier;
cross entropy lossThe expression of (2) is as follows:
；
wherein,representing a training set; />Representing an image; />Representing a corresponding category label.
7. The continuous optimization method for solar cell defect detection according to claim 6, wherein in step S6, global loss is achievedThe expression of (2) is as follows:
；
wherein,super parameters representing the effect of controlling the auxiliary classifier, for the initial model, i.e. +.>In the time of (1),。
8. the method according to claim 7, wherein in step S7, the exemplary set update method is as follows:
building an unknown class exemplar setFor unknown class data set->Random sampling to obtain an initial unknown class exemplar set +.>Use of the model->Initializing intermediate model->And at->Upper training->Performing several iterations by random gradient descent to obtain +.>The iterative process is as follows:
；
wherein,representing fine tuning->Is->Representing cross entropy loss, < >>Representation->Is a gradient of (2);
calculation ofUpper->And back-propagating the validation loss to optimize +.>Obtaining a final sample set of unknown classes +.>The propagation process is as follows:
；
wherein,learning rate representing unknown class exemplar set propagation process, < >>Representing cross entropy loss, < >>Representation->Is a gradient of (2);
adjusting a set of known class exemplarsFor the known class dataset +.>Randomly sampling to obtain->The method comprises the steps of carrying out a first treatment on the surface of the Second, use +.>Initializing intermediate model->And at->Upper training->Performing several iterations by random gradient descent to obtain +.>The iterative process is as follows:
；
calculation ofUpper->And back-propagating the validation loss to optimize +.>Obtaining the final set of known class exemplars with new feature values +.>The propagation process is as follows:
；
wherein,representing the learning rate of the known sample set propagation process.
9. The continuous optimization method for solar cell defect detection according to claim 8, wherein in step S8, the geometric centerIs represented as follows:
；
；
wherein,representation->First->First of all convolution layers>A convolution kernel with dimension +>，/>And->Respectively represent +.>The number of input channels and the number of output channels of the convolution layers;
when (when)First->Geometric center of the convolutional layers->Is located at->In the convolution kernels of the convolution layers, the following formula is satisfied:
；
wherein,indicate->The convolution layers have geometric centers in the layer +.>Convolution kernels of the same value, i.e. requiring pruning, +.>Representation->And->A second norm of the difference;
the first is calculated by a formulaThe convolution kernels of pruning are needed in the convolution layers, so that model pruning is realized, and the expression is as follows:
；
wherein,indicate->The convolution layers have geometric centers in the layer +.>Convolution kernels of the same or similar value, i.e. requiring pruning, +.>Representation->And->And the two norms of the difference.
10. The continuous optimization method for solar cell defect detection according to claim 9, wherein in step S9, unknown and known type defect data is continuously input, and feature extractor expansion, auxiliary classifier construction, feature fusion device construction and classification are sequentially performed based on the old modelUpdating the device, performing incremental training by using a random gradient descent method, constructing an unknown defect type sample set, adjusting the known defect type sample set, pruning by a feature extractor to obtain a defect detection network which is globally uniform for the unknown and known typesThe continuous optimization of defect detection is realized;
the specific detection process is as follows:
defect detection networkFor all received solar cell images to be detected +.>Performing defect detection; first, based on feature extractor->The extracted features are connected to obtain intermediate feature map +.>，
；
Then, according to step S3, the intermediate feature map is passed through a feature fusion deviceTransformation into a refining profile->，
；
；
Will refine the feature mapSend to classifier->The defect type label of the solar cell is predicted to be +.>And according to the defect category, the defect category is obtained, and the expression is as follows:
；
。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311584711.3A CN117314892B (en) | 2023-11-27 | 2023-11-27 | Incremental learning-based continuous optimization method for solar cell defect detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311584711.3A CN117314892B (en) | 2023-11-27 | 2023-11-27 | Incremental learning-based continuous optimization method for solar cell defect detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117314892A true CN117314892A (en) | 2023-12-29 |
| CN117314892B CN117314892B (en) | 2024-02-13 |
Family
ID=89285065
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311584711.3A Active CN117314892B (en) | 2023-11-27 | 2023-11-27 | Incremental learning-based continuous optimization method for solar cell defect detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117314892B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114612721A (en) * | 2022-03-15 | 2022-06-10 | 南京大学 | Image classification method based on multilevel adaptive feature fusion type increment learning |
| US20220222534A1 (en) * | 2019-05-23 | 2022-07-14 | The Trustees Of Princeton University | System and method for incremental learning using a grow-and-prune paradigm with neural networks |
| CN114821852A (en) * | 2022-06-07 | 2022-07-29 | 国网安徽省电力有限公司宣城供电公司 | Power grid defect depth identification inspection robot control system based on characteristic pyramid |
| CN115908340A (en) * | 2022-11-29 | 2023-04-04 | 中国科学技术大学 | Printed circuit board defect detection method and equipment based on incremental learning |
| CN116468948A (en) * | 2023-04-23 | 2023-07-21 | 武汉大学 | Incremental learning detection method and system for supporting detection of unknown urban garbage |
| CN116543357A (en) * | 2023-07-05 | 2023-08-04 | 南通玖方新材料科技有限公司 | Surface defect identification and early warning method and system for solar photovoltaic carrier |
| CN116630285A (en) * | 2023-05-31 | 2023-08-22 | 河北工业大学 | Photovoltaic cell type incremental defect detection method based on significance characteristic hierarchical distillation |
| CN116630277A (en) * | 2023-05-29 | 2023-08-22 | 清华大学 | PCB defect detection method and device based on continuous learning |
| CN116797928A (en) * | 2023-06-21 | 2023-09-22 | 西安电子科技大学 | SAR target increment classification method based on stability and plasticity of balance model |
| CN116843620A (en) * | 2023-05-29 | 2023-10-03 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Printed circuit board defect detection method based on non-example type increment learning |
-
2023
- 2023-11-27 CN CN202311584711.3A patent/CN117314892B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220222534A1 (en) * | 2019-05-23 | 2022-07-14 | The Trustees Of Princeton University | System and method for incremental learning using a grow-and-prune paradigm with neural networks |
| CN114612721A (en) * | 2022-03-15 | 2022-06-10 | 南京大学 | Image classification method based on multilevel adaptive feature fusion type increment learning |
| CN114821852A (en) * | 2022-06-07 | 2022-07-29 | 国网安徽省电力有限公司宣城供电公司 | Power grid defect depth identification inspection robot control system based on characteristic pyramid |
| CN115908340A (en) * | 2022-11-29 | 2023-04-04 | 中国科学技术大学 | Printed circuit board defect detection method and equipment based on incremental learning |
| CN116468948A (en) * | 2023-04-23 | 2023-07-21 | 武汉大学 | Incremental learning detection method and system for supporting detection of unknown urban garbage |
| CN116630277A (en) * | 2023-05-29 | 2023-08-22 | 清华大学 | PCB defect detection method and device based on continuous learning |
| CN116843620A (en) * | 2023-05-29 | 2023-10-03 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Printed circuit board defect detection method based on non-example type increment learning |
| CN116630285A (en) * | 2023-05-31 | 2023-08-22 | 河北工业大学 | Photovoltaic cell type incremental defect detection method based on significance characteristic hierarchical distillation |
| CN116797928A (en) * | 2023-06-21 | 2023-09-22 | 西安电子科技大学 | SAR target increment classification method based on stability and plasticity of balance model |
| CN116543357A (en) * | 2023-07-05 | 2023-08-04 | 南通玖方新材料科技有限公司 | Surface defect identification and early warning method and system for solar photovoltaic carrier |
Non-Patent Citations (3)
| Title |
|---|
| BINYI SU 等: "Deep Learning-Based Solar-Cell Manufacturing Defect Detection With Complementary Attention Network", IEEE * |
| MU YE 等: "surface defects inspection of cylindrical metal workpieces based on weakly supervised learning", RESEARCH SQUARE * |
| 谢昭 等: "独立子空间中的场景特征增量学习方法", 计算机研究与发展, no. 11 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117314892B (en) | 2024-02-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110472817B (en) | XGboost integrated credit evaluation system and method combined with deep neural network | |
| CN108596053B (en) | Vehicle detection method and system based on SSD and vehicle posture classification | |
| CN110288030B (en) | Image identification method, device and equipment based on lightweight network model | |
| CN107194336B (en) | Polarized SAR image classification method based on semi-supervised depth distance measurement network | |
| CN111274903B (en) | Cervical cell image classification method based on graph convolution neural network | |
| CN112101430B (en) | Anchor frame generation method for image target detection processing and lightweight target detection method | |
| CN107330355B (en) | Deep pedestrian re-identification method based on positive sample balance constraint | |
| CN108090472B (en) | Pedestrian re-identification method and system based on multi-channel consistency characteristics | |
| CN112699953B (en) | Feature pyramid neural network architecture searching method based on multi-information path aggregation | |
| CN109376787B (en) | Manifold learning network and computer vision image set classification method based on manifold learning network | |
| CN104732545A (en) | Texture image segmentation method combined with sparse neighbor propagation and rapid spectral clustering | |
| CN111598167B (en) | Small sample image identification method and system based on graph learning | |
| CN113298009B (en) | Entropy regularization-based self-adaptive adjacent face image clustering method | |
| CN108536844B (en) | Text-enhanced network representation learning method | |
| CN111695011B (en) | Tensor expression-based dynamic hypergraph structure learning classification method and system | |
| CN111192240B (en) | Remote sensing image target detection method based on random access memory | |
| CN114330516A (en) | Small sample logo image classification based on multi-graph guided neural network model | |
| CN114299362A (en) | Small sample image classification method based on k-means clustering | |
| CN114373092A (en) | Progressive training fine-grained vision classification method based on jigsaw arrangement learning | |
| CN104573728B (en) | A kind of texture classifying method based on ExtremeLearningMachine | |
| CN117993282A (en) | Domain adaptability information bottleneck federal learning method for intelligent manufacturing fault diagnosis | |
| CN117314892B (en) | Incremental learning-based continuous optimization method for solar cell defect detection | |
| CN114677547B (en) | Image classification method based on self-holding characterization expansion type incremental learning | |
| CN116758349A (en) | Hyperspectral image classification method based on multi-scale super-pixel node aggregate graph convolution residual network | |
| CN116433980A (en) | Image classification method, device, equipment and medium of impulse neural network structure |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |