CN113657561A - Semi-supervised night image classification method based on multi-task decoupling learning - Google Patents
Semi-supervised night image classification method based on multi-task decoupling learning Download PDFInfo
- Publication number
- CN113657561A CN113657561A CN202111220897.5A CN202111220897A CN113657561A CN 113657561 A CN113657561 A CN 113657561A CN 202111220897 A CN202111220897 A CN 202111220897A CN 113657561 A CN113657561 A CN 113657561A
- Authority
- CN
- China
- Prior art keywords
- learning
- classification
- night
- network
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 239000013598 vector Substances 0.000 claims abstract description 44
- 238000012549 training Methods 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims abstract description 26
- 230000006870 function Effects 0.000 claims abstract description 21
- 238000004821 distillation Methods 0.000 claims abstract description 16
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000012544 monitoring process Methods 0.000 claims description 8
- 238000010606 normalization Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000013140 knowledge distillation Methods 0.000 description 3
- 238000013508 migration Methods 0.000 description 3
- 230000005012 migration Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000002860 competitive effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 239000010749 BS 2869 Class C1 Substances 0.000 description 1
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a semi-supervised night image classification method based on multitask decoupling learning, wherein a sample with a label in the day and a sample without a label at night are input into a feature extraction network together, wherein a feature vector extracted from the sample in the day is input into a classification network head, and a cross entropy loss function is adopted for supervision; firstly inputting feature vectors extracted from samples at night into a classification network head to obtain pseudo labels, then constructing positive and negative sample pairs according to the pseudo labels, inputting the positive and negative sample pairs into an automatic supervision network head, and carrying out supervision training by adopting an angle comparison loss function; after the model multitask training is completed, a small number of samples with labels in the night data set are input into the feature extraction network and the classification network head for iterative self-distillation learning, and finally the effect that the night data set can be effectively classified is achieved.
Description
Technical Field
The invention relates to multi-task learning in the technical field of computer vision recognition, in particular to a semi-supervised night image classification method based on multi-task decoupling learning.
Background
Domain migration is an urgent problem to be solved in computer vision, and in the definition of the problem, a source domain and a target domain have the same task and different but related data. The core task of this type of learning is to solve the problem of differences in the distribution of the two domain data. At present, the general image recognition algorithm is trained on a supervised data set, and achieves higher performance on similarly distributed images. However, when migrating to images of other target domains, the performance tends to be extremely degraded, which is caused by the difference in data distribution between the source domain and the target domain. For example, when a network trained based on a daytime dataset predicts nighttime images, the recognition effect is often greatly reduced.
It is well known that there are currently a large number of open-source daytime image classification datasets, such as the PASCAL VOC, but tagged nighttime image classification datasets are quite lacking. Therefore, it is desirable to train a network with a dataset of daytime images and to enable efficient migration of the network to nighttime image classification, thereby improving the performance of nighttime image classification.
The self-supervision learning mainly utilizes an auxiliary task to mine self supervision information from large-scale unsupervised data, and trains a network through the constructed supervision information, thereby learning valuable characteristics of downstream tasks. The learning method is proved to capture the discriminant characteristics of the image, and is an effective solution for the task lacking the label data. The self-supervision learning is carried out on a large number of non-labeled night images, so that the characteristics of the night images can be distributed through network learning, and the accuracy of night image classification is improved.
Therefore, the task of night image classification is decoupled into the task of supervised classification of the daytime image and the task of self-supervision of the night image, and the two tasks are subjected to multi-task learning, so that the model has the capability of extracting various discriminant characteristics and can adapt to data distribution of the night image. However, in the multi-task learning, there is a competitive relationship between tasks, and how to promote the two tasks mutually, rather than restrict each other, requires designing an effective loss function.
In recent years, knowledge distillation has become a hot topic. Knowledge distillation achieves knowledge migration by introducing soft targets associated with the teacher network as part of the loss to induce training of the student network. The definition of self-distillation is that the self-distillation learns from self to induce the training of the next generation network by the soft target related to self. The method can generally enhance the robustness of the network and avoid overfitting, and therefore, the method can be suitable for further improving the performance of the model in night images.
Disclosure of Invention
In order to solve the defects of the prior art and achieve the purpose of improving the night image recognition performance, the invention adopts the following technical scheme:
a semi-supervised night image classification method based on multitask decoupling learning comprises the following steps:
s1, constructing a labeled daytime image classification data set D; constructing a nighttime image classification data set A, wherein only part of samples of the nighttime image are provided with labels, and the rest samples are not provided with category labels;
s2, inputting the samples with labels in the daytime image data set and the samples without labels in the nighttime image data set into a feature extraction network together, and outputting a daytime image feature vector and a nighttime image feature vector; the feature extraction network is a deep residual error convolution network;
s3, accessing a multi-task learning network after the feature extraction network layer, wherein the network is composed of a supervised classification network head and an automatic supervision network head;
s4, for the feature vector of the daytime image, the step is carried out through a classification network head
Loss supervision training; for the feature vector of the night image, predicting the category of the feature vector as a pseudo label through the same classification network head, and constructing a pair of positive and negative samples of the night image according to the pseudo label; the classification network head consists of a global average pooling layer and a full-connection layer;
s5, the self-monitoring network head normalizes the positive and negative sample pairs of the night image according to the weight parameter of the classification network head to obtain the normalized feature vector, and adopts the contrast loss
Guiding the learning of the feature space to enable the positive samples to be similar and the negative samples to be effectively distinguished;
s6, carrying out co-supervision training on the loss supervision training and the comparison loss;
s7, collecting the nighttime image data with the labeled sample, inputting the trained feature extraction network and classification network head, fixing the weight of the feature extraction network, and processing through the classification network head
Loss supervision training, so that the classification network head adapts to the feature distribution of the images at night; entering into self-distillation learning stage, performing multiple iteration updating, and utilizing previous iteration
Taking the classification prediction result of the loss supervision training as a soft target, and participating in supervision together with a real label;
and S8, in the reasoning stage, inputting the night image to be detected into the trained feature extraction network and classification network head, and outputting an image classification result.
Further, in S4, the feature vector of the daytime image is input into the classification network header, the category of the daytime sample is output, and the monitoring is performed through the cross entropy loss function:
wherein,Nrepresenting the total number of samples tagged in the daytime image dataset,y i is shown asiThe authenticity of the label of the individual specimen,
is shown asiThe class of samples predicts a probability value.
Further, in S4, the feature vector of the night image is input to the classification network header for calculation to obtain a predicted pseudo label, and a positive and negative sample pairing tone of the night image is constructed according to the pseudo labelk,k +,k -} m ，k +Is composed ofkA positive sample of (1), andkbelonging to the same label, and the label is a single label,k -is composed ofkA negative sample ofkBelonging to different labels, and belonging to different labels,mindicating the number of sample pairs.
Further, in S5, the positive and negative sample feature pairs are angle-normalized:
wherein x represents the input feature vector, | | x | | | represents the modular length of the feature vector x, y represents the label to which the vector x belongs, and W represents the label to which the vector x belongsyA parameter indicating the y-th row of the fully connected layer in the classified network header; checking positive and negative samplesk,k +,k -} m Carrying out angle normalization calculation on each sample feature vector to obtain normalized feature vector { Lambdak,Λk +,Λk -} m ：
Λk=Λ(k,W,y)
Λk +=Λ(k +,W,y)
Λk -=Λ(k -,W,y)。
Further, in S5, the learning of the feature space is guided by the comparison loss, so that the positive samples are similar and the negative samples are effectively distinguished, and the following loss function is adopted:
wherein, yk,yk+,ykRespectively representing samples of a sample pairk,k +,k -The real label of (a) is,𝜂is a hyper-parameter, representing a minimum threshold value for the distance between samples of different classes,
representing a similarity function.
Further, the feature vector { Lambda after normalization is subjected to cosine similarity functionk,Λk +,Λk -} m And (3) carrying out similarity comparison:
wherein A is i 、B i Representing the components of vectors A and B, respectively, with the similarity of the positive samples
1, similarity of negative examples
Is-1.
Further, the total loss function of S6 is:
and when the training epoch reaches the specified times, stopping training.
Further, in S7, the sample with the label in the nighttime image data set is input into the trained feature extraction network and the classification network header, the weight of the feature extraction network is fixed, and the classification network header is supervised by using the cross entropy loss function:
wherein,N’indicating the total number of samples with labels in the night image dataset,y iis shown asiThe authenticity of the label of the individual specimen,
is shown asiThe class of samples predicts a probability value.
Further, in S7, the self-distillation learning stage is performed to perform multiple iterative updates, and the previous iteration is utilized
Class prediction of loss supervised training as soft target
And a genuine labelyAnd (3) participating in supervision together:
wherein, lambda represents the proportion of the soft target loss, and the self-distillation training is completed after repeated iteration updating.
A semi-supervised night image classification method based on multi-task decoupling learning inputs an image to be detected into a trained feature extraction network and a classification network head and outputs an image classification result.
The invention has the advantages and beneficial effects that:
the invention firstly proposes that multi-task learning and knowledge distillation are combined to enable image classification at night, self-supervision learning is carried out by using non-label images at night, and a network can learn the feature distribution of images at night in a self-adaptive manner while learning the feature of image classification at day time; carrying out self-supervision learning through an angle normalization loss function, and reducing the competitive relation between the self-supervision loss and the supervised loss; by the self-distillation method, distillation learning is carried out by utilizing a small amount of labeled data at night, the phenomenon that the generalization capability is lost due to the fact that a network is over-fitted to a target domain can be avoided, and meanwhile, the model can be properly adapted to the data at night.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
FIG. 2 is a schematic diagram of the multitask decoupled learning phase of the present invention.
Fig. 3 is an exemplary diagram of positive and negative sample pairs in the present invention.
FIG. 4 is a schematic diagram of the self-distillation learning stage of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
The method combines the self-supervision learning of night data and the supervision learning of day data to train a feature extraction network with domain self-adaptation capability, and further performs self-distillation learning on the image recognition network through a small amount of labeled samples in night data collection, so that the classification network head migrates to the night data distribution feature, and the night image recognition performance is improved.
As shown in fig. 1 and 2, the semi-supervised night image classification method based on multitask decoupling learning of the present invention includes the following steps:
step 1: a tagged daytime image classification dataset is constructed and a nighttime image classification dataset is constructed, wherein only a small number of nighttime samples are tagged. This embodiment uses 12 categories in the starting dataset exclusive Dark (ExDARK), which are bicycles, boats, bottles, buses, cars, cats, chairs, cups, dogs, motorcycles, people, and tables, respectively. For the 12 categories, 800 corresponding images are respectively selected from the COCO public data set as a daytime image classification data set D. In addition, the ExDARK dataset was divided into 3 parts: respectively extracting 400 images from each category to construct an unsupervised night image dataset A; extracting 10 images from each category as a small number of labeled night image data sets T; finally, the remaining images are used as a nighttime image classification performance verification set V to evaluate the effectiveness of the algorithm;
step 2: and inputting the image samples with the labels in the daytime data set D and the image samples without the labels in the nighttime data set A into the feature extraction network together, and outputting the feature vectors of the data of each sample. The feature extraction network is a deep residual convolution network, and in this embodiment, a ResNet50 network is adopted to output a feature vector with a dimension of 2048 at the conv5_ x layer. Network adoption of all image samples
And using a random cropping, horizontally flipped image enhancement technique to expand sample diversity. The size of the daytime image sample batch input each time is 32, the size of the nighttime image sample batch _ size is 32, and 8-card GPU parallel training is adopted;
step 3, accessing a multitask decoupling learning network after the characteristic extraction network layer, wherein the network consists of a supervised classification network head and an automatic supervision network head;
and 4, step 4: and constructing a classification network head, wherein the classification network head is composed of a global average pooling layer and a full connection layer. In this embodiment, an average _ pool layer and a fully-connected layer with a dimension of [2048, 12] are adopted, where 12 is the number of output categories;
step 4.1: inputting the feature vectors extracted from the daytime samples in the step 2 into a classification network head, selecting the category corresponding to the highest probability as the category prediction result of the feature point, adopting a cross entropy loss function for supervision, and using a calculation formula of the cross entropy loss function
The following were used:
Nthe total number of samples is represented by,y i is shown asiThe authenticity of the label of the individual specimen,
is shown asiA class prediction probability value of each sample;
step 4.2: inputting the feature vector extracted from the night sample in the step 2 into a classification network head to obtain a pseudo label of the sample, and constructing a positive and negative sample pairingk,k +,k -} m ：k +Is composed ofkPositive samples of (1), i.e. withkBelong to the same tag;k -is composed ofkNegative examples of (i.e. AND)kBelonging to different labels, and belonging to different labels,mindicating the number of sample pairs. The specific construction method comprises the steps that in 32 night sample vectors, a class C1 is randomly selected, samples in the class are paired randomly in pairs to obtain a group of positive sample pair sets C1{ … }, 1 sample is randomly selected from other classes and combined with a positive sample pair in C1{ … } to obtain a plurality of positive and negative sample pairs; one category C2 is then selected from the remaining other categories and the above operation is repeated until 16 positive and negative sample pairs are obtained. For the extreme case of less than 16 samples from the same class, this time without the input of the unsupervised network. Thus, it is possible to providemIn most cases 16. FIG. 3 is an example of a positive and negative sample pair in the present embodiment;
and 5: constructing an automatic monitoring network head: subjecting the positive and negative samples obtained in step 3.2 to face-to-facek,k +,k -} m And inputting a weight parameter W of the classification network head into the self-monitoring network head, and firstly, carrying out angle normalization on the sample characteristics, wherein the calculation formula is as follows:
x represents the input feature vector, | x | | represents the modular length of the feature vector x, y represents the label to which the vector x belongs, WyA parameter indicating the y-th row of the fully connected layer in the classified network header. The angle normalization processing can relieve the competition relationship between the additional task and the main task in the multi-task learning task, namely, the monitoring of the self-monitoring task is reducedSupervising the negative effects of the task;
checking positive and negative samplesk,k +,k -} m Carrying out angle normalization calculation on each sample feature vector to obtain normalized feature vector { Lambdak,Λk +,Λk -} m ：
Λk=Λ(k,W,y)
Λk +=Λ(k +,W,y)
Λk -=Λ(k -,W,y)
Step 5.1: using cosine similarity function pair { Lambdak,Λk +,Λk -} m Performing similarity comparison, and similarity function thereof
The calculation formula is as follows:
A i 、B i representing the components of vectors A and B, respectively, with the similarity of the positive samples
Should be 1, similarity of negative examples
Should be-1;
step 5.2: the learning of the feature space is guided by adopting the comparison loss, so that the positive samples are similar, the negative samples are effectively distinguished, and the loss function is
The calculation formula is as follows:
yk,yk+,ykrespectively representing samples of a sample pairk,k +,k -The real label of (a) is,𝜂is a hyper-parameter indicating that the distance between samples of different classes should exceed this value;
step 6: and (3) carrying out co-supervision training on the feature extraction network and the multitask decoupling learning network by using the loss functions of the step (4.1) and the step (5.2), wherein the total loss function is as follows:
in this embodiment, an SGD optimizer is used, the initial learning rate is 0.01, and when the training epoch reaches 70, the learning rate is reduced to 0.001. Stopping training when the epoch reaches 100 times;
and 7: inputting a small number of samples with labels in a night data set into a trained feature extraction network and a classification network head, fixing the weight of the feature extraction network, and performing further supervision training on the classification network head by using a cross entropy loss function to enable the classification network head to adapt to the data distribution of the night image features, wherein the calculation formula is as follows:
N’the total number of samples is represented by,y iis shown asiThe authenticity of the label of the individual specimen,
is shown asiA class prediction probability value of each sample;
step 7.1: as shown in FIG. 4, in the self-distillation learning stage, the previous classification prediction result is used as a soft target
Participating in supervision with the real tag y, its loss function
The calculation formula of (a) is as follows:
λ represents the proportion of the soft target loss, and in this example, λ =0.5, the model performance is best. Based on loss functions
The network is reversely propagated, the learning rate is 0.005, and the network parameters are continuously updated by a batch gradient descent method;
step 7.2: repeating the step 6.1, and finishing the training of the self-distillation network after the loss difference of the model in two times is less than 0.1 after 10 times of iterative updating;
and 8: and in the reasoning stage, the night image to be detected is input into the feature extraction network and the classification network head, and an image classification result is output. The training and reasoning stages of the example are all realized on a GPU server GEFORCE RTX 2080 Ti.
According to the invention, the task of night image classification is decoupled into the task of supervised classification of daytime images and the task of self-supervision of night images, a feature extraction network with domain self-adaptation capability is trained after multitask learning is carried out, and further self-distillation learning is carried out on the image recognition network through a small number of labeled samples at night, so that the characteristics learned by a classification network head are shifted to the characteristics of the night images, and the night image recognition performance is improved. The classification performance of the verification data set V adopted in the embodiment reaches 83.8% based on the ResNet50 network, the classification performance can reach 89.2% by adopting the algorithm of the invention, the accuracy is improved by 5.4% compared with that of baseline, and the practical benefit and the application value of the invention are fully embodied.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. A semi-supervised night image classification method based on multitask decoupling learning is characterized by comprising the following steps:
s1, constructing a daytime image classification data set and a nighttime image classification data set, wherein the daytime image classification data set is sample images with class labels, and only part of the sample images in the nighttime image classification data set are labeled;
s2, inputting the sample images with labels in the daytime image classification data set and the sample images without labels in the nighttime image classification data set into a feature extraction network together, and outputting a daytime image feature vector and a nighttime image feature vector;
s3, accessing a multi-task decoupling learning network after the feature extraction network layer, wherein the network is composed of a supervised classification network head and an automatic supervision network head;
s4, for the feature vector of the daytime image, the step is carried out through a classification network head
Loss supervision training; for the feature vector of the night image, predicting the category of the feature vector as a pseudo label through a classified network head, and constructing a pair of positive and negative samples of the night image according to the pseudo label;
s5, the self-monitoring network head normalizes the positive and negative sample pairs of the night image according to the weight parameter of the classification network head to obtain the normalized feature vector, and adopts the contrast loss
Guiding the learning of the feature space to enable the positive samples to be similar and the negative samples to be effectively distinguished;
s6, the classification network head and the self-supervision network head are subjected to multi-task training;
s7, inputting the sample with the label in the night image data set, and finishing trainingThe weight of the characteristic extraction network is fixed and the operation is carried out by the classification network head
Loss supervision training, so that the classification network head adapts to the feature distribution of the images at night; entering into self-distillation learning stage, performing multiple iteration update on the weight parameters of the classification network head, and utilizing the previous time
Taking the classification prediction result of the loss supervision training as a soft target, and participating in supervision together with a real label;
and S8, in the reasoning stage, inputting the night image to be detected into the trained feature extraction network and classification network head, and outputting an image classification result.
2. The semi-supervised night image classification method based on multi-task decoupled learning as claimed in claim 1, wherein in S4, the feature vectors of the daytime images are input into a classification network header, the predicted sample classes are output, and the monitoring is performed through a cross entropy loss function:
wherein,Nrepresenting the total number of samples tagged in the daytime dataset,y i is shown asiThe authenticity of the label of the individual specimen,
is shown asiThe class of samples predicts a probability value.
3. The method for semi-supervised night image classification based on multi-task decoupling learning as claimed in claim 1, wherein in S4, the night image feature vector is input into the classification network head for calculation to obtain the predicted pseudo-feature vectorA label, and constructing a positive and negative sample pairing according to the pseudo labelk,k +,k -} m ，k +Is composed ofkA positive sample of (1), andkbelonging to the same label, and the label is a single label,k -is composed ofkA negative sample ofkBelonging to different labels, and belonging to different labels,mindicating the number of sample pairs.
4. The semi-supervised nighttime image classification method based on multi-task decoupled learning of claim 3, wherein the positive and negative sample feature pairs are angle-normalized in the step S5:
wherein x represents the input feature vector, | | x | | | represents the modular length of the feature vector x, y represents the label to which the vector x belongs, and W represents the label to which the vector x belongsyA parameter indicating the y-th row of the fully connected layer in the classified network header; checking positive and negative samplesk,k +,k -} m Carrying out angle normalization calculation on each sample feature vector to obtain normalized feature vector { Lambdak,Λk +,Λk -} m ：
Λk=Λ(k,W,y)
Λk +=Λ(k +,W,y)
Λk -=Λ(k -,W,y)。
5. The semi-supervised night image classification method based on multi-task decoupling learning as claimed in claim 4, wherein in the step S5, the following contrast loss function is adopted:
wherein, yk,yk+,ykRespectively representing samples of a sample pairk,k +,k -The real label of (a) is,𝜂is a hyper-parameter, representing a minimum threshold value for the distance between samples of different classes,
representing a similarity function.
6. The semi-supervised night image classification method based on multi-task decoupling learning as claimed in claim 5, wherein the normalized eigenvector { Lambda ] is subjected to cosine similarity functionk,Λk +,Λk -} m And (3) carrying out similarity comparison:
wherein A is i 、B i Representing the components of vectors A and B, respectively, with the similarity of the positive samples
1, similarity of negative examples
Is-1.
7. The semi-supervised night image classification method based on multi-task decoupled learning of claim 1, wherein the total loss function of S6 is as follows:
and stopping training when the training reaches the specified times.
8. The semi-supervised night image classification method based on multi-task decoupling learning of claim 1, wherein in S7, the samples with labels in the night image data set are input to the trained feature extraction network and classification network header, the weights of the feature extraction network are fixed, and the classification network header is supervised by using the cross entropy loss function:
wherein,N’indicating the total number of samples with labels in the night image dataset,y iis shown asiThe authenticity of the label of the individual specimen,
is shown asiThe class of samples predicts a probability value.
9. The semi-supervised night image classification method based on multi-task decoupling learning as claimed in claim 1, wherein in step S7, a self-distillation learning stage is entered, and multiple iterative updates are performed, using a previous iteration
Class prediction of loss supervised training as soft target
And a genuine labelyAnd (3) participating in supervision together:
wherein, lambda represents the proportion of the soft target loss, and the self-distillation training is completed after repeated iteration updating.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111220897.5A CN113657561B (en) | 2021-10-20 | 2021-10-20 | Semi-supervised night image classification method based on multi-task decoupling learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111220897.5A CN113657561B (en) | 2021-10-20 | 2021-10-20 | Semi-supervised night image classification method based on multi-task decoupling learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113657561A true CN113657561A (en) | 2021-11-16 |
| CN113657561B CN113657561B (en) | 2022-03-18 |
Family
ID=78494703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111220897.5A Active CN113657561B (en) | 2021-10-20 | 2021-10-20 | Semi-supervised night image classification method based on multi-task decoupling learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113657561B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113869333A (en) * | 2021-11-29 | 2021-12-31 | 山东力聚机器人科技股份有限公司 | Image identification method and device based on semi-supervised relationship measurement network |
| CN113918743A (en) * | 2021-12-15 | 2022-01-11 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Model training method for image classification under long-tail distribution scene |
| CN114037876A (en) * | 2021-12-16 | 2022-02-11 | 马上消费金融股份有限公司 | Model optimization method and device |
| CN114255371A (en) * | 2021-12-21 | 2022-03-29 | 中国石油大学(华东) | Small sample image classification method based on component supervision network |
| CN114565808A (en) * | 2022-04-27 | 2022-05-31 | 南京邮电大学 | Double-action contrast learning method for unsupervised visual representation |
| CN114881937A (en) * | 2022-04-15 | 2022-08-09 | 北京医准智能科技有限公司 | Detection method and device for ultrasonic section and computer readable medium |
| CN114898141A (en) * | 2022-04-02 | 2022-08-12 | 南京大学 | Multi-view semi-supervised image classification method based on contrast loss |
| CN115496955A (en) * | 2022-11-18 | 2022-12-20 | 之江实验室 | Image classification model training method, image classification method, apparatus and medium |
| CN115564960A (en) * | 2022-11-10 | 2023-01-03 | 南京码极客科技有限公司 | Network image label denoising method combining sample selection and label correction |
| CN117058492A (en) * | 2023-10-13 | 2023-11-14 | 之江实验室 | Two-stage training disease identification method and system based on learning decoupling |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110110745A (en) * | 2019-03-29 | 2019-08-09 | 上海海事大学 | Based on the semi-supervised x-ray image automatic marking for generating confrontation network |
| US20200160177A1 (en) * | 2018-11-16 | 2020-05-21 | Royal Bank Of Canada | System and method for a convolutional neural network for multi-label classification with partial annotations |
| CN112990371A (en) * | 2021-04-27 | 2021-06-18 | 之江实验室 | Unsupervised night image classification method based on feature amplification |
| CN113378632A (en) * | 2021-04-28 | 2021-09-10 | 南京大学 | Unsupervised domain pedestrian re-identification algorithm based on pseudo label optimization |
-
2021
- 2021-10-20 CN CN202111220897.5A patent/CN113657561B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200160177A1 (en) * | 2018-11-16 | 2020-05-21 | Royal Bank Of Canada | System and method for a convolutional neural network for multi-label classification with partial annotations |
| CN110110745A (en) * | 2019-03-29 | 2019-08-09 | 上海海事大学 | Based on the semi-supervised x-ray image automatic marking for generating confrontation network |
| CN112990371A (en) * | 2021-04-27 | 2021-06-18 | 之江实验室 | Unsupervised night image classification method based on feature amplification |
| CN113378632A (en) * | 2021-04-28 | 2021-09-10 | 南京大学 | Unsupervised domain pedestrian re-identification algorithm based on pseudo label optimization |
Non-Patent Citations (1)
| Title |
|---|
| 贾鹏: "基于改进梯形网络的半监督图像分类研究", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113869333A (en) * | 2021-11-29 | 2021-12-31 | 山东力聚机器人科技股份有限公司 | Image identification method and device based on semi-supervised relationship measurement network |
| CN113869333B (en) * | 2021-11-29 | 2022-03-25 | 山东力聚机器人科技股份有限公司 | Image identification method and device based on semi-supervised relationship measurement network |
| CN113918743A (en) * | 2021-12-15 | 2022-01-11 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Model training method for image classification under long-tail distribution scene |
| CN114037876A (en) * | 2021-12-16 | 2022-02-11 | 马上消费金融股份有限公司 | Model optimization method and device |
| CN114037876B (en) * | 2021-12-16 | 2024-08-13 | 马上消费金融股份有限公司 | Model optimization method and device |
| CN114255371A (en) * | 2021-12-21 | 2022-03-29 | 中国石油大学(华东) | Small sample image classification method based on component supervision network |
| CN114898141A (en) * | 2022-04-02 | 2022-08-12 | 南京大学 | Multi-view semi-supervised image classification method based on contrast loss |
| CN114881937A (en) * | 2022-04-15 | 2022-08-09 | 北京医准智能科技有限公司 | Detection method and device for ultrasonic section and computer readable medium |
| CN114881937B (en) * | 2022-04-15 | 2022-12-09 | 北京医准智能科技有限公司 | Detection method and device for ultrasonic section and computer readable medium |
| CN114565808B (en) * | 2022-04-27 | 2022-07-12 | 南京邮电大学 | Double-action contrast learning method for unsupervised visual representation |
| CN114565808A (en) * | 2022-04-27 | 2022-05-31 | 南京邮电大学 | Double-action contrast learning method for unsupervised visual representation |
| CN115564960A (en) * | 2022-11-10 | 2023-01-03 | 南京码极客科技有限公司 | Network image label denoising method combining sample selection and label correction |
| CN115564960B (en) * | 2022-11-10 | 2023-03-03 | 南京码极客科技有限公司 | Network image label denoising method combining sample selection and label correction |
| CN115496955A (en) * | 2022-11-18 | 2022-12-20 | 之江实验室 | Image classification model training method, image classification method, apparatus and medium |
| CN115496955B (en) * | 2022-11-18 | 2023-03-24 | 之江实验室 | Image classification model training method, image classification method, device and medium |
| CN117058492A (en) * | 2023-10-13 | 2023-11-14 | 之江实验室 | Two-stage training disease identification method and system based on learning decoupling |
| CN117058492B (en) * | 2023-10-13 | 2024-02-27 | 之江实验室 | Two-stage training disease identification method and system based on learning decoupling |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113657561B (en) | 2022-03-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113657561B (en) | Semi-supervised night image classification method based on multi-task decoupling learning | |
| Gu et al. | Stack-captioning: Coarse-to-fine learning for image captioning | |
| Liu et al. | Incdet: In defense of elastic weight consolidation for incremental object detection | |
| CN109993100B (en) | Method for realizing facial expression recognition based on deep feature clustering | |
| CN113590819B (en) | Large-scale category hierarchical text classification method | |
| CN113179276B (en) | Intelligent intrusion detection method and system based on explicit and implicit feature learning | |
| CN112232395B (en) | Semi-supervised image classification method for generating countermeasure network based on joint training | |
| CN114580638A (en) | Knowledge graph representation learning method and system based on text graph enhancement | |
| CN111753995A (en) | Local interpretable method based on gradient lifting tree | |
| CN117557886A (en) | Noise-containing tag image recognition method and system integrating bias tags and passive learning | |
| CN117153268A (en) | Cell category determining method and system | |
| Duan et al. | Rda: Reciprocal distribution alignment for robust semi-supervised learning | |
| CN115130651A (en) | Pulse neural network inspired by multilayer heterogeneous mechanism of memory loop | |
| CN105787045A (en) | Precision enhancing method for visual media semantic indexing | |
| CN117746084A (en) | Unsupervised domain adaptive pedestrian re-identification method based on attention residual error and contrast learning | |
| Hao et al. | A Model-Agnostic approach for learning with noisy labels of arbitrary distributions | |
| CN112668633A (en) | Adaptive graph migration learning method based on fine granularity field | |
| US20230031512A1 (en) | Surrogate hierarchical machine-learning model to provide concept explanations for a machine-learning classifier | |
| CN117034110A (en) | Stem cell exosome detection method based on deep learning | |
| CN116563602A (en) | Fine granularity image classification model training method based on category-level soft target supervision | |
| Jing et al. | NASABN: A neural architecture search framework for attention-based networks | |
| Zuo et al. | Transfer learning in hierarchical feature spaces | |
| CN113361652A (en) | Individual income prediction oriented depolarization method and device | |
| Zhou et al. | Research on Underwater Image Recognition Based on Transfer Learning | |
| Cao et al. | Detection and fine-grained classification of malicious code using convolutional neural networks and swarm intelligence algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |