CN113657455A - Semi-supervised learning method based on triple network and labeling consistency regularization - Google Patents

Semi-supervised learning method based on triple network and labeling consistency regularization Download PDF

Info

Publication number
CN113657455A
CN113657455A CN202110837568.9A CN202110837568A CN113657455A CN 113657455 A CN113657455 A CN 113657455A CN 202110837568 A CN202110837568 A CN 202110837568A CN 113657455 A CN113657455 A CN 113657455A
Authority
CN
China
Prior art keywords
data
network
label
labeled
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110837568.9A
Other languages
Chinese (zh)
Other versions
CN113657455B (en
Inventor
蒋雯
苗旺
耿杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN202110837568.9A priority Critical patent/CN113657455B/en
Publication of CN113657455A publication Critical patent/CN113657455A/en
Application granted granted Critical
Publication of CN113657455B publication Critical patent/CN113657455B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a semi-supervised learning method based on a triple network and labeling consistency regularization, which comprises the following steps of: step one, inputting an image data set and a corresponding label set thereof; secondly, preprocessing the labeled and unlabeled data sets; thirdly, constructing and training a depth network of the self-adaptive vision mechanism for extracting the depth characteristics of the image data set; step four, constructing a twin network, and acquiring a forward propagation result and a pseudo label by using the depth characteristics of the marked data and the unmarked data; fifthly, constructing loss functions of training labeled data and unlabeled data by using the forward propagation result and the pseudo label, and performing semi-supervised training on the twin network; the method constructs a triple network to train a data set with insufficient annotation data, firstly establishes a generation confrontation network of a self-adaptive vision mechanism, and carries out unsupervised learning on an image data set for more effective feature extraction, thereby eliminating the difference between the feature extraction of heterogeneous annotation data and the feature extraction of non-annotation data; then, a twin network is established and trained based on the principle of label consistency, so that the difference of characteristic discrimination between similar labeled data and unlabeled data can be eliminated, the number of network training parameters is reduced, and the unlabeled data is more effectively utilized for semi-supervised learning.

Description

Semi-supervised learning method based on triple network and labeling consistency regularization
Technical Field
The invention belongs to the technical field of deep learning, and particularly relates to a semi-supervised learning method based on a triple network and labeling consistency regularization.
Background
In recent years, the deep learning method enables the field of artificial intelligence to be rapidly developed, and has a breakthrough meaning of 'paradigm' on informatization in the fields of intelligence, nerves, thinking and the like. The deep learning technology is a branch of machine learning, and is an algorithm for performing characterization learning on data by taking an artificial neural network as a framework. Deep learning method applications typically use a large amount of labeled data, learn fully supervised models, and have achieved good application results. However, such fully supervised learning is costly and time consuming, and data annotation must be done manually by researchers with expertise in the relevant field. Because some image data sets have the characteristics of high intra-class diversity and high inter-class similarity, the data is difficult to accurately label.
Therefore, for different tasks of deep learning in an actual scene, due to the diversity of data sources, only a subset of the training set usually has a label, and the rest of the data do not have a label. This occurs in all types of tasks, especially for image multi-classification tasks. Under the condition that the labeling supervision information is insufficient, the model cannot be sufficiently fitted, so that the difference exists between the extraction of the characteristics of the labeled data and the extraction of the characteristics of the unlabeled data, the correlation among the data cannot be sufficiently utilized, and the model with strong generalization capability is obtained.
The problem of data labeling is always the key research field of computer vision and artificial intelligence, and in order to improve the efficiency of deep learning models, a semi-supervised multi-classification technology for researching labeling consistency needs to eliminate the difference of an insufficient fitting model on model feature extraction.
The difference exists when the labeled data and the unlabeled data are extracted in the prior art, so that a semi-supervised learning method with labeling consistency is urgently needed, the unlabeled data in a data set is fully learned, and convenience is provided for a deep learning multi-classification task with insufficient labeled data of a subsequent actual scene.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a semi-supervised learning method based on the regularization of the triple network and the labeling consistency aiming at the defects in the prior art, and the semi-supervised learning method is simple in structure and reasonable in design.
In consideration of the characteristic that an actual scene has incomplete information, the phenomenon that labels are absent generally exists in the obtained data, so that the monitoring information is seriously insufficient, the difference exists between the extraction of the labeled data and the extraction of the unlabeled data, and the training effect and the generalization capability of deep learning network classification are limited. In order to solve the technical problems, the invention adopts the technical scheme that: the semi-supervised learning method based on the regularization of the triple network and the labeling consistency is characterized by comprising the following steps of: the method comprises the following steps:
step one, inputting an image data set and a corresponding label set:
step 101, an image data set V is input, in particular V ═ V1,...vi,...vlDividing the data into marked data X ═ X1,...xf,...xnAnd un-annotated data U ═ U1,...uj,...umIn which v isiRepresenting ith image sample data, i is more than or equal to 1 and less than or equal to l, f is more than or equal to 1 and less than or equal to n, j is more than or equal to 1 and less than or equal to m, l is m + n, and n, m and l are positive integers;
step 102, inputting a label set corresponding to the image set V, wherein the label data X is { X ═ X }1,...xi,...xnThe label of is p ═ p1,...pi,...pnU ═ U }, no-mark data U ═ U1,...uj,...umThere is no label.
Step two, preprocessing the labeled and unlabeled data sets:
step 201, performing data enhancement on the image labeled data X and the unlabeled data U, wherein the labeled data is subjected to single enhancement to obtain enhanced data X'. For the data without the label, performing random enhancement for K times to obtain enhanced data U';
step 202, mixing the data X 'and the data U', and randomly arranging to obtain a data combination W, wherein the label of the enhanced data is consistent with the original label.
Step three, constructing and training a depth network of the self-adaptive visual mechanism, and extracting the depth characteristics of the image data set:
step 301, constructing and generating the countermeasure network G, which is divided into a data generator and a discriminator.
Step 302, using a self-convolution layer in the generation of the countermeasure network, and setting a self-adaptive convolution kernel generation function based on the principles of spatial specificity and frequency domain independence. And outputting a convolution kernel with the same size as the feature map according to the input image features, controlling the scaling ratio to adjust the parameter quantity, and scaling the feature map channel.
And 303, deleting labels of the labeled data in the image set V, carrying out unsupervised learning on the countermeasure network G by using all the image sets V without labels, enabling the pseudo data characteristics generated by the generator to be close to real image characteristics, and enhancing the characteristic representation capability of the discriminator by using the self-convolution layer.
Step 304, generating the trained discriminator G of the countermeasure network GdAs a feature extractor FdAnd is used for extracting target image labeling data X ═ X1,...xf,...xnAnd un-annotated data U ═ U1,...uj,...umDepth feature x oflabeled＝Fd(xf) And xunlabeled＝Fd(uj)。
Step four, constructing a twin network, and acquiring a forward propagation result and a pseudo label by using the depth characteristics of the marked data and the unmarked data:
step 401, constructing two shallow classification networks Net1And Net2As a twin network, a data combination W is input;
step 402, inputting the enhanced data X' and the corresponding label p for the labeled data to obtain the depth feature Xlabeled＝Fd(X'), prediction is carried out by using twin network, and forward propagation results are
Figure BDA0003177713900000031
Wherein p isd1And pd2Is Net1And Net2W is a hyperparameter;
step 403, inputting enhanced data U 'to the unmarked data'Obtaining a depth feature xunlabeled＝Fd(U') predicting using twin networks, and taking the weighted average of the outputs as the forward propagation result pn，
Figure BDA0003177713900000032
Wherein,
Figure BDA0003177713900000033
and
Figure BDA0003177713900000034
and theta is a network training parameter for the prediction of the twin network on the unmarked data.
And step 404, sharpening the prediction of the label-free data to obtain a pseudo label q. Wherein the sharpening operation is specifically
Figure BDA0003177713900000035
T is the sharpening parameter, K is the number of enhancements, and P (U; θ) is the prediction probability of the network for each class.
And 405, performing label fusion on the pseudo label q predicted by the twin network. Specifically, the fused pseudo tag is:
Figure BDA0003177713900000036
wherein,
Figure BDA0003177713900000037
for the network Net1A pseudo-label of the sharpening is generated,
Figure BDA0003177713900000038
for the network Net2The sharpened pseudo-label, λ, obeys a probability distribution set from the actual data set.
And fifthly, constructing loss functions of training labeled data and unlabeled data by using the forward propagation result and the pseudo label, and performing semi-supervised training on the twin network:
step 501, establishing a semi-supervised labeling consistency regularization loss function, calculating a regularization term of the difference between labeled data and unlabeled data according to each category, and eliminating the difference between labeled data of the same category and unlabeled data, as follows:
Figure BDA0003177713900000041
where num is the number of categories, xlabeled、xunlabeledMarking the depth features of the image marking data and the image unmarked data, wherein class-k is the kth class;
step 502, for the enhanced labeled data X', establishing a loss function as follows:
Figure BDA0003177713900000042
step 503, for the enhanced unmarked data U', establishing a loss function as follows:
Figure BDA0003177713900000043
wherein, | X '| equals to the number of samples in each batch, | U' | equals to K times the number of samples in each batch,
Figure BDA0003177713900000044
is a cross entropy function, x, p are enhanced labeled data and labels, and u, q are enhanced unlabeled data and pseudo labels.
Step 504, the overall loss function L is a weighting of the three, as follows:
L＝LX+λULU+βULosssemi-supervised
wherein λ isU、βUIs a hyper-parameter. And carrying out classification test on the trained twin network model by using the overall loss function L through continuous iteration.
Compared with the prior art, the invention has the following advantages:
1. the invention has simple structure, reasonable design and convenient realization, use and operation.
2. The method adopts the generation countermeasure network of the self-adaptive vision mechanism to carry out unsupervised learning on the data set, and carries out data set depth feature extraction on the trained model, so that the difference between the feature extraction of the labeled data and the feature extraction of the unlabeled data among different classes can be effectively eliminated, the feature extraction and selection have higher robustness, the integrity of image information is protected, and the semi-supervised multi-classification performance is improved.
3. The invention is based on the idea of label consistency, uses the twin network to carry out semi-supervised learning, can effectively eliminate the difference of characteristic discrimination between labeled data and non-labeled data among the same types, avoids different classification results caused by characteristic difference, has relatively small training parameter amount and has higher effectiveness and correctness.
In conclusion, the invention has simple structure and reasonable design. The method constructs a triple network to train a data set with insufficient annotation data, firstly establishes a generation confrontation network of a self-adaptive vision mechanism, and carries out unsupervised learning on an image data set for more effective feature extraction, thereby eliminating the difference between the feature extraction of heterogeneous annotation data and the feature extraction of non-annotation data; then, a twin network is established and trained based on the principle of label consistency, so that the difference of characteristic discrimination between similar labeled data and unlabeled data can be eliminated, the number of network training parameters is reduced, and the unlabeled data is more effectively utilized for semi-supervised learning.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may be otherwise variously oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
As shown in fig. 1, the present invention comprises the steps of:
step one, inputting an image data set and a corresponding label set:
step 101, an image data set V is input, in particular V ═ V1,...vi,...vlDividing the data into marked data X ═ X1,...xf,...xnAnd un-annotated data U ═ U1,...uj,...umIn which v isiRepresenting ith image sample data, i is more than or equal to 1 and less than or equal to l, f is more than or equal to 1 and less than or equal to n, j is more than or equal to 1 and less than or equal to m, l is m + n, and n, m and l are positive integers;
step 102, inputting a label set corresponding to the image set V, wherein the label data X is { X ═ X }1,...xi,...xnThe label of is p ═ p1,...pi,...pnU ═ U }, no-mark data U ═ U1,...uj,...umThere is no label.
Step two, preprocessing the labeled and unlabeled data sets:
step 201, performing data enhancement on the image labeled data X and the unlabeled data U, wherein the labeled data is subjected to single enhancement to obtain enhanced data X'. For the data without the label, performing random enhancement for K times to obtain enhanced data U';
step 202, mixing the data X 'and the data U', and randomly arranging to obtain a data combination W, wherein the label of the enhanced data is consistent with the original label.
Step three, constructing and training a depth network of the self-adaptive visual mechanism, and extracting the depth characteristics of the image data set:
step 301, constructing a generation countermeasure network G, which is divided into a data generator and a discriminator, wherein the generation countermeasure network uses DCGAN, and specifically adopts Resnet-18.
Step 302, using a self-convolution layer in the generation of the countermeasure network, and setting a self-adaptive convolution kernel generation function based on the principles of spatial specificity and frequency domain independence. Outputting convolution kernels with the same size as the feature map according to the input image features, controlling the scaling adjustment parameter, and scaling the feature map channels by using a 1x1 convolution kernel to obtain the feature map, wherein the number of output feature channels is (Z x Z Gs), Z is the size of the subsequent self-convolution kernel, and Gs represents the grouping number of self-convolution operation.
And 303, deleting labels of the labeled data in the image set V, carrying out unsupervised learning on the countermeasure network G by using all the image sets V without labels, enabling the pseudo data characteristics generated by the generator to be close to real image characteristics, and enhancing the characteristic representation capability of the discriminator by using the self-convolution layer.
Step 304, generating the trained discriminator G of the countermeasure network GdRemoving the full connection layer, and retaining the convolution layer as the feature extractor FdAnd is used for extracting target image labeling data X ═ X1,...xf,...xnAnd un-annotated data U ═ U1,...uj,...umDepth feature x oflabeled＝Fd(xf) And xunlabeled＝Fd(uj)。
Step four, constructing a twin network, and acquiring a forward propagation result and a pseudo label by using the depth characteristics of the marked data and the unmarked data:
step 401, constructing two shallow classification networks Net1And Net2Inputting a data combination W as a twin network, wherein the shallow classification network uses VGG-11;
step 402, inputting the enhanced data X' and the corresponding label p for the labeled data to obtain the depth feature Xlabeled＝Fd(X'), prediction is carried out by using twin network, and forward propagation results are
Figure BDA0003177713900000071
Wherein p isd1And pd2Is Net1And Net2W is a hyperparameter;
step 403, inputting the enhanced data U' for the unmarked data to obtain the depth feature xunlabeled＝Fd(U') predicting using twin networks, and taking the weighted average of the outputs as the forward propagation result
Figure BDA0003177713900000072
Wherein,
Figure BDA0003177713900000073
and
Figure BDA0003177713900000075
and theta is a network training parameter for the prediction of the twin network on the unmarked data.
And step 404, sharpening the prediction of the label-free data to obtain a pseudo label q. Wherein the sharpening operation is specifically
Figure BDA0003177713900000081
T is the sharpening parameter, K is the number of enhancements, and P (U; θ) is the prediction probability of the network for each class.
And 405, performing label fusion on the pseudo label q predicted by the twin network. Specifically, the fused pseudo tag is:
Figure BDA0003177713900000082
wherein,
Figure BDA0003177713900000083
for the network Net1A pseudo-label of the sharpening is generated,
Figure BDA0003177713900000084
for the network Net2The sharpened pseudo-label, λ -Beta (α, α), α sets the probability distribution of obedience according to the actual dataset.
And fifthly, constructing loss functions of training labeled data and unlabeled data by using the forward propagation result and the pseudo label, and performing semi-supervised training on the twin network:
step 501, establishing a semi-supervised labeling consistency regularization loss function, calculating a regularization term of the difference between labeled data and unlabeled data according to each category, and eliminating the difference between labeled data of the same category and unlabeled data, as follows:
Figure BDA0003177713900000085
where num is the number of categories, xlabeled、xunlabeledMarking the depth features of the image marking data and the image unmarked data, wherein class-k is the kth class;
step 502, for the enhanced labeled data X', establishing a loss function as follows:
Figure BDA0003177713900000086
step 503, for the enhanced unmarked data U', establishing a loss function as follows:
Figure BDA0003177713900000087
wherein, | X '| equals to the number of samples in each batch, | U' | equals to K times the number of samples in each batch,
Figure BDA0003177713900000088
is a cross entropy function, x, p are enhanced labeled data and labels, and u, q are enhanced unlabeled data and pseudo labels.
Step 504, the overall loss function L is a weighting of the three, as follows:
L＝LX+λULU+βULosssemi-sup ervised
wherein λ isU、βUIs a hyper-parameter. And carrying out classification test on the trained twin network model by using the overall loss function L through continuous iteration.
The above embodiments are only examples of the present invention, and are not intended to limit the present invention, and all simple modifications, changes and equivalent structural changes made to the above embodiments according to the technical spirit of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims (1)

1. A semi-supervised learning method based on a triple network and labeling consistency regularization is characterized by comprising the following steps:
step one, inputting an image data set and a corresponding label set:
step 101, an image data set V is input, in particular V ═ V1,...vi,...vlDividing the data into marked data X ═ X1,...xf,...xnAnd un-annotated data U ═ U1,...uj,...umIn which v isiRepresenting ith image sample data, i is more than or equal to 1 and less than or equal to l, f is more than or equal to 1 and less than or equal to n, j is more than or equal to 1 and less than or equal to m, l is m + n, and n, m and l are positive integers;
step 102, inputting a label set corresponding to the image set V, wherein the label data X is { X ═ X }1,...xi,...xnThe label of is p ═ p1,...pi,...pnU ═ U }, no-mark data U ═ U1,...uj,...umThere is no label.
Step two, preprocessing the labeled and unlabeled data sets:
step 201, performing data enhancement on the image labeled data X and the unlabeled data U, wherein the labeled data is subjected to single enhancement to obtain enhanced data X'. For the data without the label, performing random enhancement for K times to obtain enhanced data U';
step 202, mixing the data X 'and the data U', and randomly arranging to obtain a data combination W, wherein the label of the enhanced data is consistent with the original label.
Step three, constructing and training a depth network of the self-adaptive visual mechanism, and extracting the depth characteristics of the image data set:
step 301, constructing and generating the countermeasure network G, which is divided into a data generator and a discriminator.
Step 302, using a self-convolution layer in the generation of the countermeasure network, and setting a self-adaptive convolution kernel generation function based on the principles of spatial specificity and frequency domain independence. And outputting a convolution kernel with the same size as the feature map according to the input image features, controlling the scaling ratio to adjust the parameter quantity, and scaling the feature map channel.
And 303, deleting labels of the labeled data in the image set V, carrying out unsupervised learning on the countermeasure network G by using all the image sets V without labels, enabling the pseudo data characteristics generated by the generator to be close to real image characteristics, and enhancing the characteristic representation capability of the discriminator by using the self-convolution layer.
Step 304, generating the trained discriminator G of the countermeasure network GdAs a feature extractor FdAnd is used for extracting target image labeling data X ═ X1,...xf,...xnAnd un-annotated data U ═ U1,...uj,...umDepth feature x oflabeled＝Fd(xf) And xunlabeled＝Fd(uj)。
Step four, constructing a twin network, and acquiring a forward propagation result and a pseudo label by using the depth characteristics of the marked data and the unmarked data:
step 401, constructing two shallow classification networks Net1And Net2As a twin network, a data combination W is input;
step 402, inputting the enhanced data X' and the corresponding label p for the labeled data to obtain the depth feature Xlabeled＝Fd(X'), prediction is carried out by using twin network, and forward propagation results are
Figure FDA0003177713890000021
Wherein p isd1And pd2Is Net1And Net2W is a hyperparameter;
step 403, inputting the enhanced data U' for the unmarked data to obtain the depth feature xunlabeled＝Fd(U') predicting using twin networks, and taking the weighted average of the outputs as the forward propagation result pn，
Figure FDA0003177713890000022
Wherein,
Figure FDA0003177713890000023
and
Figure FDA0003177713890000024
and theta is a network training parameter for the prediction of the twin network on the unmarked data.
And step 404, sharpening the prediction of the label-free data to obtain a pseudo label q. Wherein the sharpening operation is specifically
Figure FDA0003177713890000025
T is the sharpening parameter, K is the number of enhancements, and P (U; θ) is the prediction probability of the network for each class.
And 405, performing label fusion on the pseudo label q predicted by the twin network. Specifically, the fused pseudo tag is:
Figure FDA0003177713890000026
wherein,
Figure FDA0003177713890000027
for the network Net1A pseudo-label of the sharpening is generated,
Figure FDA0003177713890000028
for the network Net2The sharpened pseudo-label, λ, obeys a probability distribution set from the actual data set.
And fifthly, constructing loss functions of training labeled data and unlabeled data by using the forward propagation result and the pseudo label, and performing semi-supervised training on the twin network:
step 501, establishing a semi-supervised labeling consistency regularization loss function, calculating a regularization term of the difference between labeled data and unlabeled data according to each category, and eliminating the difference between labeled data of the same category and unlabeled data, as follows:
Figure FDA0003177713890000031
where num is the number of categories, xlabeled、xunlabeledMarking the depth features of the image marking data and the image unmarked data, wherein class-k is the kth class;
step 502, for the enhanced labeled data X', establishing a loss function as follows:
Figure FDA0003177713890000032
step 503, for the enhanced unmarked data U', establishing a loss function as follows:
Figure FDA0003177713890000033
wherein, | X '| equals to the number of samples in each batch, | U' | equals to K times the number of samples in each batch,
Figure FDA0003177713890000034
is a cross entropy function, x, p are enhanced labeled data and labels, and u, q are enhanced unlabeled data and pseudo labels.
Step 504, the overall loss function L is a weighting of the three, as follows:
L＝LX+λULU+βULosssemi-supervised
wherein λ isU、βUIs a hyper-parameter. And carrying out classification test on the trained twin network model by using the overall loss function L through continuous iteration.
CN202110837568.9A 2021-07-23 2021-07-23 Semi-supervised learning method based on triple play network and labeling consistency regularization Active CN113657455B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110837568.9A CN113657455B (en) 2021-07-23 2021-07-23 Semi-supervised learning method based on triple play network and labeling consistency regularization

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110837568.9A CN113657455B (en) 2021-07-23 2021-07-23 Semi-supervised learning method based on triple play network and labeling consistency regularization

Publications (2)

Publication Number Publication Date
CN113657455A true CN113657455A (en) 2021-11-16
CN113657455B CN113657455B (en) 2024-02-09

Family

ID=78477710

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110837568.9A Active CN113657455B (en) 2021-07-23 2021-07-23 Semi-supervised learning method based on triple play network and labeling consistency regularization

Country Status (1)

Country Link
CN (1) CN113657455B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114048536A (en) * 2021-11-18 2022-02-15 重庆邮电大学 Road structure prediction and target detection method based on multitask neural network
CN114494973A (en) * 2022-02-14 2022-05-13 中国科学技术大学 Training method, system, equipment and storage medium of video semantic segmentation network
CN114612685A (en) * 2022-03-22 2022-06-10 中国科学院空天信息创新研究院 Self-supervision information extraction method combining depth features and contrast learning
CN114648077A (en) * 2022-05-18 2022-06-21 合肥高斯智能科技有限公司 Method and device for multi-point industrial data defect detection
CN114742119A (en) * 2021-12-30 2022-07-12 浙江大华技术股份有限公司 Cross-supervised model training method, image segmentation method and related equipment
CN114781526A (en) * 2022-04-26 2022-07-22 西安理工大学 Depth semi-supervised image classification method based on discriminant feature learning and entropy
CN115792807A (en) * 2023-02-13 2023-03-14 北京理工大学 Semi-supervised learning underwater sound source positioning method based on twin network
CN116403074A (en) * 2023-04-03 2023-07-07 上海锡鼎智能科技有限公司 Semi-automatic image labeling method and device based on active labeling
CN117649528A (en) * 2024-01-29 2024-03-05 山东建筑大学 Semi-supervised image segmentation method, system, electronic equipment and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112598053A (en) * 2020-12-21 2021-04-02 西北工业大学 Active significance target detection method based on semi-supervised learning
CN112837338A (en) * 2021-01-12 2021-05-25 浙江大学 Semi-supervised medical image segmentation method based on generation countermeasure network
KR20210071378A (en) * 2019-12-06 2021-06-16 인하대학교 산학협력단 Hierarchical object detection method for extended categories
US20220129735A1 (en) * 2019-05-20 2022-04-28 Institute of intelligent manufacturing, Guangdong Academy of Sciences Semi-supervised Hyperspectral Data Quantitative Analysis Method Based on Generative Adversarial Network

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220129735A1 (en) * 2019-05-20 2022-04-28 Institute of intelligent manufacturing, Guangdong Academy of Sciences Semi-supervised Hyperspectral Data Quantitative Analysis Method Based on Generative Adversarial Network
KR20210071378A (en) * 2019-12-06 2021-06-16 인하대학교 산학협력단 Hierarchical object detection method for extended categories
CN112598053A (en) * 2020-12-21 2021-04-02 西北工业大学 Active significance target detection method based on semi-supervised learning
CN112837338A (en) * 2021-01-12 2021-05-25 浙江大学 Semi-supervised medical image segmentation method based on generation countermeasure network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
WANG MIAO ET AL.: ""A SEMI-SUPERVISED SIAMESE NETWORK WITH LABEL FUSION FOR REMOTE SENSING IMAGE SCENE CLASSIFICATION"", 《 2021 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM IGARSS》 *
徐哲等: "联合训练生成对抗网络的半监督分类方法", 光学精密工程, vol. 29, no. 5 *
耿艳磊;邹峥嵘;何帅帅;: "基于半监督生成对抗网络的遥感影像地物语义分割", 测绘与空间地理信息, no. 04 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114048536A (en) * 2021-11-18 2022-02-15 重庆邮电大学 Road structure prediction and target detection method based on multitask neural network
CN114742119A (en) * 2021-12-30 2022-07-12 浙江大华技术股份有限公司 Cross-supervised model training method, image segmentation method and related equipment
CN114494973A (en) * 2022-02-14 2022-05-13 中国科学技术大学 Training method, system, equipment and storage medium of video semantic segmentation network
CN114494973B (en) * 2022-02-14 2024-03-29 中国科学技术大学 Training method, system, equipment and storage medium of video semantic segmentation network
CN114612685A (en) * 2022-03-22 2022-06-10 中国科学院空天信息创新研究院 Self-supervision information extraction method combining depth features and contrast learning
CN114781526A (en) * 2022-04-26 2022-07-22 西安理工大学 Depth semi-supervised image classification method based on discriminant feature learning and entropy
CN114648077B (en) * 2022-05-18 2022-09-06 合肥高斯智能科技有限公司 Method and device for multi-point industrial data defect detection
CN114648077A (en) * 2022-05-18 2022-06-21 合肥高斯智能科技有限公司 Method and device for multi-point industrial data defect detection
CN115792807A (en) * 2023-02-13 2023-03-14 北京理工大学 Semi-supervised learning underwater sound source positioning method based on twin network
CN116403074A (en) * 2023-04-03 2023-07-07 上海锡鼎智能科技有限公司 Semi-automatic image labeling method and device based on active labeling
CN116403074B (en) * 2023-04-03 2024-05-14 上海锡鼎智能科技有限公司 Semi-automatic image labeling method and device based on active labeling
CN117649528A (en) * 2024-01-29 2024-03-05 山东建筑大学 Semi-supervised image segmentation method, system, electronic equipment and storage medium
CN117649528B (en) * 2024-01-29 2024-05-31 山东建筑大学 Semi-supervised image segmentation method, system, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN113657455B (en) 2024-02-09

Similar Documents

Publication Publication Date Title
CN113657455A (en) Semi-supervised learning method based on triple network and labeling consistency regularization
Samek et al. Evaluating the visualization of what a deep neural network has learned
EP4027300B1 (en) Apparatus, program, and method for anomaly detection and classification
US20200027002A1 (en) Category learning neural networks
Deng et al. Unsupervised image segmentation using a simple MRF model with a new implementation scheme
CN110443293B (en) Zero sample image classification method for generating confrontation network text reconstruction based on double discrimination
Ju et al. Fusing global and local features for generalized ai-synthesized image detection
CN103942749B (en) A kind of based on revising cluster hypothesis and the EO-1 hyperion terrain classification method of semi-supervised very fast learning machine
CN112132149A (en) Semantic segmentation method and device for remote sensing image
Akhtar et al. Attack to fool and explain deep networks
CN116258719A (en) Flotation foam image segmentation method and device based on multi-mode data fusion
Reddy et al. AdaCrowd: Unlabeled scene adaptation for crowd counting
CN111639186A (en) Multi-class multi-label text classification model and device dynamically embedded with projection gate
CN111241933A (en) Pig farm target identification method based on universal countermeasure disturbance
Hada et al. Sparse oblique decision trees: A tool to understand and manipulate neural net features
Zhong et al. Automatic aurora image classification framework based on deep learning for occurrence distribution analysis: A case study of all‐sky image data sets from the yellow river station
Yu et al. Man-made object recognition from underwater optical images using deep learning and transfer learning
Bykov et al. DORA: Exploring outlier representations in deep neural networks
CN117456232A (en) Semi-supervised few-sample image classification method based on multi-scale features
Zhang et al. Energy-based generative cooperative saliency prediction
Cheng et al. Deeply explain CNN via hierarchical decomposition
Casini et al. Current trends on the use of deep learning methods for image analysis in energy applications
Hu et al. Meta-dm: Applications of diffusion models on few-shot learning
Gulshad et al. Hierarchical explanations for video action recognition
Hirata et al. Deep learning for nonlinear time series: examples for inferring slow driving forces

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant