CN113378942B - Small sample image classification method based on multi-head feature cooperation - Google Patents

Small sample image classification method based on multi-head feature cooperation Download PDF

Info

Publication number
CN113378942B
CN113378942B CN202110667364.5A CN202110667364A CN113378942B CN 113378942 B CN113378942 B CN 113378942B CN 202110667364 A CN202110667364 A CN 202110667364A CN 113378942 B CN113378942 B CN 113378942B
Authority
CN
China
Prior art keywords
feature
classifier
features
representing
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110667364.5A
Other languages
Chinese (zh)
Other versions
CN113378942A (en
Inventor
刘宝弟
兴雷
邵帅
刘伟锋
王延江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Petroleum East China
Original Assignee
China University of Petroleum East China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Petroleum East China filed Critical China University of Petroleum East China
Priority to CN202110667364.5A priority Critical patent/CN113378942B/en
Publication of CN113378942A publication Critical patent/CN113378942A/en
Application granted granted Critical
Publication of CN113378942B publication Critical patent/CN113378942B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/285—Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a small sample image classification method based on multi-head feature cooperation, which belongs to the technical field of pattern recognition, simultaneously uses embedded features extracted by a plurality of feature extractors, and introduces a subspace learning method to convert the original multi-head features into a uniform low-dimensional representation space, thereby being beneficial to reducing redundant information and effectively solving the problem that the measuring scale degrees of different embedded features are inconsistent when the different embedded features are in different feature spaces. In addition, the combined weight of each multi-head feature is automatically updated by designing a weight calculation part, and the processed multi-head embedded features are cascaded to obtain the cooperative representation of the sample, so that the problem of reasonable use of the multi-head features is effectively solved.

Description

Small sample image classification method based on multi-head feature cooperation
Technical Field
The invention relates to the technical field of pattern recognition, in particular to a small sample image classification method based on multi-head feature cooperation.
Background
Inspired by human cognitive learning, scholars put forward the problem of small sample image classification, and after learning a large number of samples of limited classes, the scholars can quickly and accurately learn by using a small amount of sample data when encountering new classes by using priori knowledge. In recent years, problems associated with small sample learning have become a new important research direction in the field of machine learning, and are considered as one of the development directions of next-generation artificial intelligence.
At present, the main small sample image classification methods include the following methods:
(1) the small sample image classification method based on data expansion comprises the following steps: a small sample image classification method based on data expansion is proposed in 2018, a new sample data set is generated from an original data set by using a generated countermeasure network, and in order to solve the problem that sample data is insufficient during training of small sample image classification of the generated countermeasure network, a generator is designed to map sample data of a large probability class to sample data of a small probability class. The small sample image classification method has a plurality of specific advantages in pattern recognition through expanding data. However, the process of generating samples only ensures the correctness of the generated samples, and does not consider the distribution of the samples, which is very disadvantageous for the classification.
(2) The small sample image classification method based on the prototype network comprises the following steps: the small sample image classification method based on the prototype network is proposed in 2017 by Snell J and Swersky K, and uses the mean value of the features of each type of sample in a support set as one representation of the type, measures similarity through Euclidean distance, and predicts the label of an unknown sample. The method is simple and effective, and achieves better performance in a small sample image classification task. However, since the training data of the small sample image is very small, it is very difficult to estimate the sample distribution by using only the training sample, which may cause a certain deviation in the final classification.
(3) The small sample image classification method based on optimization comprises the following steps: the optimized small sample image classification method is proposed in 2017 by Ravi S and Larochelle H, and provides a framework for meta-learning and model independence, wherein the framework only has one weight initialization and can use any number of gradient step lengths to carry out self-adaptive learning, the model is trained by a random gradient descent method and is easier to fine tune to adapt to new sample data, and the model can be quickly trained on a small sample data set. But the testing phase does not adequately mine the value of the unlabeled exemplars.
Disclosure of Invention
In order to solve the problems existing in the image classification process of the small sample image classification method in the prior art, the embodiment of the invention provides a small sample image classification method based on multi-head feature cooperation. The technical scheme is as follows:
the invention provides a small sample image classification method based on multi-head feature cooperation, which comprises the following steps:
extracting image features by adopting a convolutional neural network;
training a classifier by directly optimizing a first objective function, and predicting the category of the test sample by using the classifier, wherein the first objective function is as follows:
Figure BDA0003117411190000021
wherein the content of the first and second substances,
Figure BDA0003117411190000022
diml and N denote the size and number of samples, C denotes the number of classes, xn，yn(N ═ 1, 2.., N) denotes NthThe embedded feature vector and the tag vector of the sample,
Figure BDA0003117411190000023
representing a classifier to be learned, | | · | luminanceFRepresentation regularization is carried out on (-) and mu represents the weight of a constraint term of a classifier W;
the classifier W is as follows:
W＝YXT(XXT+μI)-1
wherein, I represents an identity matrix;
the test sample characteristics
Figure BDA0003117411190000024
The categories of (A) are:
Figure BDA0003117411190000025
wherein max represents an operator for obtaining the index of the maximum value in the vector;
introducing a subspace learning method, reconstructing the original multi-head features to a uniform low-dimensional space, and obtaining new embedded features through the learning subspace;
solving the optimal weight combination in the new embedded features;
calculating a final cooperative characteristic through a first formula, calculating a final cooperative classifier and predicting the category of the cooperative characteristic according to the final cooperative classifier, wherein the first formula is as follows:
Figure BDA0003117411190000031
wherein the content of the first and second substances,
Figure BDA0003117411190000032
the final collaboration feature is represented as a result of,
Figure BDA0003117411190000033
represents PhAnd ZhThe nth feature of (1);
the final co-classifier is:
Wz＝YZT(ZZT+μI)-1
wherein the content of the first and second substances,
Figure BDA0003117411190000034
is the final collaborative classifier;
the categories of the predicted collaboration features are:
Figure BDA0003117411190000035
wherein the content of the first and second substances,
Figure BDA0003117411190000036
for testing sample characteristics
Figure BDA0003117411190000037
The collaboration feature of (1).
Expanding the small sample image feature classification of multi-head feature cooperation to semi-supervised setting, utilizing unlabelled data to strengthen a classifier, and utilizing an optimal classifier to predict the category of a query tag, wherein the category of the query tag is as follows:
Figure BDA0003117411190000038
wherein Z isqRepresenting collaboration features of the query set data.
Optionally, the solving of the optimal weight combination in the new embedded feature specifically includes: recalculating the loss of the first objective function on the h-th feature using the new embedded feature and the new classifier
Figure BDA0003117411190000039
Calculating an optimal weight combination using a second objective function, wherein the first objective function has a loss in the h-th feature
Figure BDA00031174111900000310
Comprises the following steps:
Figure BDA00031174111900000311
wherein, PhA new embedded feature is represented that is embedded in the image,
Figure BDA00031174111900000312
representing a new classifier;
the second objective function is:
Figure BDA00031174111900000313
Figure BDA00031174111900000314
wherein Ω is [ Ω ]1，Ω2，...，ΩH]TRepresents the optimal weight combination, ΩhWeight representing h-th feature, | | · |. non-woven phosphor2Is represented by2Regularization,. l2Expressing the squaring and root re-opening of all elements in the vector, wherein eta is a parameter;
the optimal weight of the h-th feature calculated by adopting the second objective function is as follows:
Figure BDA00031174111900000315
wherein the content of the first and second substances,
Figure BDA00031174111900000316
is the optimal weight of the h-th feature.
Optionally, the calculating the optimal weight combination by using the second objective function specifically includes: and introducing Lagrange quantity on the basis of the second objective function, and obtaining the optimal weight combination by adopting a Newton method.
Optionally, the classifying of the small sample image features of the multi-head feature cooperation is expanded to a semi-supervised setting, and the classifier is enhanced by using the unlabeled data, specifically:
training a basic classifier by using each feature of the support set data to obtain a classifier:
Figure BDA0003117411190000041
wherein the content of the first and second substances,
Figure BDA0003117411190000042
to represent
Figure BDA0003117411190000043
The hh feature of (1), wherein
Figure BDA0003117411190000044
And
Figure BDA0003117411190000045
respectively representing support set data, unlabeled data and query set data,
Figure BDA0003117411190000046
to represent
Figure BDA0003117411190000047
Is characterized in that it is a mixture of two or more of the above-mentioned components,
Figure BDA0003117411190000048
is a classifier obtained by training with support set data, YsIs a label matrix supporting the set data;
obtaining support set cooperation characteristics and support set cooperation classifiers by using each characteristic of support set data, and predicting label-free data by using a second formula
Figure BDA0003117411190000049
Wherein the second formula is:
Figure BDA00031174111900000410
wherein Z isuRepresenting the collaboration feature of the non-tagged data,
Figure BDA00031174111900000411
to represent
Figure BDA00031174111900000412
And ZuCollaboration feature of unlabeled exemplars, zunIs shown at ZuThe nth feature of (1), YpseudoA soft pseudo label representing an unlabeled data prediction;
and selecting a most reliable sample through a soft pseudo label predicted by label-free data, expanding the sample to a support set, and repeatedly training to obtain the optimal classifier with stable performance.
Predicting the category of the query label by using the optimal classifier, wherein the category of the query label is as follows:
Figure BDA00031174111900000413
wherein Z isqRepresenting collaboration features of the query set data.
The technical scheme provided by the embodiment of the invention has the following beneficial effects:
the small sample image classification method based on multi-head feature cooperation provided by the embodiment of the invention simultaneously uses the embedded features extracted by multiple feature extractors, and introduces a subspace learning method to convert the original multi-head features into a uniform low-dimensional representation space, which is simultaneously beneficial to reducing redundant information and effectively solves the problem of inconsistent measuring scale caused by different embedded features in different feature spaces. In addition, the combined weight of each multi-head feature is automatically updated by designing a weight calculation part, and the processed multi-head embedded features are cascaded to obtain the cooperative representation of the sample, so that the problem of reasonable use of the multi-head features is effectively solved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a small sample image classification method based on multi-head feature cooperation according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
The method for classifying small sample images based on multi-head feature cooperation according to the embodiment of the present invention will be described in detail below with reference to fig. 1.
Referring to fig. 1, a small sample image classification method based on multi-head feature collaboration according to an embodiment of the present invention includes:
step 110: and extracting image features by adopting a convolutional neural network.
And extracting image features by adopting a convolutional neural network model Resnet-12 model. Specifically, firstly, the image scale size is changed into 84x84 size, and then the Resnet-12 model is called to obtain the characteristics of the image to be processed. The process of extracting image features by using the convolutional neural network is not the protection content of the present invention, the process of extracting image features by using the convolutional neural network belongs to the prior art, and the process of extracting image features by using the convolutional neural network is a common image feature extraction method.
Step 120: training a classifier by directly optimizing a first objective function, predicting test samples using the classifier
Figure BDA0003117411190000051
The category (2).
Definition of
Figure BDA0003117411190000052
diml and N denote the size and number of samples, C denotes the number of classes, xn，yn(N ═ 1, 2.., N) denotes NthThe embedded feature vector and the tag vector of the sample,
Figure BDA0003117411190000053
representing a classifier to be learned;
using a first objective function
Figure BDA0003117411190000054
Training a classifier, wherein | L | · L calculationFRepresenting regularization on (-) and mu represents the weight of a constraint term of the classifier W;
the classifier W obtained by directly optimizing the first objective function is:
W＝YXT(XXT+μI)-1
wherein I represents an identity matrix;
predicting test samples
Figure BDA0003117411190000055
To obtain
Figure BDA0003117411190000056
The categories of (A) are:
Figure BDA0003117411190000061
where max represents the operator that obtains the index of the maximum value in the vector.
Step 130: and (3) introducing a subspace learning method, reconstructing the original multi-head features to a uniform low-dimensional space, and obtaining new embedded features through the learning subspace.
Defining a total of H sample embedding features, xhDenotes H, where H is 1, 2, …, H.
Introduced subspace learning approach (denoted as
Figure BDA0003117411190000062
) Reconstructing the original features into a uniform low-dimensional space, regarding the H features of the same sample as the sample, and expressing the features of the extended data set as the samples
Figure BDA0003117411190000063
Performing subspace learning operations
Figure BDA0003117411190000064
Obtaining new embedded features
Figure BDA0003117411190000065
Wherein the content of the first and second substances,
Figure BDA0003117411190000066
representing the h-th feature after subspace transformation, dim2 represents the dimension of the feature after subspace transformation.
Step 140: and solving the optimal weight combination in the new embedded features.
Of different characteristicsThe importance is different, and the optimal weight combination omega is found to be [ omega ]1，Ω2，...，ΩH]TLet these features have different influence on the final decision, where Ω denotes a weight vector, and Ωh(H ═ 1, 2, …, H) represents the H-th element in Ω.
Using the converted features PhReplacement of xhObtaining
Figure BDA0003117411190000067
A new classifier of
Figure BDA0003117411190000068
According to the formula W ═ YX in step 120T(XXT+μI)-1Calculating to obtain a new classifier
Figure BDA0003117411190000069
Using a new embedded feature PhAnd a new classifier
Figure BDA00031174111900000610
Recalculating the first objective function
Figure BDA00031174111900000611
Loss in h-th characteristic
Figure BDA00031174111900000612
The calculation result is as follows:
Figure BDA00031174111900000613
calculating an optimal weight combination by adopting a second objective function, wherein the second objective function is as follows:
Figure BDA00031174111900000614
Figure BDA00031174111900000615
wherein | · | charging2Is represented by2Regularization,. l2Expressing the squaring and root re-opening of all elements in the vector, wherein eta is a parameter;
introducing a lagrangian quantity, the second objective function is rewritten as:
Figure BDA00031174111900000616
where ζ is a constant and Λ ═ Λ1，Λ2，...，ΛH]TIs a vector.
The above equation (1) is rewritten into a matrix form as follows:
Figure BDA0003117411190000071
wherein the content of the first and second substances,
Figure BDA0003117411190000072
suppose that
Figure BDA0003117411190000073
Is an optimal solution according to the Karush-Kuhn-Tucker (KKT) condition in
Figure BDA0003117411190000074
Obtaining:
Figure BDA0003117411190000075
the above equation (2) is rewritten as follows:
Figure BDA0003117411190000076
solving for
Figure BDA0003117411190000077
The procedure of (2) is as follows:
Figure BDA0003117411190000078
solving for
Figure BDA0003117411190000079
The procedure of (2) is as follows:
Figure BDA00031174111900000710
order to
Figure BDA00031174111900000711
Wherein
Figure BDA00031174111900000712
Is that
Figure BDA00031174111900000713
The above formula (3) is rewritten as follows:
Figure BDA00031174111900000714
optimization of the h-th feature
Figure BDA00031174111900000715
Expressed as:
Figure BDA00031174111900000716
order to
Figure BDA0003117411190000081
The above formula (4) is rewritten as:
Figure BDA0003117411190000082
in conjunction with the above equation (2), the above equation (5) is rewritten as:
Figure BDA0003117411190000083
therefore, the temperature of the molten metal is controlled,
Figure BDA0003117411190000084
the rewrite is:
Figure BDA0003117411190000085
order to
Figure BDA0003117411190000086
Combining the second objective function, the above equation (6), and the above equation (7), obtain:
Figure BDA0003117411190000087
combining Newton's method to obtain:
Figure BDA0003117411190000088
wherein f' (. cndot.) represents a derivative function of f (. cndot.), t is an iteration number, and the optimal solution hat Lambda can be obtained through t iterationavg。
The optimal weight of the h-th feature is obtained:
Figure BDA0003117411190000089
step 150: calculating a final cooperative characteristic through a first formula, calculating a final cooperative classifier and predicting the category of the cooperative characteristic according to the final cooperative classifier, wherein the first formula is as follows:
Figure BDA00031174111900000810
wherein the content of the first and second substances,
Figure BDA00031174111900000811
the final collaboration feature is represented as such,
Figure BDA00031174111900000812
represents PhAnd ZhThe nth feature of (1);
according to the formula W ═ YX in step 120T(XXT+μI)-1And replacing x with Z to obtain the final collaborative classifier
Figure BDA00031174111900000813
Expression (c):
Wz＝YZT(ZZT+μI)-1
obtaining the characteristics of the test sample by the first formula
Figure BDA0003117411190000091
Of
Figure BDA0003117411190000092
Predicting collaboration features
Figure BDA0003117411190000093
The categories of (A) are:
Figure BDA0003117411190000094
step 160: expanding the small sample image feature classification of multi-head feature cooperation to semi-supervised setting, utilizing unlabelled data to strengthen a classifier, and utilizing an optimal classifier to predict the category of a query tag, wherein the category of the query tag is as follows:
Figure BDA0003117411190000095
wherein Z isqRepresenting collaboration features of the query set data.
Definition of
Figure BDA0003117411190000096
Is characterized by
Figure BDA0003117411190000097
Wherein, the first and the second end of the pipe are connected with each other,
Figure BDA0003117411190000098
and
Figure BDA0003117411190000099
respectively represent support set data, unlabeled data, and query set data, and thus
Figure BDA00031174111900000910
Is characterized by being defined as
Figure BDA00031174111900000911
According to different use data, the design of the current classifier is divided into induction setting, semi-supervision setting and conversion setting, wherein the semi-supervision setting adopts a support set
Figure BDA00031174111900000912
And tagless collections
Figure BDA00031174111900000913
The classifier is trained and then the query label is predicted.
The method comprises the following steps of expanding the small sample image feature classification of multi-head feature cooperation to semi-supervised setting, and utilizing unlabelled data to strengthen a classifier, wherein the method specifically comprises the following steps:
training a basic classifier by using each feature of the support set data to obtain a classifier:
Figure BDA00031174111900000914
wherein the content of the first and second substances,
Figure BDA00031174111900000915
is a classifier obtained by training with support set data, YsIs a label matrix supporting the set data;
the combined weight for each feature is calculated according to the formula in step 140:
Figure BDA00031174111900000916
obtaining the collaborative features of the support set and the classifier according to the formula in step 150:
Figure BDA00031174111900000917
wherein the content of the first and second substances,
Figure BDA00031174111900000918
to represent
Figure BDA00031174111900000919
And ZsSupporting the cooperative property of set samples, zsnIs shown at ZsThe nth feature of (1).
Support set cooperation feature zsn and support set cooperation classifier W are obtained by using each feature of support set datazUsing a second formula
Figure BDA0003117411190000101
Predicting unlabeled data
Figure BDA0003117411190000102
Wherein Z isuRepresenting the collaboration feature of the non-tagged data,
Figure BDA0003117411190000103
to represent
Figure BDA0003117411190000104
And ZuCollaboration feature of unlabeled exemplars, zunIs shown at ZuThe nth feature of (1), YpseudoSoft pseudo labels representing unlabeled data predictions;
soft pseudo label Y predicted by unlabeled datapseudoSelecting a most reliable sample, and defining a corresponding pseudo label and characteristic as YSelectAnd pSelectExtension to support set acquisition
Figure BDA0003117411190000105
And repeating the training to obtain the optimal classifier with stable performance.
By the formula
Figure BDA0003117411190000106
Obtaining a collaborative embedding feature of the query data, wherein,
Figure BDA0003117411190000107
to represent
Figure BDA0003117411190000108
And ZqCollaborative properties of query set samples, zqnIs shown at ZqThe nth feature of (1).
Predicting the category of the query label by using the optimal classifier, wherein the category of the query label is as follows:
Figure BDA0003117411190000109
the small sample image classification method based on multi-head feature cooperation provided by the embodiment of the invention simultaneously uses the embedded features extracted by multiple feature extractors, and introduces a subspace learning method to convert the original multi-head features into a uniform low-dimensional representation space, which is simultaneously beneficial to reducing redundant information and effectively solves the problem of inconsistent measuring scale caused by different embedded features in different feature spaces. In addition, the combined weight of each multi-head feature is automatically updated by designing a weight calculation part, and the processed multi-head embedded features are cascaded to obtain the cooperative representation of the sample, so that the problem of reasonable use of the multi-head features is effectively solved.

Claims (4)

1. A small sample image classification method based on multi-head feature collaboration is characterized by comprising the following steps:
extracting image features by adopting a convolutional neural network;
training a classifier by directly optimizing a first objective function, and predicting the class of the test sample by using the classifier, wherein the first objective function is as follows:
Figure FDA0003656665890000011
wherein the content of the first and second substances,
Figure FDA0003656665890000012
dim1 and N denote the size and number of samples, respectively, C denotes the number of classes, xn，yn(N ═ 1, 2.., N) denotes NthThe embedded feature vector and the tag vector of the sample,
Figure FDA0003656665890000013
represents a classifier to be learned, | · |. non-woven phosphorFRepresents regularization of (-) and μ represents the weight of the classifier W constraint term:
the classifier W is as follows:
W＝YXT(XXT+μI)-1
wherein I represents an identity matrix;
the test sample characteristics
Figure FDA0003656665890000014
The categories of (1) are:
Figure FDA0003656665890000015
wherein max represents an operator for obtaining the index of the maximum value in the vector;
introducing a subspace learning method, reconstructing the original multi-head features to a uniform low-dimensional space, and obtaining new embedded features through the learning subspace; wherein the multi-head feature represents an image embedding feature extracted by simultaneously using a plurality of feature extractors;
solving the optimal weight combination in the new embedded features;
calculating a final cooperative characteristic through a first formula, calculating a final cooperative classifier and predicting the category of the cooperative characteristic according to the final cooperative classifier, wherein the first formula is as follows:
Figure FDA0003656665890000016
wherein the content of the first and second substances,
Figure FDA0003656665890000017
the final collaboration feature is represented as such,
Figure FDA0003656665890000018
represents PhAnd ZhThe nth feature of (1); omega-omega1，Ω2，...，ΩH]TWhich represents the optimal combination of weights, and,
Figure FDA0003656665890000019
weight of H-th feature, H number of kinds of features, PnRepresenting the embedded features of the nth image,
Figure FDA00036566658900000110
to
Figure FDA00036566658900000115
Is shown and
Figure FDA00036566658900000112
to
Figure FDA00036566658900000116
The optimal combining weights are in a one-to-one correspondence,
Figure FDA00036566658900000114
an nth feature representing an H-th feature;
the final co-classifier is:
Wz＝YZT(ZZT+μI)-1
wherein the content of the first and second substances,
Figure FDA0003656665890000021
is the final co-classifier;
the categories of the predicted collaboration features are:
Figure FDA0003656665890000022
wherein the content of the first and second substances,
Figure FDA0003656665890000023
for testing sample characteristics
Figure FDA0003656665890000024
A collaboration feature of (1);
expanding the small sample image feature classification of multi-head feature cooperation to semi-supervised setting, utilizing unlabelled data to strengthen a classifier, and utilizing an optimal classifier to predict the category of a query tag, wherein the category of the query tag is as follows:
Figure FDA0003656665890000025
wherein Z isqRepresenting collaboration features of the query set data.
2. The image classification method according to claim 1, wherein the solving for the optimal weight combination in the new embedded features specifically comprises: recalculating the loss of the first objective function on the h-th feature using the new embedded feature and the new classifier
Figure FDA0003656665890000026
Calculating an optimal weight combination using a second objective function, wherein the first objective function has a loss in the h-th feature
Figure FDA0003656665890000027
Comprises the following steps:
Figure FDA0003656665890000028
wherein, PhA new embedded feature is represented that is embedded in,
Figure FDA0003656665890000029
representing a new classifier;
the second objective function is:
Figure FDA00036566658900000210
Figure FDA00036566658900000211
wherein Ω is [ Ω ]1，Ω2，...，ΩH]TRepresents the optimal weight combination, ΩhWeight representing h-th feature, | | · |. non-woven phosphor2Is represented by2Regularization,. l2Representing all elements in a pair vectorSolving the square sum and then opening the root number, wherein eta is a parameter;
the optimal weight of the h-th feature calculated by adopting the second objective function is as follows:
Figure FDA00036566658900000212
wherein the content of the first and second substances,
Figure FDA00036566658900000213
is the optimal weight of the H-th feature, H represents the number of kinds of the features, FhRepresenting the loss of function for the h-th feature, Λ ═ Λ1，Λ2，...，ΛH]TA vector is represented that is a function of,
Figure FDA00036566658900000214
the optimal solution for a is represented as,
Figure FDA00036566658900000215
to represent
Figure FDA00036566658900000216
Average value of (a).
3. The image classification method according to claim 2, wherein the calculating of the optimal weight combination by using the second objective function is specifically: and introducing Lagrange quantity on the basis of the second objective function, and obtaining the optimal weight combination by adopting a Newton method.
4. The image classification method according to claim 1, wherein the small sample image feature classification with multi-head feature collaboration is extended to a semi-supervised setting, and a classifier is enhanced by using unlabeled data, specifically:
training a basic classifier by using each feature of the support set data to obtain a classifier:
Figure FDA0003656665890000031
wherein the content of the first and second substances,
Figure FDA0003656665890000032
to represent
Figure FDA0003656665890000033
The h feature of (1), wherein
Figure FDA0003656665890000034
And
Figure FDA0003656665890000035
respectively representing support set data, unlabeled data and query set data,
Figure FDA0003656665890000036
to represent
Figure FDA0003656665890000037
Is characterized in that it is a mixture of two or more of the above-mentioned components,
Figure FDA0003656665890000038
is a classifier obtained by training with support set data, YsIs a matrix of labels that supports the set data,
Figure FDA0003656665890000039
representing test set data;
obtaining support set cooperation characteristics and support set cooperation classifier by using each characteristic of support set data, and predicting label-free data by using a second formula
Figure FDA00036566658900000310
Wherein the second formula is:
Figure FDA00036566658900000311
wherein Z isuRepresenting the collaboration feature of the non-tagged data,
Figure FDA00036566658900000312
to represent
Figure FDA00036566658900000313
And ZuCollaboration feature of unlabeled exemplars, zunIs shown at ZuThe nth feature of (1), YpseudoSoft pseudo labels representing unlabeled data predictions;
selecting a most credible sample through a soft pseudo label predicted by label-free data, expanding the sample to a support set, and repeatedly training to obtain an optimal classifier with stable performance;
predicting the category of a query label by using an optimal classifier, wherein the category of the query label is as follows:
Figure FDA00036566658900000314
wherein Z isqRepresenting collaboration features of the query set data.
CN202110667364.5A 2021-06-16 2021-06-16 Small sample image classification method based on multi-head feature cooperation Active CN113378942B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110667364.5A CN113378942B (en) 2021-06-16 2021-06-16 Small sample image classification method based on multi-head feature cooperation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110667364.5A CN113378942B (en) 2021-06-16 2021-06-16 Small sample image classification method based on multi-head feature cooperation

Publications (2)

Publication Number Publication Date
CN113378942A CN113378942A (en) 2021-09-10
CN113378942B true CN113378942B (en) 2022-07-01

Family

ID=77572839

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110667364.5A Active CN113378942B (en) 2021-06-16 2021-06-16 Small sample image classification method based on multi-head feature cooperation

Country Status (1)

Country Link
CN (1) CN113378942B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114782752B (en) * 2022-05-06 2023-09-05 兰州理工大学 Small sample image integrated classification method and device based on self-training

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104166982A (en) * 2014-06-30 2014-11-26 复旦大学 Image optimization clustering method based on typical correlation analysis
CN106663037A (en) * 2014-06-30 2017-05-10 亚马逊科技公司 Feature processing tradeoff management
CN106909946A (en) * 2017-03-02 2017-06-30 深圳明创自控技术有限公司 A kind of picking system of multi-modal fusion
CN109359694A (en) * 2018-10-24 2019-02-19 中国石油大学(华东) A kind of image classification method and device of the classifier indicated based on mixing collaboration
CN111783831A (en) * 2020-05-29 2020-10-16 河海大学 Complex image accurate classification method based on multi-source multi-label shared subspace learning
CN112101381A (en) * 2020-08-30 2020-12-18 西南电子技术研究所（中国电子科技集团公司第十研究所） Tensor collaborative drawing discriminant analysis remote sensing image feature extraction method
CN112232438A (en) * 2020-11-05 2021-01-15 华东理工大学 High-dimensional image representation-oriented multi-kernel subspace learning framework
CN112819110A (en) * 2021-04-19 2021-05-18 中国科学院自动化研究所 Incremental small sample target detection method and system based on weight generation

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8224042B2 (en) * 2009-03-12 2012-07-17 Seiko Epson Corporation Automatic face recognition
CN111931795B (en) * 2020-09-25 2020-12-25 湖南大学 Multi-modal emotion recognition method and system based on subspace sparse feature fusion

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104166982A (en) * 2014-06-30 2014-11-26 复旦大学 Image optimization clustering method based on typical correlation analysis
CN106663037A (en) * 2014-06-30 2017-05-10 亚马逊科技公司 Feature processing tradeoff management
CN106909946A (en) * 2017-03-02 2017-06-30 深圳明创自控技术有限公司 A kind of picking system of multi-modal fusion
CN109359694A (en) * 2018-10-24 2019-02-19 中国石油大学(华东) A kind of image classification method and device of the classifier indicated based on mixing collaboration
CN111783831A (en) * 2020-05-29 2020-10-16 河海大学 Complex image accurate classification method based on multi-source multi-label shared subspace learning
CN112101381A (en) * 2020-08-30 2020-12-18 西南电子技术研究所（中国电子科技集团公司第十研究所） Tensor collaborative drawing discriminant analysis remote sensing image feature extraction method
CN112232438A (en) * 2020-11-05 2021-01-15 华东理工大学 High-dimensional image representation-oriented multi-kernel subspace learning framework
CN112819110A (en) * 2021-04-19 2021-05-18 中国科学院自动化研究所 Incremental small sample target detection method and system based on weight generation

Also Published As

Publication number Publication date
CN113378942A (en) 2021-09-10

Similar Documents

Publication Publication Date Title
Zhu et al. Multi-attention Meta Learning for Few-shot Fine-grained Image Recognition.
CN107122375B (en) Image subject identification method based on image features
Breitenbach et al. Clustering through ranking on manifolds
CN104966105A (en) Robust machine error retrieving method and system
CN109492673A (en) A kind of unbalanced data prediction technique based on spectral clustering sampling
CN113128478B (en) Model training method, pedestrian analysis method, device, equipment and storage medium
CN111639540A (en) Semi-supervised character re-recognition method based on camera style and human body posture adaptation
CN110705591A (en) Heterogeneous transfer learning method based on optimal subspace learning
CN104091038A (en) Method for weighting multiple example studying features based on master space classifying criterion
CN110175235A (en) Intelligence commodity tax sorting code number method and system neural network based
CN112364893B (en) Semi-supervised zero-sample image classification method based on data enhancement
CN113378942B (en) Small sample image classification method based on multi-head feature cooperation
CN112766400A (en) Semi-supervised classification integration method for high-dimensional data based on multiple data transformation spaces
CN117308077A (en) Water supply control system for evaporator of nuclear power unit of reactor
CN114283083B (en) Aesthetic enhancement method of scene generation model based on decoupling representation
CN111027582A (en) Semi-supervised feature subspace learning method and device based on low-rank graph learning
CN114329124A (en) Semi-supervised small sample classification method based on gradient re-optimization
CN111950619B (en) Active learning method based on dual-generation countermeasure network
CN113837046A (en) Small sample remote sensing image scene classification method based on iterative feature distribution learning
CN110288002B (en) Image classification method based on sparse orthogonal neural network
Osumi et al. Domain adaptation using a gradient reversal layer with instance weighting
CN113835964B (en) Cloud data center server energy consumption prediction method based on small sample learning
CN107045727B (en) Texture synthesis method and device
CN113378941B (en) Multi-decision fusion small sample image classification method
CN110045691A (en) A kind of multitasking fault monitoring method of multi-source heterogeneous big data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant