CN111858991A - Small sample learning algorithm based on covariance measurement - Google Patents
Small sample learning algorithm based on covariance measurement Download PDFInfo
- Publication number
- CN111858991A CN111858991A CN202010783893.7A CN202010783893A CN111858991A CN 111858991 A CN111858991 A CN 111858991A CN 202010783893 A CN202010783893 A CN 202010783893A CN 111858991 A CN111858991 A CN 111858991A
- Authority
- CN
- China
- Prior art keywords
- covariance
- local
- representation
- small sample
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005259 measurement Methods 0.000 title claims abstract description 33
- 238000012549 training Methods 0.000 claims abstract description 34
- 238000002474 experimental method Methods 0.000 claims abstract description 11
- 230000007246 mechanism Effects 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims description 45
- 239000011159 matrix material Substances 0.000 claims description 21
- 238000012360 testing method Methods 0.000 claims description 16
- 230000008569 process Effects 0.000 claims description 11
- 239000013598 vector Substances 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 2
- 230000002860 competitive effect Effects 0.000 abstract description 3
- 230000000052 comparative effect Effects 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 5
- 238000013508 migration Methods 0.000 description 4
- 230000005012 migration Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000013434 data augmentation Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- 230000001154 acute effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012733 comparative method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000001667 episodic effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Pure & Applied Mathematics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Algebra (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a small sample learning algorithm based on covariance measurement, and belongs to the field of computer vision. The small sample learning algorithm based on the covariance measure comprises the following steps: (1) introducing an interpolation training mechanism to learn migratable knowledge, (2) designing a local covariance representation, then embedding the local covariance representation into a deep network to learn and express each concept, and (3) constructing a covariance measurement layer based on the local covariance representation to measure distribution consistency between a query sample and the concept. The invention provides a novel and simple end-to-end covariance measurement network CovaMNet, a second-order local covariance representation is designed to replace the traditional first-order conceptual representation, and a new covariance measurement function is provided. The results of comparison experiments on a plurality of reference data sets are analyzed, so that the CovaMNet framework provided by the invention has competitive effects on a general small sample classification task and a fine-grained small sample classification task.
Description
Technical Field
The invention relates to a small sample learning algorithm based on covariance measurement, and belongs to the field of computer vision.
Background
Human beings can learn new concepts from few examples and have strong generalization ability which the machine learning algorithm does not have at present, namely, human beings can learn a new concept from one or a few examples, but the machine learning standard algorithm needs more examples to reach the same ability marginally. At present, machine learning algorithms depend too much on labeled data, while in practical application, the cost of data labeling is often high, and how to learn by using a limited small amount of labeled data and make a model have high generalization capability becomes an important subject.
In response to the above problems, it is necessary to utilize more a priori knowledge to assist in characterization and learning.
The currently available research schemes, as shown in fig. 1, can be roughly divided into the following three categories according to the departure point and motivation:
(1) small sample learning based on data expansion;
(2) small sample learning based on meta-learning;
(3) metric-based small sample learning.
The starting point of the small sample learning based on data expansion is as follows: new data is generated to augment the training data. When the data is enough, the traditional machine learning algorithm can be directly adopted to solve the recognition task, and the currently proposed work is mainly based on auto-encoders or generation countermeasure networks (GANs). For example, Antoniou et al propose a data augmentation countermeasure GAN (DAGAN) to generate new samples that are used to help improve the performance of small sample learning, especially single sample learning tasks. Bharath Hariharan et al propose a non-parametric method to learn the mapping analogy relationship between samples by constructing a quadruple in the training data to generate new samples. Elischwartz et al propose a delta-encoder model that learns the migratable deformation (transferable deformations) between pairs of samples by using an encoder, and applies this deformation to other samples to synthesize new samples, and finally train the final classifier using the synthesized samples. Then Yu-Xiong Wang et al propose a one-stage method, which integrates data expansion and final classification tasks, and combines two processes of generating new samples by a generator and expanding a sample learning classifier for learning. Yongqin Xian et al propose a unified feature generation framework, which designs a generation model by combining the respective advantages of a variational auto-encoder (VAE) and a generation countermeasure network, and performs data augmentation in a feature space. Zitian Chen et al also synthesize new samples in the feature space, using a novel auto-encoder to directly synthesize instance features. Bo Liu et al propose a feature migration network FATTEN to model the feature trajectory space caused by the change in the target pose, so that on one trajectory line in the manifold space, the input features and the migrated features can be mapped to the same point in the appearance space.
The small sample learning based on the meta learning is to use the thought of the meta learning and to help the base learner to learn from the task distribution angle. The difference between meta-learning and traditional base learning is that meta-learning mainly learns how to dynamically select the correct bias, whereas the bias in base learning is a priori fixed or user defined. As shown in fig. 2, base learning can only learn a single fixed hypothesis for a particular task data set, while meta learning can learn different hypotheses for different guided base learners of a task using a meta learner. Related work has been proposed in recent years. For example, a meta-learning neural network (MANN) with memory extension solves the problem of how to quickly encode important information of a new task by introducing an additional memory module. Meta-learning long-short term memory networks (Meta-LSTM) learns how to initialize a model and how to update a new task quickly. Model-independent meta-learning (MAML) attempts to learn a good initialization for model learning, so that a model can achieve good performance by only updating a few steps of gradient in the face of a new task. Qianru Sun et al propose a meta-learning migration learning Method (MTL) by pre-training a feature extractor on the auxiliary set, then fine-tuning to get the base learner by combining a small amount of training data of the new task, and then enabling the base learner to adapt to the new task by learning the scaling and offset parameters. Task independent learning (TAML) proposes an entropy-based approach to learn the initial model with the greatest uncertainty. Task-aware feature embedding networks (TAFE-Nets for short) learn how to adapt the picture representations to new tasks through a meta-learning paradigm.
Metric-based small sample learning leverages the idea of metric learning to learn a feature-embedded network with migration capability by exploiting metric learning penalties, as shown in fig. 3. Such methods are simple and effective, and in recent years, a number of related works and methods have been proposed. In 2015, Gregory Koch et al learned a twin network (Simense network) from source data (auxiliary set) for the first time using metric loss, and then multiplexed the feature representation learned by the twin network to solve the target small sample classification task. Oriol Vinyals et al propose matching networks (matching networks) to directly compare the similarity between the query pictures and the support set pictures for classification, and propose an episode training mechanism. Jake Snell et al propose a prototype network (prototypical network), which takes the mean vector as a prototype to characterize each category in the support set, and then finds the nearest prototype (category) corresponding to each query picture for classification. Flood Sung et al propose a relationship network to learn a depth metric function instead of the fixed metric function (e.g., euclidean distance or cosine similarity) used in the conventional classification of small samples. Kelsey r.allen et al propose to use infinite mixture prototypes (infinite mixture prototypes) to characterize each class on the basis of a prototype network in which each class is characterized by only a single prototype, and it is difficult to sufficiently express a complex distribution of classes. Other work includes using methods of target detection to help solve small sample problems, using graph models to solve small sample problems, using dense classification networks to solve small sample problems, etc.
Although the starting point of the above methods is different, the three methods are all from the perspective of transfer learning, and an additional auxiliary data set is utilized to help solve the problem of small sample learning. The present invention focuses mainly on the third category of methods, namely metric-based small sample learning methods.
Disclosure of Invention
The purpose of the invention is as follows: a small sample learning algorithm based on covariance measurement is provided to solve the above problems in the prior art.
The technical scheme is as follows: a small sample learning algorithm based on covariance measurement specifically comprises the following steps:
step (1): dividing a data set adopted by an experiment to obtain a training set, a verification set and a test set;
step (2): scaling the pictures in the data set to ensure that the resolution is 84 x 84;
and (3): constructing a classification task on a data set, sampling an episode training mechanism, and respectively and randomly constructing episodes in a training stage and a testing stage;
and (4): embedding the local covariance representation into a deep network to learn the feature representation of each concept, and extracting a deep local descriptor feature for the query image;
and (5): on the basis of the local covariance representation, calculating distribution consistency between the query image and each category by using a covariance measurement layer;
and (6): and fusing the two modules into a framework based on the local covariance representation and the covariance measurement layer, and performing end-to-end training.
In a further embodiment, in the data set partitioning process of step (1), there are 5 data sets involved, namely 2 small sample image classification data sets miniImageNet and tiededImageNet, and 3 fine-grained benchmark data sets Stanford logs, Stanford Cars and CUB circles; dividing each data set, taking miniImageNet and Standard logs as examples, using 64 classes of 100 classes of the miniImageNet data set as training sets, 16 classes as verification sets and 20 classes as test sets; 120 classes in the Standford logs dataset are used as a training set (auxiliary set), 20 classes are used as a verification set, and 30 classes are used as a test set.
In a further embodiment, in the episode construction process in the step (3), in the training phase, each constructed episode includes a support set and a query set; a 5-way 1-shot classification task, wherein 5 categories each contain 1 support picture and 15 query pictures; the 5-way5-shot classification task is characterized in that each of 5 categories comprises 5 support pictures and 15 query pictures, and the number of the built episodes in the training stage is different from that in the testing stage.
In a further embodiment, the local covariance representation of step (4) differs from the covariance matrices widely used in other tasks in that a sample-based covariance matrix is first defined
The calculation process is shown as formula (1), and richer deep local descriptor feature generations are adoptedRepresenting each picture by replacing global features, and calculating local descriptors of all pictures in a certain category together to obtain local covariance representation of the category
The calculation process is shown as formula (2):
wherein the content of the first and second substances,
represents the mean of the K samples and represents the average of the K samples,
representing a matrix of mean vectors.
In a further embodiment, the covariance measurement layer of step (5) is used to measure the consistency relationship between the samples and the classes; representing pictures in a query set using depth locality descriptors (M d-dimensional depth locality descriptors)
By the covariance measurement formula f (x, sigma) xTSigma x to calculate the relation between the sample and the class, wherein
On behalf of the sample of the query,
representing a covariance matrix representation of a particular class, computing X and
the local covariance measure between them, the calculation process is shown in equation (3):
wherein the content of the first and second substances,
contains M local similarity values between the query picture and a specific category, and diag (circle) represents a column vector formed by elements on the main diagonal of the matrix.
In a further embodiment, the framework of step (6) comprises two key modules: a convolution embedding module and a covariance measurement module, wherein the convolution embedding module comprises four volume blocks (Conv64F for short), wherein each volume block is composed of a convolution layer, a Batch normalization layer (Batch normalization layer) and a leakage linear rectification layer (leakage ReLU layer); adding a 2 x 2 maximum pooling layer (max-Poolingclayer) after the first two volume blocks respectively, calculating the query picture and C categories to obtain all local similarity values, connecting the local similarity values in series, then realizing the mapping operation by adopting one-dimensional convolution with the step length of M, and finally calculating the final classification result by using softmax and a cross entropy loss function.
Has the advantages that:
(1) the invention provides a novel and simple end-to-end covariance measurement network CovaMNet;
(2) the invention designs a second-order local covariance representation to replace the traditional first-order conceptual representation;
(3) the invention proposes a new covariance measurement function.
The results of comparison experiments on a plurality of reference data sets are analyzed, so that the CovaMNet framework provided by the invention has competitive effects on a general small sample classification task and a fine-grained small sample classification task.
Drawings
FIG. 1 is a diagram of a small sample learning method partition and corresponding representative method of the present invention.
FIG. 2 is a schematic diagram of meta learning and base learning of the present invention.
FIG. 3 is a schematic diagram of the relationship between the query picture and the category picture in the learning comparison of the present invention.
Fig. 4 is a block diagram of the covannet of the present invention for the 5-way 1-shot classification task.
Fig. 5 is a diagram of a four-layer embedded network Conv64F according to the present invention.
Detailed Description
To more fully illustrate the objects, features and advantages of the present invention, the following detailed description of the invention is given in conjunction with the accompanying drawings and specific examples.
The present invention is directed to learning and understanding new concepts (categories) from one instance or a few instances. Since there is only one or a few samples labeled in each concept, directly using machine learning models (e.g., support vector machine SVM, convolutional neural networks CNNs) easily results in model overfitting. At this point, the data per category is too little to adequately express a category through these fewer samples, and it is critical how to efficiently use the auxiliary data set to aid learning, and how to learn migratable knowledge from the auxiliary data set. Three problems to be solved are mainly faced in the present invention:
(1) how to leverage the auxiliary data set to learn and store migratable knowledge;
(2) how to express each concept (category) robustly;
(3) how to reasonably compute the relationship between the test query sample and the concept.
For the first problem, the meta-learning based thumbnail method (meta-learning based methods) is mainly implemented by using meta-learners, and the metric based thumbnail method mainly uses an episode training mechanism (episodic training mechanism) to capture migratable knowledge. The present invention employs an episode training mechanism to solve this problem.
For the second problem, existing work such as Prototypical Nets directly regard the mean value of each class as a prototype (prototype), and do the representation of the class by the prototype; and GNNs use a graph (graph) to represent categories. The present invention is distinguished from these methods that use first order statistics, which are used as a representation of the class.
For the third problem, most of the current work measures the similarity between query samples and categories using simple euclidean distance or cosine similarity. In the working relationship Net, a deep nonlinear relationship network is adopted for measurement. In the invention, a depth covariance measurement function is newly defined to measure the consistency between the samples and the classes.
The invention provides a small sample learning algorithm based on covariance measurement, and a novel and effective covariance measurement-based framework is designed for the small sample learning, so that three important aspects of problems in small sample learning, namely knowledge migration, concept characterization and relation measurement, are considered. Covannet is a proposed end-to-end method, mainly comprising two key modules: local covariance representation and covariance metrics. The first module is used for learning feature representation of the picture, extracting depth local description sub-features of the query picture and calculating local covariance representation of concept categories; the second module mainly comprises a covariance measurement layer, and on the basis of the first module, the distribution consistency between the query picture and each category is calculated. Further, the two modules may be integrated into a unified framework for end-to-end training. In this way, the feature representation and the similarity measure can complement and contribute to each other.
The invention provides a small sample learning algorithm based on covariance measurement, which specifically comprises the following steps:
step (1): dividing a data set adopted by an experiment to obtain a training set, a verification set and a test set;
step (2): scaling the pictures in the data set to ensure that the resolution is 84 x 84;
and (3): constructing a classification task on a data set, sampling an episode training mechanism, and respectively and randomly constructing an episode (episode) in a training stage and a testing stage;
and (4): embedding the local covariance representation into a deep network to learn the feature representation of each concept, and extracting a deep local descriptor feature for the query image;
and (5): on the basis of the local covariance representation, calculating distribution consistency between the query image and each category by using a covariance measurement layer;
and (6): and fusing the two modules into a framework based on the local covariance representation and the covariance measurement layer, and performing end-to-end training.
1. Local covariance representation
Covariance matrices have been widely used in many tasks as region descriptors or general characterizations. Covariance matrices also have many good properties, such as second order statistics and symmetric semi-positivity.
Order to
Represents a data matrix, an
The sample-based covariance matrix
Can be defined as shown in formula (1):
wherein
Is the mean of K samples.
Unlike the covariance representation of a general picture set, in small sample learning, there are only a few pictures (for example, K ═ 1 to 5) per class, and it is difficult to calculate a covariance matrix that is effective and guarantees non-singularity. Furthermore, when there are too few samples in a class, the resulting covariance matrix also has difficulty accurately describing the data distribution for that class.
In order to solve the problems, the invention adopts richer deep local descriptor characteristics to replace global characteristics to represent the pictures, and then the local characteristics of all the pictures in the category are usedThe partial descriptor computes a local covariance representation of the analogy. Given a set of pictures of the c-th category
Wherein DcContaining K pictures and XiIt can be expressed as M depth local descriptors with d dimension (M441, d 64 in the experiment). Thus the local covariance representation of the c-th class
Can be defined as shown in formula (2):
wherein
A matrix is represented that consists of mean vectors, all columns of which are the mean vectors of the MK depth local descriptors.
In general, the number of all depth local descriptors in a class is far larger than the feature dimension, and the non-singularity of the class can be ensured when the covariance matrix of the class is calculated. Computing the covariance matrix using the depth local descriptors can also capture important local detail information for each class. Meanwhile, the local covariance matrix is embedded in the deep neural network, and can be continuously iteratively learned along with network updating. Furthermore, the method does not need to define the number of samples (shot) of each class, which means that in the training and testing phase, the classes with different numbers of samples can be represented by covariance.
2. Covariance measure
To measure the relationship between the samples and the classes, a covariance metric function is defined as shown in equation (3):
f(x，∑)＝xT∑x#(3)
wherein the content of the first and second substances,
on behalf of the sample of the query,
representing a covariance matrix representation of a particular class. According to theory 1, when the direction of the sample x is consistent with the direction of the eigenvectors corresponding to the k eigenvalues before Σ, the value of f (x, Σ) is maximized.
Hypothesis of theory 1
Is represented by a covariance matrix corresponding to a specific class in the support set S and satisfies the condition of sigma-V lambda VTWherein a diagonal matrix
Contains d eigenvalues arranged in descending order, and the corresponding eigenvectors can represent orthogonal matrix
For any non-zero sample
When the direction of x is consistent with the direction of the eigenvector corresponding to k eigenvalues before the sigma, xTThe value of Σ x is largest.
Likewise, each query picture in the query set Q is represented by a depth local descriptor (M d-dimensional depth local descriptors)
Further X and
the local covariance measure between can be formalized as shown in equation (4):
wherein
Representing M local similarity values between the query picture and the category,
the local covariance representation of the representative class, diag (·) represents the column vector of the elements on the main diagonal of the matrix. Finally, the global similarity value Z may be calculated by linear weighting of the M local similarity values, i.e., Z ═ ωTz, where ω is a weight vector.
The covannet structure is shown in fig. 4, and mainly consists of two modules: a convolution embedding module and a covariance measurement module. Referring to the previous work, the convolution-embedded module is shown in fig. 5 and is mainly composed of four convolution blocks (Conv64F for short), each of which is composed of a convolution layer (containing 64 filters of size 3 × 3), a batch normalization layer and a linear rectifying layer with leakage. In addition, the invention also adds a 2X 2 maximum pooling layer after the first two volume blocks respectively.
As shown in fig. 4, the present invention provides a small sample learning algorithm based on covariance measure. The method comprises the following steps in a model training phase:
step (1) inputting a query picture to the embedded network to obtain a tensor of hxwxd. This tensor contains M units (M ═ hw), and each unit represents a d-dimensional depth local descriptor;
step (2) inputting support set to embedded network
Wherein DcCalculating K pictures containing the category c to obtain the c category DcLocal covariance representation of
Step (3) in the covariance measurement module, calculating the query picture X and each category
Local covariance similarity values z between;
mapping Z into a global similarity value Z through a full connected layer (full connected layer);
and (5) obtaining a final classification result through softmax and cross entropy loss.
To evaluate the performance of the covannet model proposed in the present invention, a comparison was made over five data sets with one reference model and seven leading edge small sample learning models. The data sets consist of two common small sample picture classification data sets miniImageNet and tiededImageNet, and three fine-grained benchmark data sets Standford Dogs, CUB Dogs, while the models consist of benchmark k neighbors (Baseline k-NN), Matching networks (Matching NetsFCE), Meta-Learner LSTM (Meta-Learner), model independent Meta-learning (MAML), prototype networks (ProtopicalNet), relational networks (relationship Net), Graph Neural Networks (GNN), and simple neural attention Learner (SNAIL). In a benchmark method Baseline k-NN, a classification network with 64 classes is trained by using the same embedded convolutional neural network (Conv64F) and three full-connection layers, after the training stage is finished, a test stage directly uses the trained network to extract features, and a k nearest neighbor classifier is used to obtain a final classification result. In other comparative models, experimental setup and results were consistent with the presenter. In some methods, different network architectures are used for the embedded modules. For example, metric-based methods (such as Prototypical Nets) often employ a four-layer convolutional neural network, each convolutional layer typically using Conv 64F; whereas methods based on meta-learning (such as MAML) use the same four-layer convolutional neural network, in order to mitigate the over-fitting phenomenon, Conv32F is used for each convolutional layer. Since the covannet proposed by the present invention belongs to the metric-based method category, for fairness the covannet uses Conv64F as an embedded module. In addition, the GNN results were retrained after replacing the embedded module with Conv64F, and the SNAIL results were trained using the shallow embedded module Conv 32F.
Table 1 shows the results of the comparative experiments on the data set miniImageNet. The second column indicates what kind of embedded module is used, the third column indicates the type of the method, the fourth column indicates whether the method is subjected to model fine tuning, and the last two columns are classification results set by classification tasks 5-way 1-shot and 5-way 5-shot. It can be seen from the table that the Baeline k-NN method performs much worse than the other comparative methods because the method does not employ an episode training mechanism. The remaining seven methods can be divided into two broad categories, meta-learning based methods and metric based methods. Compared with a small sample learning method based on meta-learning, the CovaMNet result is more competitive when a similar network structure is adopted. Compared with a small sample learning method based on measurement, the CovaMNet has better effect under the same experimental setting, the local covariance representation adopted in the CovaMNet belongs to second-order conceptual representation, and the information acquired by the method is more effective than the information acquired by using the first-order conceptual representation. Table 2 shows the results of the comparative experiments on the data set tiered imagenet.
TABLE 1 average accuracy of classification tasks on miniImageNet dataset
TABLE 2 mean accuracy of classification tasks on tiered ImageNet dataset
The experimental setup on the fine-grained dataset and the miniImageNet dataset remained the same. Unlike the general picture classification task, the fine-grained picture classification task is more challenging because the inter-class spacing is small but the intra-class spacing is large in the fine-grained dataset. This problem becomes particularly acute in small sample scenes. Existing work rarely tests on these three fine-grained datasets. The invention formally introduces a fine-grained data set into a small sample learning problem for the first time. Table 3 shows the results of comparative experiments on the dataset Standford logs, Table 4 shows the results of comparative experiments on the dataset Standford Cars, and Table 5 shows the results of comparative experiments on the dataset CUB logs.
TABLE 3 average accuracy of classification tasks on Standford logs dataset
TABLE 4 average accuracy of classification tasks on the Standford Cars dataset
TABLE 5 average accuracy of classification tasks on CUB Birds dataset
Claims (6)
1. A small sample learning algorithm based on covariance measurement specifically comprises the following steps:
step (1): dividing a data set adopted by an experiment to obtain a training set, a verification set and a test set;
step (2): zooming the pictures in the data set to ensure that the resolution reaches an expected value;
and (3): constructing a classification task on a data set, sampling an episode training mechanism, and respectively and randomly constructing episodes in a training stage and a testing stage;
and (4): embedding the local covariance representation into a deep network to learn the feature representation of each concept, and extracting a deep local descriptor feature for the query image;
and (5): on the basis of the local covariance representation, calculating distribution consistency between the query image and each category by using a covariance measurement layer;
and (6): and fusing the two modules into a framework based on the local covariance representation and the covariance measurement layer, and performing end-to-end training.
2. The small sample learning algorithm based on covariance measure as claimed in claim 1, wherein: in the data set dividing process of the step (1), the related data sets are 5, namely 2 small sample image classification data sets miniImageNet and tieredImageNet, 3 fine-grained benchmark data sets Stanford logs, Stanfordcars and CUB files;
dividing each data set, taking miniImageNet and Standard logs as examples, using 64 classes of 100 classes of the miniImageNet data set as training sets, 16 classes as verification sets and 20 classes as test sets;
120 classes in the Standford logs dataset are used as a training set, 70 classes are used as a verification set, and 30 classes are used as a test set.
3. The small sample learning algorithm based on covariance measure as claimed in claim 1, wherein: in the construction process of the episodes in the step (3), in a training stage, each constructed episode comprises a support set and a query set; a 5-way 1-shot classification task, wherein 5 categories each contain 1 support picture and 15 query pictures; the 5-way5-shot classification task is characterized in that each of 5 categories comprises 5 support pictures and 15 query pictures, and the number of built episodes in the training stage is different from that in the testing stage.
4. The small sample learning algorithm based on covariance measure as claimed in claim 1, wherein: the local covariance representation of step (4) differs from the covariance matrices widely used in other tasks in that a sample-based covariance matrix is first defined
The calculation process is shown as formula (1), each picture is represented by adopting richer deep local descriptor characteristics to replace global characteristics, and the local of all pictures in a certain category is representedThe partial descriptors are calculated together to obtain a local covariance representation of the class
The calculation process is shown as formula (2):
wherein the content of the first and second substances,
represents the mean of the K samples and represents the average of the K samples,
representing a matrix of mean vectors.
5. The small sample learning algorithm based on covariance measure as claimed in claim 1, wherein: the covariance measurement layer of the step (5) is used for measuring the consistency relation between the samples and the classes; representing pictures in a query set using depth-locality descriptors
By a covariance measure formula
To calculate the relationship between the sample and the category, wherein
On behalf of the sample of the query,
a covariance matrix representation representing a particular class,calculating the sum of X
The local covariance measure between them, the calculation process is shown in equation (3):
wherein the content of the first and second substances,
contains M local similarity values between the query picture and a specific category, and diag (circle) represents a column vector formed by elements on the main diagonal of the matrix.
6. The small sample learning algorithm based on covariance measure as claimed in claim 1, wherein: the framework of step (6) comprises two key modules: a convolution embedding module and a covariance measurement module,
the convolution embedded module comprises four convolution blocks, wherein each convolution block consists of a convolution layer, a batch normalization layer and a linear rectification layer with leakage; adding a 2 multiplied by 2 maximum pooling layer after the first two volume blocks respectively, firstly calculating the query picture and C categories to obtain all local similarity values in series, then adopting one-dimensional convolution with the step length of M to realize the mapping operation, and finally calculating the final classification result by using softmax and a cross entropy loss function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010783893.7A CN111858991A (en) | 2020-08-06 | 2020-08-06 | Small sample learning algorithm based on covariance measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010783893.7A CN111858991A (en) | 2020-08-06 | 2020-08-06 | Small sample learning algorithm based on covariance measurement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111858991A true CN111858991A (en) | 2020-10-30 |
Family
ID=72971492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010783893.7A Pending CN111858991A (en) | 2020-08-06 | 2020-08-06 | Small sample learning algorithm based on covariance measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111858991A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487805A (en) * | 2020-11-30 | 2021-03-12 | 武汉大学 | Small sample Web service classification method based on meta-learning framework |
| CN112487169A (en) * | 2020-12-11 | 2021-03-12 | 北京邮电大学 | Meta-learning-based personalized dialogue rewriting method |
| CN112786030A (en) * | 2020-12-30 | 2021-05-11 | 中山大学 | Countersampling training method and device based on meta-learning |
| CN112949740A (en) * | 2021-03-17 | 2021-06-11 | 重庆邮电大学 | Small sample image classification method based on multilevel measurement |
| CN113076976A (en) * | 2021-03-17 | 2021-07-06 | 中山大学 | Small sample image classification method based on local feature relation exploration |
| CN113177521A (en) * | 2021-05-26 | 2021-07-27 | 电子科技大学 | Intelligent radiation source identification method based on combined twin network |
| CN113221964A (en) * | 2021-04-22 | 2021-08-06 | 华南师范大学 | Single sample image classification method, system, computer device and storage medium |
| CN113255824A (en) * | 2021-06-15 | 2021-08-13 | 京东数科海益信息科技有限公司 | Method and device for training classification model and data classification |
| CN113344729A (en) * | 2021-06-04 | 2021-09-03 | 中国石油大学(华东) | Residual oil submergence digging method based on small sample learning |
| CN113435509A (en) * | 2021-06-28 | 2021-09-24 | 山东力聚机器人科技股份有限公司 | Small sample scene classification and identification method and system based on meta-learning |
| CN113688878A (en) * | 2021-07-30 | 2021-11-23 | 华东师范大学 | Small sample image classification method based on memory mechanism and graph neural network |
| CN114202028A (en) * | 2021-12-13 | 2022-03-18 | 四川大学 | Rolling bearing life stage identification method based on MAMTL |
| CN114387524A (en) * | 2022-03-24 | 2022-04-22 | 军事科学院系统工程研究院网络信息研究所 | Image identification method and system for small sample learning based on multilevel second-order representation |
| CN114782779A (en) * | 2022-05-06 | 2022-07-22 | 兰州理工大学 | Small sample image feature learning method and device based on feature distribution migration |
| CN114998956A (en) * | 2022-05-07 | 2022-09-02 | 北京科技大学 | Small sample image data expansion method and device based on intra-class difference |
| CN115527269A (en) * | 2022-10-10 | 2022-12-27 | 动自由(北京)科技有限公司 | Intelligent human body posture image identification method and system |
| CN116091867A (en) * | 2023-01-12 | 2023-05-09 | 北京邮电大学 | Model training and image recognition method, device, equipment and storage medium |
| CN117557840A (en) * | 2023-11-10 | 2024-02-13 | 中国矿业大学 | Fundus lesion grading method based on small sample learning |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070133878A1 (en) * | 2005-12-14 | 2007-06-14 | Porikli Fatih M | Method for constructing covariance matrices from data features |
| US20100111396A1 (en) * | 2008-11-06 | 2010-05-06 | Los Alamos National Security | Object and spatial level quantitative image analysis |
| CN107748871A (en) * | 2017-10-27 | 2018-03-02 | 东南大学 | A kind of three-dimensional face identification method based on multiple dimensioned covariance description with the sparse classification of local sensitivity Riemann's core |
| CN108804784A (en) * | 2018-05-25 | 2018-11-13 | 江南大学 | A kind of instant learning soft-measuring modeling method based on Bayes's gauss hybrid models |
| CN109376578A (en) * | 2018-08-27 | 2019-02-22 | 杭州电子科技大学 | A kind of small sample target identification method based on depth migration metric learning |
| CN110188864A (en) * | 2019-05-06 | 2019-08-30 | 南京大学 | The small-sample learning method of measurement is indicated and is distributed based on distribution |
| CN110532911A (en) * | 2019-08-19 | 2019-12-03 | 南京邮电大学 | Covariance measurement drives the short-sighted frequency emotion identification method of small sample GIF and system |
| CN111488951A (en) * | 2020-05-22 | 2020-08-04 | 南京大学 | Countermeasure metric learning algorithm based on RGB-D image classification problem |
-
2020
- 2020-08-06 CN CN202010783893.7A patent/CN111858991A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070133878A1 (en) * | 2005-12-14 | 2007-06-14 | Porikli Fatih M | Method for constructing covariance matrices from data features |
| US20100111396A1 (en) * | 2008-11-06 | 2010-05-06 | Los Alamos National Security | Object and spatial level quantitative image analysis |
| CN107748871A (en) * | 2017-10-27 | 2018-03-02 | 东南大学 | A kind of three-dimensional face identification method based on multiple dimensioned covariance description with the sparse classification of local sensitivity Riemann's core |
| CN108804784A (en) * | 2018-05-25 | 2018-11-13 | 江南大学 | A kind of instant learning soft-measuring modeling method based on Bayes's gauss hybrid models |
| CN109376578A (en) * | 2018-08-27 | 2019-02-22 | 杭州电子科技大学 | A kind of small sample target identification method based on depth migration metric learning |
| CN110188864A (en) * | 2019-05-06 | 2019-08-30 | 南京大学 | The small-sample learning method of measurement is indicated and is distributed based on distribution |
| CN110532911A (en) * | 2019-08-19 | 2019-12-03 | 南京邮电大学 | Covariance measurement drives the short-sighted frequency emotion identification method of small sample GIF and system |
| CN111488951A (en) * | 2020-05-22 | 2020-08-04 | 南京大学 | Countermeasure metric learning algorithm based on RGB-D image classification problem |
Non-Patent Citations (4)
| Title |
|---|
| SHINICHI SHIRAKAWA 等: "Sample Reuse in the Covariance Matrix Adaptation Evolution Strategy Based on Importance Sampling", 《GECCO \'15: PROCEEDINGS OF THE 2015 ANNUAL CONFERENCE ON GENETIC AND EVOLUTIONARY COMPUTATION》, pages 305 - 312 * |
| WENBIN LI 等: "Distribution Consistency Based Covariance Metric Networks for Few-Shot Learning", 《THE THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE (AAAI-19)》, vol. 33, no. 1, pages 8642 - 8649 * |
| 李文斌: "基于度量的小样本图像分类", 《中国博士学位论文全文数据库 信息科技辑》, no. 1, pages 138 - 157 * |
| 李涛涛: "基于局部描述子的小样本学习方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 7, pages 138 - 1121 * |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487805A (en) * | 2020-11-30 | 2021-03-12 | 武汉大学 | Small sample Web service classification method based on meta-learning framework |
| CN112487805B (en) * | 2020-11-30 | 2024-02-02 | 武汉大学 | Small sample Web service classification method based on meta-learning framework |
| CN112487169A (en) * | 2020-12-11 | 2021-03-12 | 北京邮电大学 | Meta-learning-based personalized dialogue rewriting method |
| CN112487169B (en) * | 2020-12-11 | 2023-02-07 | 北京邮电大学 | Meta-learning-based personalized dialogue rewriting method |
| CN112786030A (en) * | 2020-12-30 | 2021-05-11 | 中山大学 | Countersampling training method and device based on meta-learning |
| CN112949740A (en) * | 2021-03-17 | 2021-06-11 | 重庆邮电大学 | Small sample image classification method based on multilevel measurement |
| CN113076976A (en) * | 2021-03-17 | 2021-07-06 | 中山大学 | Small sample image classification method based on local feature relation exploration |
| CN113076976B (en) * | 2021-03-17 | 2023-08-18 | 中山大学 | Small sample image classification method based on local feature relation exploration |
| CN112949740B (en) * | 2021-03-17 | 2022-11-25 | 重庆邮电大学 | Small sample image classification method based on multilevel measurement |
| CN113221964A (en) * | 2021-04-22 | 2021-08-06 | 华南师范大学 | Single sample image classification method, system, computer device and storage medium |
| CN113221964B (en) * | 2021-04-22 | 2022-06-24 | 华南师范大学 | Single sample image classification method, system, computer device and storage medium |
| CN113177521A (en) * | 2021-05-26 | 2021-07-27 | 电子科技大学 | Intelligent radiation source identification method based on combined twin network |
| CN113177521B (en) * | 2021-05-26 | 2022-07-01 | 电子科技大学 | Intelligent radiation source identification method based on combined twin network |
| CN113344729A (en) * | 2021-06-04 | 2021-09-03 | 中国石油大学(华东) | Residual oil submergence digging method based on small sample learning |
| CN113344729B (en) * | 2021-06-04 | 2022-09-09 | 中国石油大学(华东) | Residual oil submergence digging method based on small sample learning |
| CN113255824A (en) * | 2021-06-15 | 2021-08-13 | 京东数科海益信息科技有限公司 | Method and device for training classification model and data classification |
| CN113255824B (en) * | 2021-06-15 | 2023-12-08 | 京东科技信息技术有限公司 | Method and apparatus for training classification model and data classification |
| CN113435509A (en) * | 2021-06-28 | 2021-09-24 | 山东力聚机器人科技股份有限公司 | Small sample scene classification and identification method and system based on meta-learning |
| CN113435509B (en) * | 2021-06-28 | 2022-03-25 | 山东力聚机器人科技股份有限公司 | Small sample scene classification and identification method and system based on meta-learning |
| CN113688878A (en) * | 2021-07-30 | 2021-11-23 | 华东师范大学 | Small sample image classification method based on memory mechanism and graph neural network |
| CN113688878B (en) * | 2021-07-30 | 2022-08-19 | 华东师范大学 | Small sample image classification method based on memory mechanism and graph neural network |
| CN114202028A (en) * | 2021-12-13 | 2022-03-18 | 四川大学 | Rolling bearing life stage identification method based on MAMTL |
| CN114202028B (en) * | 2021-12-13 | 2023-04-28 | 四川大学 | MAMTL-based rolling bearing life stage identification method |
| CN114387524A (en) * | 2022-03-24 | 2022-04-22 | 军事科学院系统工程研究院网络信息研究所 | Image identification method and system for small sample learning based on multilevel second-order representation |
| CN114782779A (en) * | 2022-05-06 | 2022-07-22 | 兰州理工大学 | Small sample image feature learning method and device based on feature distribution migration |
| CN114998956A (en) * | 2022-05-07 | 2022-09-02 | 北京科技大学 | Small sample image data expansion method and device based on intra-class difference |
| CN115527269A (en) * | 2022-10-10 | 2022-12-27 | 动自由(北京)科技有限公司 | Intelligent human body posture image identification method and system |
| CN116091867A (en) * | 2023-01-12 | 2023-05-09 | 北京邮电大学 | Model training and image recognition method, device, equipment and storage medium |
| CN116091867B (en) * | 2023-01-12 | 2023-09-29 | 北京邮电大学 | Model training and image recognition method, device, equipment and storage medium |
| CN117557840A (en) * | 2023-11-10 | 2024-02-13 | 中国矿业大学 | Fundus lesion grading method based on small sample learning |
| CN117557840B (en) * | 2023-11-10 | 2024-05-24 | 中国矿业大学 | Fundus lesion grading method based on small sample learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111858991A (en) | Small sample learning algorithm based on covariance measurement | |
| Chrysos et al. | Deep polynomial neural networks | |
| Shang et al. | SAR targets classification based on deep memory convolution neural networks and transfer parameters | |
| Jia et al. | Factorized latent spaces with structured sparsity | |
| Yu et al. | On combining multiple features for cartoon character retrieval and clip synthesis | |
| Bo et al. | Fast algorithms for large scale conditional 3D prediction | |
| Chen et al. | Model Metric Co-Learning for Time Series Classification. | |
| Zhang et al. | Unsupervised nonnegative adaptive feature extraction for data representation | |
| Wei et al. | Robotic grasping recognition using multi-modal deep extreme learning machine | |
| Zheng et al. | Principal characteristic networks for few-shot learning | |
| CN104392250A (en) | Image classification method based on MapReduce | |
| Zhou et al. | Exploiting operation importance for differentiable neural architecture search | |
| Park et al. | Fast and scalable approximate spectral matching for higher order graph matching | |
| Zeng et al. | Accelerating convolutional neural networks by removing interspatial and interkernel redundancies | |
| Bai et al. | Correlative channel-aware fusion for multi-view time series classification | |
| Lin et al. | Visual feature coding based on heterogeneous structure fusion for image classification | |
| Ahuja et al. | Deterministic Multi-kernel based extreme learning machine for pattern classification | |
| Shen et al. | Person re-identification with deep kronecker-product matching and group-shuffling random walk | |
| Li et al. | Unsupervised transfer learning via low-rank coding for image clustering | |
| Ge et al. | Stacked denoising extreme learning machine autoencoder based on graph embedding for feature representation | |
| Yu et al. | 3D object representation learning: A set-to-set matching perspective | |
| Terefe et al. | Time series averaging using multi-tasking autoencoder | |
| Mishne et al. | Co-manifold learning with missing data | |
| Wu et al. | Semisupervised feature learning by deep entropy-sparsity subspace clustering | |
| CN115017366B (en) | Unsupervised video hash retrieval method based on multi-granularity contextualization and multi-structure preservation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |