CN113033698A - Method for improving classification accuracy of few samples by using distribution strategy - Google Patents
Method for improving classification accuracy of few samples by using distribution strategy Download PDFInfo
- Publication number
- CN113033698A CN113033698A CN202110412397.5A CN202110412397A CN113033698A CN 113033698 A CN113033698 A CN 113033698A CN 202110412397 A CN202110412397 A CN 202110412397A CN 113033698 A CN113033698 A CN 113033698A
- Authority
- CN
- China
- Prior art keywords
- sample data
- variance
- feature
- data
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009826 distribution Methods 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 27
- 239000013598 vector Substances 0.000 claims abstract description 53
- 238000005070 sampling Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000006185 dispersion Substances 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
一种利用分布策略提升少样本分类准确性的方法，包括以下步骤：获取与少样本数据外形特征向量相似度最高的多组不同类别的样本数据集训练特征提取器，利用特征提取器获取每张照片的特征向量xj；利用特征向量xj对每组不同类别的样本数据集全部进行均值和方差的求取；再次利用特征向量xj计算少样本数据的均值和方差；根据每组不同类别的样本数据集的均值和方差校准少样本数据的均值和方差；利用校准后的均值和方差随机抽样生成新特征数据去训练线性分类器，通过训练得到分类器模型；本发明通过对数据集中少样本的分布进行校准，同时无需添加其他参数和模块，便能够提高分类器对少样本类别识别的准确性。
A method for improving the accuracy of few-sample classification by using a distribution strategy, comprising the following steps: obtaining multiple sets of sample data sets with the highest similarity to the shape feature vector of the few-sample data of different categories to train a feature extractor, and using the feature extractor to obtain each The feature vector x j of the photo; use the feature vector x j to obtain the mean and variance of the sample data sets of each group of different categories; use the feature vector x j again to calculate the mean and variance of the few-sample data; according to each group of different categories The mean and variance of the sample data set are used to calibrate the mean and variance of the few-sample data; the calibrated mean and variance are randomly sampled to generate new feature data to train the linear classifier, and the classifier model is obtained through training; The distribution of the samples is calibrated, and the accuracy of the classification of the few-shot categories can be improved without adding other parameters and modules.
Description
Technical Field
The invention relates to the technical field of image classification, in particular to a method for improving the classification accuracy of few samples by using a distribution strategy.
Background
Deep learning has been widely used in various fields, mainly for solving the problem of image classification, however, deep learning is a technique that relies heavily on data, requires a large number of labeled samples to play a core role, in real life, many scene applications may not have too many labeled sample data, such as in the medical field, the security field, etc., and the cost of retrieving labeled data is very large, therefore, major problems faced by deep learning techniques when faced with the problem of small samples include the fact that, data distributions formed by very few samples tend to differ significantly from the true data distributions, training a model on a skewed data distribution can lead to severe overfitting phenomena and severely destroy the generalization ability of the model, so learning a model with good generalization performance from a very small number of samples is extremely difficult, and learning from a limited number of samples is a very challenging problem.
Disclosure of Invention
The invention aims to provide a method for improving the classification accuracy of few samples by using a distribution strategy aiming at the defects in the prior art, solves the problem that a model with good generalization performance cannot be obtained due to less sample data in the prior art aiming at the classification problem of few sample data images, and is realized by the following technical scheme for realizing the purposes:
a method for improving the classification accuracy of few samples by using a distribution strategy is characterized by comprising the following steps:
s1, obtaining a plurality of groups of sample data sets of different categories with the highest similarity with the appearance feature vectors of less sample data, training a feature extractor, and obtaining the feature vector x of each photo by using the feature extractorj；
S2, utilizing the feature vector x obtained in the step S1jCalculating the mean value and the variance of all the sample data sets of different types in each group;
s3, reusing the feature vector xjCalculating the mean value and the variance of less sample data;
s4, calibrating the mean value and the variance of the few sample data in the step S3 according to the mean value and the variance of each group of different types of sample data sets in the step S2;
and S5, randomly sampling the mean value and the variance after calibration in the step S4 to generate new feature data to train a linear classifier, and training the classifier through cross entropy loss to obtain a classifier model.
Preferably, the specific implementation manner of step S1 is: obtaining a plurality of groups of sample data sets of different types with the highest similarity to the appearance feature vector of the few sample data, collectively referring the plurality of groups of sample data sets of different types as base class data, arbitrarily training a feature extractor with the same high performance as Resnet on the base class data, and counting the feature vector x of each photo in each group of sample data sets of different types in the base class data by using the feature extractorj。
Preferably, the specific implementation manner of the calculation of the mean and the variance of each group of different types of sample data sets in step S2 is as follows: using the feature vector x acquired in step S1jSubstituting the data into a formula 1 and a formula 2 to respectively calculate the mean value and the variance of each group of different types of sample data sets;
wherein i represents the number of groups, niNumber of samples, μ, representing the ith groupi1Represents the mean, sigma, of each set of different classes of sample data setsi1Representing the variance, x, of each set of different classes of sample data setsjFeature vectors for each photo within the sets of different categories of sample data sets.
Preferably, the specific implementation manner of the calculation of the mean and the variance of the few sample data in step S3 is as follows: using the feature vector x acquired in step S1jSubstituting into formula 3 and formula 4 to respectively calculate the mean value and variance of the less sample data;
wherein i represents the number of groups, niNumber of samples, μ, representing the ith groupj2Represents the mean, Σ, of a small sample of datai2Variance, x, representing few sample datajFeature vectors for each photo within the sets of different categories of sample data sets.
Preferably, the implementation process of the calibration in step S4 includes the following steps:
s41, first, the feature vector x is reduced by equation 5jThe deviation degree of dispersion, wherein a TLPT method formula is adopted in formula 5;
wherein, λ is the set hyper-parameter, and the original characteristic distribution of less sample data can be restored when λ is set to 1; when the lambda is set to be less than 1, the positive skewness of the characteristic distribution of less sample data can be reduced; when lambda is set to be larger than 1, the positive skewness of the feature distribution of less sample data can be increased;
s42, next, using the feature vector x of the base class datajAnd transferring at least sample data, and acquiring the mean value and the variance of the small amount of sample data after calibration.
Preferably, the specific implementation steps of step S42 are as follows:
s421, first select and sample from the base class data
K base class data samples with the most similar appearance characteristic vectors;
s422, the distance to the sample is obtained by using the formula 6
Set of characteristic distances SdUsing equation 7 to obtain a sample
Nearest k base class data sets SN；
S423, passing SNAnd comparing and calibrating to obtain the mean value and the variance of the calibrated few-sample data.
Preferably, the specific implementation steps of step S5 are as follows:
s51, randomly sampling the mean and variance calibrated in the step S4 to generate new feature data to train a linear classifier, and using a set of standard data SySampling is carried out from the mean value and the variance after the calibration of less sample data, and a group of characteristic vectors marked as y types are generated:
s52, setting the total number of features generated by each group of different types of sample data sets as hyper-parameters, then using the y types of original data set feature vectors in less sample data and the y types of feature vectors generated in the step S51 as training data of a linear classifier, training the linear classifier by using cross entropy loss, and obtaining a classifier model:
and ∑ -logPr (y | x; θ) formula 11.
The invention has the beneficial effects that:
1. because similar characteristic representations generally have similar mean values and variances, the method can correct data with less sample distribution according to the mean values and the variances, and the input of the classifier is expanded by transferring data with enough samples similar to statistical information and then extracting enough samples from the corrected sample data distribution, so that the accuracy of classifying the few samples is greatly improved;
2. compared with the traditional method, the method disclosed by the invention has the advantages that for few-sample classification, the traditional method usually focuses on developing a stronger model to adapt to the distribution of few samples, the basic strategy is to train a model with only a few training samples, the model tends to be over-fitted by reducing the training loss of the samples as much as possible, however, the inclination of the few samples can reduce the generalization capability of the model, so that the real distribution of the samples can not be reflected far; the method of the invention calibrates the distribution of few samples in the data set, uses the characteristic data sampled from the calibration distribution to input the trained model, not only fits a few samples, but also does not need to add other parameters and modules, and can improve the accuracy of the classifier in identifying the classes of the few samples.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings and the detailed description below:
example 1:
as shown in fig. 1, a method for improving the classification accuracy of few samples by using a distribution strategy includes the following steps:
s1, obtaining a plurality of groups of sample data sets of different categories with the highest similarity with the appearance feature vectors of less sample data, training a feature extractor, and obtaining the feature vector x of each photo by using the feature extractorj；
The specific implementation manner of the step S1 is as follows: acquiring a plurality of groups of sample data sets of different classes with the highest similarity with the appearance feature vectors of less sample data,collectively referring a plurality of groups of sample data sets of different types as base class data, training a feature extractor with the same high performance as Resnet on the base class data at will, and counting the feature vector x of each photo in the sample data sets of different types of each group in the base class data by using the feature extractorj。
S2, utilizing the feature vector x obtained in the step S1jCalculating the mean value and the variance of all the sample data sets of different types in each group;
the specific implementation manner of solving the mean and the variance of each group of different types of sample data sets in step S2 is as follows: using the feature vector x acquired in step S1jSubstituting the data into a formula 1 and a formula 2 to respectively calculate the mean value and the variance of each group of different types of sample data sets;
wherein i represents the number of groups, niNumber of samples, μ, representing the ith groupi1Represents the mean, sigma, of each set of different classes of sample data setsi1Representing the variance, x, of each set of different classes of sample data setsjFeature vectors for each photo within the sets of different categories of sample data sets.
S3, reusing the feature vector xjCalculating the mean value and the variance of less sample data;
the specific implementation of the calculation of the mean and the variance of the few sample data in step S3 is as follows: using the feature vector x acquired in step S1jSubstituting into formula 3 and formula 4 to respectively calculate the mean value and variance of the less sample data;
wherein i represents the number of groups, niNumber of samples, μ, representing the ith groupi2Represents the mean, Σ, of a small sample of datai2Variance, x, representing few sample datajFeature vectors for each photo within the sets of different categories of sample data sets.
S4, calibrating the mean value and the variance of the few sample data in the step S3 according to the mean value and the variance of each group of different types of sample data sets in the step S2;
the implementation process of the calibration in step S4 includes the following steps:
s41, in order to make the feature vector xjThe feature distribution of (2) is closer to the gaussian distribution by first reducing the feature vector x using equation 5jThe deviation degree of dispersion, wherein a TLPT method formula is adopted in formula 5;
wherein, λ is the set hyper-parameter, and the original characteristic distribution of less sample data can be restored when λ is set to 1; when the lambda is set to be less than 1, the positive skewness of the characteristic distribution of less sample data can be reduced; when lambda is set to be larger than 1, the positive skewness of the feature distribution of less sample data can be increased;
s42, next, using the feature vector x of the base class datajTransferring at least sample data, and acquiring a mean value and a variance of the sample data after calibration; migration is based on feature vectors x of base class datajEuclidean distance from the characteristic mean value of the base class data;
the specific implementation steps of step S42 are as follows:
s421, first select and sample from the base class data
K of the most similar outline feature vectorsBase class data samples; k is a modifiable hyper-parameter which can be freely set according to the state;
s422, the distance to the sample is obtained by using the formula 6
Set of characteristic distances SdUsing equation 7 to obtain a sample
Nearest k base class data sets SN；
S423, passing SNAnd comparing and calibrating to obtain the mean value and the variance of the calibrated few-sample data.
Wherein alpha is also a modifiable hyper-parameter, set in situ according to the calibration state.
S5, randomly sampling and generating new feature data by using the mean value and the variance calibrated in the step S4 to train a linear classifier, and training the classifier through cross entropy loss to obtain a classifier model;
the specific implementation steps of step S5 are as follows:
s51, randomly sampling the mean and variance calibrated in the step S4 to generate new feature data to train a linear classifier, and using a set of standard data SyMean and variance after calibration from few sample dataSampling to generate a group of characteristic vectors marked as y types:
s52, setting the total number of features generated by each group of different types of sample data sets as hyper-parameters, then using the y types of original data set feature vectors in less sample data and the y types of feature vectors generated in the step S51 as training data of a linear classifier, training the linear classifier by using cross entropy loss, and obtaining a classifier model:
and ∑ -logPr (y | x; θ) formula 11.
Various other modifications and changes may be made by those skilled in the art based on the above-described technical solutions and concepts, and all such modifications and changes should fall within the scope of the claims of the present invention.
Claims (7)
1. A method for improving the classification accuracy of few samples by using a distribution strategy is characterized by comprising the following steps:
s1, obtaining a plurality of groups of sample data sets of different categories with the highest similarity with the appearance feature vectors of less sample data, training a feature extractor, and obtaining the feature vector x of each photo by using the feature extractorj；
S2, utilizing the feature vector x obtained in the step S1jCalculating the mean value and the variance of all the sample data sets of different types in each group;
s3, reusing the feature vector xjCalculating the mean value and the variance of less sample data;
s4, calibrating the mean value and the variance of the few sample data in the step S3 according to the mean value and the variance of each group of different types of sample data sets in the step S2;
and S5, randomly sampling the mean value and the variance after calibration in the step S4 to generate new feature data to train a linear classifier, and training the classifier through cross entropy loss to obtain a classifier model.
2. The method of claim 1, wherein the step S1 is implemented by the following steps: obtaining a plurality of groups of sample data sets of different types with the highest similarity to the appearance feature vector of the few sample data, collectively referring the plurality of groups of sample data sets of different types as base class data, arbitrarily training a feature extractor with the same high performance as Resnet on the base class data, and counting the feature vector x of each photo in each group of sample data sets of different types in the base class data by using the feature extractorj。
3. The method of claim 1, wherein the specific implementation of the mean and variance calculation of each set of different sample data sets in step S2 is as follows: using the feature vector x acquired in step S1jSubstituting the data into a formula 1 and a formula 2 to respectively calculate the mean value and the variance of each group of different types of sample data sets;
wherein i represents the number of groups, niNumber of samples, μ, representing the ith groupi1Represents the mean, sigma, of each set of different classes of sample data setsi1Representing the variance, x, of each set of different classes of sample data setsjFeature vectors for each photo within the sets of different categories of sample data sets.
4. The method of claim 1, wherein the sample-less classification accuracy of step S3 is improved by using a distribution strategyThe specific implementation of the mean and variance calculation is as follows: using the feature vector x acquired in step S1jSubstituting into formula 3 and formula 4 to respectively calculate the mean value and variance of the less sample data;
wherein i represents the number of groups, niNumber of samples, μ, representing the ith groupi2Represents the mean, Σ, of a small sample of datai2Variance, x, representing few sample datajFeature vectors for each photo within the sets of different categories of sample data sets.
5. The method of claim 1, wherein the step S4 of calibrating comprises the following steps:
s41, first, the feature vector x is reduced by equation 5jThe deviation degree of dispersion, wherein a TLPT method formula is adopted in formula 5;
wherein, λ is the set hyper-parameter, and the original characteristic distribution of less sample data can be restored when λ is set to 1; when the lambda is set to be less than 1, the positive skewness of the characteristic distribution of less sample data can be reduced; when lambda is set to be larger than 1, the positive skewness of the feature distribution of less sample data can be increased;
s42, next, using the feature vector x of the base class datajAnd transferring at least sample data, and acquiring the mean value and the variance of the small amount of sample data after calibration.
6. The method of claim 5, wherein the step S42 is implemented by the following steps:
s421, first select and sample from the base class data
K base class data samples with the most similar appearance characteristic vectors;
s422, the distance to the sample is obtained by using the formula 6
Set of characteristic distances SdUsing equation 7 to obtain a sample
Nearest k base class data sets SN；
S423, passing SNAnd comparing and calibrating to obtain the mean value and the variance of the calibrated few-sample data.
7. The method of claim 1, wherein the step S5 is implemented by the following steps:
s51, randomly sampling the mean and variance calibrated in the step S4 to generate new feature data to train a linear classifier, and using a set of standard data SySampling is carried out from the mean value and the variance after the calibration of less sample data, and a group of characteristic vectors marked as y types are generated:
s52, setting the total number of features generated by each group of different types of sample data sets as hyper-parameters, then using the y types of original data set feature vectors in less sample data and the y types of feature vectors generated in the step S51 as training data of a linear classifier, training the linear classifier by using cross entropy loss, and obtaining a classifier model:
and ∑ -logPr (y | x; θ) formula 11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110412397.5A CN113033698A (en) | 2021-04-16 | 2021-04-16 | Method for improving classification accuracy of few samples by using distribution strategy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110412397.5A CN113033698A (en) | 2021-04-16 | 2021-04-16 | Method for improving classification accuracy of few samples by using distribution strategy |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113033698A true CN113033698A (en) | 2021-06-25 |
Family
ID=76458022
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110412397.5A Pending CN113033698A (en) | 2021-04-16 | 2021-04-16 | Method for improving classification accuracy of few samples by using distribution strategy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113033698A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114328921A (en) * | 2021-12-27 | 2022-04-12 | 湖南大学 | Small sample entity relation extraction method based on distribution calibration |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109961089A (en) * | 2019-02-26 | 2019-07-02 | 中山大学 | Few-shot and zero-shot image classification methods based on metric learning and meta-learning |
-
2021
- 2021-04-16 CN CN202110412397.5A patent/CN113033698A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109961089A (en) * | 2019-02-26 | 2019-07-02 | 中山大学 | Few-shot and zero-shot image classification methods based on metric learning and meta-learning |
Non-Patent Citations (1)
| Title |
|---|
| SHUO YANG等: "Free lunch for few-shot learning：distribution calibration", 《ARXIV》, 16 January 2021 (2021-01-16), pages 3 - 5 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114328921A (en) * | 2021-12-27 | 2022-04-12 | 湖南大学 | Small sample entity relation extraction method based on distribution calibration |
| CN114328921B (en) * | 2021-12-27 | 2024-04-09 | 湖南大学 | Small sample entity relation extraction method based on distribution calibration |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Dong et al. | Deep manifold learning of symmetric positive definite matrices with application to face recognition | |
| CN107341463B (en) | Face feature recognition method combining image quality analysis and metric learning | |
| CN109711461B (en) | Transfer learning picture classification method and device based on principal component analysis | |
| CN112070768B (en) | Anchor-Free based real-time instance segmentation method | |
| CN103793702A (en) | Pedestrian re-identifying method based on coordination scale learning | |
| CN110457514A (en) | A Multi-label Image Retrieval Method Based on Deep Hash | |
| CN107066958A (en) | A kind of face identification method based on HOG features and SVM multi-categorizers | |
| CN105631416A (en) | Method for carrying out face recognition by using novel density clustering | |
| CN113076927A (en) | Finger vein identification method and system based on multi-source domain migration | |
| Zhao et al. | Deep Adaptive Log‐Demons: Diffeomorphic Image Registration with Very Large Deformations | |
| Kragh et al. | Predicting embryo viability based on self-supervised alignment of time-lapse videos | |
| CN112580502A (en) | SICNN-based low-quality video face recognition method | |
| CN107784284B (en) | Face recognition method and system | |
| CN109711442A (en) | Unsupervised layer-by-layer generation confrontation feature representation learning method | |
| CN113033698A (en) | Method for improving classification accuracy of few samples by using distribution strategy | |
| CN113033345B (en) | V2V video face recognition method based on public feature subspace | |
| CN110780270B (en) | A subspace feature extraction method for target library attribute discrimination local regular learning | |
| CN111553417B (en) | Image data dimension reduction method and system based on discriminant regularization local preserving projection | |
| CN115019175B (en) | A pest identification method based on transfer meta-learning | |
| CN114821219B (en) | Unsupervised multi-source domain adaptive classification method based on deep joint semantics | |
| CN116958809A (en) | Remote sensing small sample target detection method for feature library migration | |
| CN115331134A (en) | New golden monkey individual identification method | |
| CN110363227B (en) | LED classification method based on manifold learning | |
| CN114399422A (en) | Remote sensing image registration method based on local information and global information | |
| CN113378807A (en) | Continuous spectrum projection image identification method of class-consistent neighborhood |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |