CN111639679A - Small sample learning method based on multi-scale metric learning - Google Patents
Small sample learning method based on multi-scale metric learning Download PDFInfo
- Publication number
- CN111639679A CN111639679A CN202010384767.4A CN202010384767A CN111639679A CN 111639679 A CN111639679 A CN 111639679A CN 202010384767 A CN202010384767 A CN 202010384767A CN 111639679 A CN111639679 A CN 111639679A
- Authority
- CN
- China
- Prior art keywords
- scale
- sample
- learning
- feature mapping
- support set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a small sample learning method based on multi-scale metric learning, which comprises the following steps: step one, establishing a data set; step two, generating a multi-scale feature mapping layer; step three, transfer learning: the conversion module carries out secondary mapping on the multi-scale characteristics of the sample; step four, generating a multi-scale feature mapping pair; calculating a relationship score of the multi-scale feature mapping pair in the multi-scale relationship generation network; and step six, measuring the sample similarity by adopting a multi-scale measurement learning model. The invention has simple structure and reasonable design, obtains multi-scale feature mapping pairs through transfer learning, ensures that the trained model has mobility, adds a loss term brought by sample spacing to the whole model on the basis of a mean square error loss function to form a new loss function, realizes metric learning and is suitable for the training of small sample learning.
Description
Technical Field
The invention belongs to the technical field of image processing and recognition, and particularly relates to a small sample learning method based on multi-scale metric learning.
Background
Humans are very good at identifying a new object with a very small number of samples, for example, a small child may only need some pictures in a book to know what is "zebra" and what is "rhinoceros". Inspired by the fast Learning capability of human beings, researchers hope that after a machine Learning model learns a large amount of data of a certain class, only a small amount of samples are needed for a new class to fast learn, and the problem to be solved by Few-shot Learning is solved.
For machine learning, although deep learning on an image recognition task can obtain quite satisfactory results in some scenarios by virtue of a deep and complex network model, huge training data support and strong hardware support, general model training becomes quite difficult in some rare task scenarios because sufficient training samples cannot be obtained. Instead, a human can learn a new category with only a few reference samples. Therefore, in order to make the model have the same capability, learning based on a small sample that is learned quickly from a small number of samples becomes a more popular research direction in the field of image recognition recently.
Small sample learning aims to have the ability to identify unknown classes with a small number of training samples, which is the process by which the human brain associates and infers unknown things based on a priori knowledge. At the core of small sample learning, a new class is identified, which is the domain of migrating source knowledge to target knowledge. However, the conventional algorithm of the transfer learning cannot solve the small sample learning task well, wherein the most important difference is that the small sample learning needs to obtain the capability of identifying unknown classes, which means that the method has the principle of identifying a large number of classes which are not trained, and obviously the transfer learning cannot do the same.
Currently, small sample learning is mainly divided into three categories: transfer-based learning, metric-based learning, and meta-based learning. Transfer-based learning has been widely used in many fields, which is fine-tuned by a certain amount of self-labeling data on a pre-trained base network. The method based on metric learning mainly models the distance between samples, so that similar samples are close to heterogeneous samples and are far away from the heterogeneous samples. The meta-learning-based method aims to create a model so that the model can adapt to a multi-task learning scene, and for small sample learning tasks, because the training input types at each time are different, the training task at each time can be regarded as different learning tasks, and therefore the model can be trained by using the meta-learning method.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a small sample learning method based on multi-scale metric learning, which has a simple structure and a reasonable design, obtains multi-scale feature mapping pairs through migration learning, learns the multi-scale relationship pairs by constructing a hierarchical relationship learning network, so as to mine the feature association between a support set and an inquiry set, and adds a sample interval to a loss term brought by an overall model on the basis of a mean square error loss function to form a new loss function, so as to adapt to the training of small sample learning.
In order to solve the technical problems, the invention adopts the technical scheme that: a small sample learning method based on multi-scale metric learning is characterized by comprising the following steps:
step one, establishing a data set: establishing a support set SsuAnd a set of queries Squ，
Wherein xiRepresentation support set SsuThe ith sample in (1), yiRepresents a sample xiLabel of (2), xjRepresenting a set of queries SquThe jth sample in (2), yjRepresents a sample xjIs a positive integer, C × K represents the support set SsuC × N denotes the challenge set SquThe number of samples of (a);
step two, generating a multi-scale feature mapping layer:
step 201, generating a support set multi-scale feature mapping prototype: will support set SsuThe sample in the system is processed by a support set feature extraction module to obtain a plurality of support set multi-scale feature mapping prototypes;
step 202, generating a multi-scale feature mapping layer of the query set: will ask set SquObtaining a plurality of inquiry set multi-scale feature mapping layers by the inquiry set feature extraction module of each sample;
step three, transfer learning: the conversion module receives the support set multi-scale feature mapping prototype and maps the support set multi-scale feature mapping prototype into a feature space suitable for target knowledge to generate a support set multi-scale feature mapping layer;
step four, generating a multi-scale feature mapping pair: correspondingly combining the support set multi-scale feature mapping layer and the inquiry set multi-scale feature mapping layer to obtain a multi-scale feature mapping pair;
step five, calculating the relation score RES of the multi-scale feature mapping pair in the multi-scale relation generation networkω；
Step six, measuring the sample similarity by adopting a multi-scale measurement learning model: sample similarity Ltotal＝λLIIRL+μLavmseWherein
Representing a set of queries SquThe predicted average of the samples of the second ω -th class,
representation query set SquIn a predicted average value of samples not belonging to the ω -th class, atrRepresents the interval between the prediction average values of the homogeneous sample and the heterogeneous sample,
Lavmserepresenting the sample class prediction loss in the query set, and λ and μ represent multi-scale weights.
The small sample learning method based on multi-scale metric learning is characterized in that: in the sixth step
Wherein
The small sample learning method based on multi-scale metric learning is characterized in that: scoring the relationship in step five
Wherein RESωRepresents a relationship score, fgeAnd B represents a multi-scale relation generation network and a multi-scale relation generation module.
The small sample learning method based on multi-scale metric learning is characterized in that: step 201, in the generation of the support set multi-scale feature mapping prototype, the support set S is generatedsuAnd C × K samples are subjected to feature extraction to obtain a multi-scale feature mapping layer corresponding to each sample, and then the multi-scale feature mapping layers corresponding to the samples in each category are fused into the multi-scale feature mapping layer in the category.
The small sample learning method based on multi-scale metric learning is characterized in that: the support set feature extraction module in step 201 may be different from the query set feature extraction module in step 202, and the support set feature extraction module in step 201 is the same as the query set feature extraction module in step 202 in feature layer scale.
Compared with the prior art, the invention has the following advantages:
1. the invention has simple structure, reasonable design and convenient realization, use and operation.
2. In the application, a feature extraction module receives sample data from a support set and an inquiry set and maps the sample data to a feature space, the feature extraction module outputs sample multi-scale feature output, and a conversion module performs secondary mapping on the sample multi-scale feature and maps the sample multi-scale feature to another feature space suitable for target knowledge, so that a trained model has mobility and can identify a new class.
3. The method and the device have the advantages that a hierarchical relation learning network is constructed to learn multi-scale relation pairs, so that feature association between a support set and a query set is mined.
4. According to the method, a new loss function is reconstructed, and the multi-scale metric learning model adds a loss term brought by a sample interval to an integral model on the basis of a mean square error loss function to form a new loss function LtotalMetric learning can be achieved to adapt to training of small sample learning.
5. According to the method, a multi-scale metric learning model is built, a model is built, label prediction can be carried out on an inquiry set through a small number of reference samples by combining transfer learning and metric learning, and meanwhile the method has the capability of identifying new categories and improves the identification precision.
In conclusion, the invention has simple structure and reasonable design, and obtains the multi-scale feature mapping pair through transfer learning, so that the trained model has mobility, and the loss items brought by the sample spacing to the whole model are added on the basis of the mean square error loss function to form a new loss function, thereby realizing metric learning and adapting to the training of small sample learning.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a network architecture according to the present invention.
Fig. 3 is a schematic diagram of a network structure of the conversion module according to the present invention.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may be otherwise variously oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
As shown in fig. 1 to 3, the present invention includes the steps of:
step one, establishing a data set: establishing a support set SsuAnd a set of queries Squ，
Wherein xiRepresentation support set SsuThe ith sample in (1), yiRepresents a sample xiLabel of (2), xjRepresenting a set of queries SquThe jth sample in (2), yjRepresents a sample xjIs a positive integer, C × K represents the support set SsuC × N denotes the challenge set SquThe number of samples.
It should be noted that the image set with the labeled sample is the support set SsuSupport set SsuComprises C classes randomly drawn from a training set, each class comprising K samples, a support set SsuA total of C × K sample images, each support set SsuAll consisting of a two-dimensional vector (x)i,yi) And (4) showing. It should be noted that the support set SsuThe C classes of (a) may be not previously learned.
The set of images to be classified is represented as a query set SquQuery set SquExtracting a portion of the remaining data from the C classes of the training set, and thus, the query set SquAlso included are classes C, each class comprising N samples, a set of queries SquTotal C × N sample images, each challenge set SquAll consisting of a two-dimensional vector (x)j,yj) And (4) showing.
The invention aims to solve the problems that a model is constructed, migration learning and metric learning are combined, label prediction can be carried out on an inquiry set through a small number of reference samples, a new class can be identified, and the identification precision is improved.
Step two, generating a multi-scale feature mapping layer:
step 201, generating a support set multi-scale feature mapping prototype: will support set SsuThe sample in (1) is processed by a support set feature extraction module to obtain a plurality of support set multi-scale feature mapping prototypes.
Feature extraction module ffeReceiving data from a support set SsuMapping the sample data to a feature space, and using a feature pyramid structure to support the set SsuC × K samples are subjected to feature extraction to obtain a multi-scale feature mapping layer corresponding to each sample, and then the multi-scale feature mapping layers corresponding to each sample in each category are fused into the multi-scale feature mapping layer of the category, which is called as a support set multi-scale feature mapping prototype, so that a support set S is obtainedsuAnd obtaining C support set multi-scale feature mapping prototypes. In this embodiment, the fusion uses a weighted average method.
When in actual use, the computer is according to the formula
Calculating to obtain support set multi-scale feature mapping, wherein VsuRepresentation support set SsuM represents a feature layer scale, ffeRepresentation support set SsuAdopted feature extraction module, feature extraction module ffeThe method mainly comprises a deep convolutional neural network which is pre-trained in a large scene classification data set. In practice, m is 3. In specific implementation, the feature extraction module ffeIt is made up of a convolutional neural network ResNet50 with its fully connected layers removed. The ResNet50 network generally consists of a stack of four different bottleneck layer structures, and since there is no change in feature layer dimensions between the same bottleneck layer, the tops of the last three different bottleneck layers are selected as inputs of the multi-scale features.
Step 202, generating a multi-scale feature mapping layer of a query set
Computer according to formula
Calculating to obtain a multi-scale feature map of the query set, wherein VquRepresenting a set of queries SquCharacteristic vector of ffuRepresenting a set of queries SquAnd the adopted feature extraction module.
In actual use, the feature extraction module ffuMainly comprises a deep convolutional neural network pre-trained in a large scene classification data set, and a feature extraction module ffuReceiving a query from a query set SquAnd maps it to the feature space. For each input sample xj, the feature extraction module ffeWill output a multi-scale feature mapping layer corresponding thereto, and thus query set SquResulting in C × N different classes of multi-scale feature mapping layers.
It should be noted that the query set SquFeature extraction module f adoptedfuAnd support set SsuFeature extraction module f adoptedfeThe characteristic layer dimension m of (2) is the same, and m is 3.
Step three, transfer learning: the conversion module receives the support set multi-scale feature mapping prototype and maps the support set multi-scale feature mapping prototype into a feature space adaptive to target knowledge to generate a support set multi-scale feature mapping layer.
Unlike other small sample learning methods, in the present application, the feature extraction module ffeThe output is the multi-scale feature output of the sample, simultaneously, a corresponding conversion module is designed, secondary mapping is carried out on the feature embedding, and a support set S is used based on the feature mapping methodsuFrom the original high-dimensional feature space to the low-dimensional feature space where the support set S issuC support set multi-scale feature mapping prototypes and query set SquThe multi-scale feature mapping layers of the middle C × N different classes have the same or close distribution, so that the labeled support set S represented by the low-dimensional space can be utilizedsuSample data pair query set SquAnd predicting the sample data, so that the trained model has mobility and the capability of identifying a new category.
As shown in FIG. 2, in actual use, the conversion module adopts threeAnd the three convolutional neural networks are respectively called a first conversion block, a second conversion block and a third conversion block. Input of the first conversion block is a feature extraction module ffeThe third layer outputs, and the input of the second conversion block is a characteristic extraction module ffeThe output of the second layer and the output of the first conversion block, and the input of the third conversion block is a feature extraction module ffeA first layer output and an output of the second conversion block. The output of the multi-scale feature mapping layer after passing through the first conversion block, the second conversion block and the third conversion block is a feature matrix with the same dimension as the scale.
As shown in fig. 3, conv denotes a volume block and interplate denotes an interpolation block. Input1 of the conversion module is the output from the top layer of the convolutional neural network ResNet50, and input2 is the output of the upper layer conversion block which is deconvolved to have the same size characteristic as input 1. The conversion module can combine more accurate feature semantic expression of the upper conversion block and more comprehensive target feature information of the lower conversion block by using a top-to-bottom structure.
Step four, generating a multi-scale feature mapping pair: and correspondingly combining the support set multi-scale feature mapping layer and the inquiry set multi-scale feature mapping layer to obtain a multi-scale feature mapping pair.
In actual use, the query set SquMiddle C × N multi-scale feature mapping layers and support set SsuAnd obtaining corresponding scales of the C classes of multi-scale feature mapping layers, and performing head-to-tail connection to obtain C × (C × N) multi-scale feature mapping pairs, wherein the multi-scale feature mapping pairs are used as the input of a multi-scale relation generation network.
Step five, calculating the relationship score of the multi-scale feature mapping pair in the multi-scale relationship generation network: relationship score
Wherein RESωRepresents a relationship score, fgeAnd B represents a multi-scale relation generation network and a multi-scale relation generation module.
The multi-scale relation generation module B mainly learns the multi-scale by constructing a hierarchical relation learning networkDegree relation pair, thereby mining support set SsuAnd a set of queries SquThe characteristic association between them.
For a multi-scale relational mapping pair, a lower scale means that the features represented by the feature mapping layer are coarser and also more common, whereas a higher scale means that the features represented by the feature mapping are finer and more unique. Therefore, the feature mapping pairs with lower scales need to be subjected to relationship learning in a deeper network, and the feature mapping pairs with higher scales are more obvious in difference or similarity, so that only a shallower network is needed to be sufficient. Based on the thought, a hierarchical relation learning network is used for carrying out relation learning on the multi-scale feature mapping pair, namely the multi-scale relation generation network, and the low learning level network and the high learning level network share parameters.
Obtaining a relationship score RES for learning results in the hierarchical relationship learning network by integrating the multi-scale feature mappingωTo screen out features that are favorable to the relationship. RES (resource representation)ωValues limited to [0, 1 ]]Relationship score RESωReflecting from the query set SquA certain sample and support set SsuSimilarity between classes. Score RES for C relationshipsωCarry out sequencing, RESωThe higher the value, the higher the query set SquThe j sample x in (1)jCorresponding multi-scale feature mapping layer
And support set SsuMulti-scale feature mapping layer of the middle omega category
The closer the relationship of (a) is, the greater the probability of belonging to the same class is, thereby realizing sample classification.
Multi-scale relationship generation network fgeThe method can enable the feature layer with larger scale to use deeper network for learning, and simultaneously, the learning network with small scale and the learning network with large scale share the training parameters, so that the complexity of the network is reduced. Multi-scale relationship generation network fgeIs composed of a graded nerveThe network is constructed by adopting a four-layer feedforward convolutional neural network, the convolutional neural network comprises convolutional layers, full-link layers and a classifier, when the network is actually used, the first full-link layer uses ReLU as an activation function, and the second full-link layer uses sigmoid as an activation function. A four-layer convolution structure is used to handle the most scaled feature map pair and the number of layers of the convolution structure will decrease as its scale decreases, while the low scale will share network training parameters with the larger scale.
Step six, measuring the sample similarity by adopting a multi-scale measurement learning model: sample similarity Ltotal＝λLIIRL+μLavmseWherein
Representing a set of queries SquThe predicted average of the samples of the second ω -th class,
representation query set SquIn a predicted average value of samples not belonging to the ω -th class, atrRepresents the interval between the prediction average values of the homogeneous sample and the heterogeneous sample,
λ and μ represent multi-scale weights.
The sample classes of each round are uncertain in the small sample learning and training process, and the training samples are few, so that the problems of slow convergence or poor stability of the model trained to the later stage may exist in the general mean square error loss function and the cross entropy loss function. Since the metric learning model in the present application is a hierarchical depth metric network, a new loss function needs to be designed so that the network can implement metric learning.
The method reconstructs a new loss function, and obtains a multi-scale metric learning model L through loss function trainingavmseUsing the mean square error loss function, LIIRLRepresenting the loss term brought by the sample spacing to the whole model, and the multi-scale metric learning model has a mean square error loss function LavmseOf (2) aOn the basis of adding a loss term L brought by the sample spacing to the whole modelIIRLForming a new loss function LtotalTo adapt to the training of small sample learning.
In addition, L isavmseCross entropy loss function, mean square error loss function or other metric distance functions commonly used by metric learning models can also be selected.
As shown in table 1, the experimental results of the multi-scale metric learning model used in the present application and other learning models in the prior art on the miniImageNet data set are compared, the multi-scale metric learning model is represented by MSML in the table, small sample image classification may only provide very few samples, and the experiment includes a 5-way 1-shot comparison benchmark test and a 5-way 5-shot comparison benchmark test. As shown in table 1, the multi-scale metric learning model (i.e., MSML) obtained good experimental results on the miniImageNet dataset, including an accuracy of 72% achieved on the 5-way 1-shot comparison benchmark test, and also obtained an accuracy of 84% on the 5-way 5-shot task, which is much higher than the experimental results of other learning modules in the prior art.
TABLE 1 comparison of experimental results of multi-scale metric learning model and other learning models in miniImageNet dataset
In table 1, Matching Network represents a Matching Network model, Meta-leaner LSTM represents a Meta learning loop model, MAML represents a model independent learning model, complex Nets represents a comparison Network model, SNAIL represents an attention-based Meta learning model, adarsenet represents an adaptive residual Network model, Prototypical Nets represents a prototype Network model, K-tuple Net represents a K-tuple Network model, MTL represents a Meta migration learning model, and LEO represents a hidden space optimization model.
As shown in table 2, the multi-scale metric learning model used in the present application and the experimental results of other learning models in the prior art on the tiered imagenet data set are compared, the multi-scale metric learning model is represented by MSML in the table, and the experiment includes a 5-way 1-shot comparison benchmark test and a 5-way 5-shot comparison benchmark test. As shown in table 2, the multi-scale metric learning model obtains a good experimental result on the tieedimagenet dataset, including 70% of accuracy achieved on the 5-way 1-shot comparison benchmark test, and also obtains 83% of accuracy on the 5-way 5-shot task, which is much higher than the experimental results of other learning modules in the prior art.
TABLE 2 comparison of experimental results of the multi-scale metric learning model and other learning models in the tieredIMAGENET dataset
As shown in table 3, the classification accuracy of the loss function used in the present application and other loss functions in the prior art under the same conditions were compared. As shown in Table 3, the loss function proposed in the present application achieves the highest classification accuracy regardless of whether the 5-way 1-shot task or the 5-way 5-shot task is performed.
TABLE 3 comparison table of classification accuracy between loss function of the present application and other loss functions
The above embodiments are only examples of the present invention, and are not intended to limit the present invention, and all simple modifications, changes and equivalent structural changes made to the above embodiments according to the technical spirit of the present invention still fall within the protection scope of the technical solution of the present invention.
Claims (5)
1. A small sample learning method based on multi-scale metric learning is characterized by comprising the following steps:
step one, establishing a data set: establishing a support set SsuAnd a set of queries Squ，
Wherein xiRepresentation support set SsuThe ith sample in (1), yiRepresents a sample xiLabel of (2), xjRepresenting a set of queries SquThe jth sample in (2), yjRepresents a sample xjIs a positive integer, C × K represents the support set SsuC × N denotes the challenge set SquThe number of samples of (a);
step two, generating a multi-scale feature mapping layer:
step 201, generating a support set multi-scale feature mapping prototype: will support set SsuThe sample in the system is processed by a support set feature extraction module to obtain a plurality of support set multi-scale feature mapping prototypes;
step 202, generating a multi-scale feature mapping layer of the query set: will ask set SquThe sample in the system is subjected to an inquiry set feature extraction module to obtain a plurality of inquiry set multi-scale feature mapping layers;
step three, transfer learning: the conversion module receives the support set multi-scale feature mapping prototype and maps the support set multi-scale feature mapping prototype into a feature space suitable for target knowledge to generate a support set multi-scale feature mapping layer;
step four, generating a multi-scale feature mapping pair: correspondingly combining the support set multi-scale feature mapping layer and the inquiry set multi-scale feature mapping layer to obtain a multi-scale feature mapping pair;
step five, calculating the relation score RES of the multi-scale feature mapping pair in the multi-scale relation generation networkω；
Step six, measuring the sample similarity by adopting a multi-scale measurement learning model: sample similarity Ltotal＝λLIIRL+μLavmseWherein
Representing a set of queries SquThe predicted average of the samples of the second ω -th class,
representation query set SquIn a predicted average value of samples not belonging to the ω -th class, atrRepresents the interval between the prediction average values of the homogeneous sample and the heterogeneous sample,
Lavmserepresenting the sample class prediction loss in the query set, and λ and μ represent multi-scale weights.
2. The small sample learning method based on multi-scale metric learning according to claim 1, characterized in that: in the sixth step
Wherein
3. The small sample learning method based on multi-scale metric learning according to claim 1, characterized in that: scoring the relationship in step five
Wherein RESωRepresents a relationship score, fgeAnd B represents a multi-scale relation generation network and a multi-scale relation generation module.
4. The small sample learning method based on multi-scale metric learning according to claim 1, characterized in that: step 201, in the generation of the support set multi-scale feature mapping prototype, the support set S is generatedsuC × K samples are subjected to feature extraction to obtain a multi-scale feature mapping layer corresponding to each sample, and then the multi-scale feature mapping layers corresponding to the samples in each category are fused into the multi-scale feature mapping of the categoryAnd (4) irradiating the layer.
5. The small sample learning method based on multi-scale metric learning according to claim 1, characterized in that: the support set feature extraction module in step 201 may be different from the query set feature extraction module in step 202, and the support set feature extraction module in step 201 is the same as the query set feature extraction module in step 202 in feature layer scale.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010384767.4A CN111639679B (en) | 2020-05-09 | 2020-05-09 | Small sample learning method based on multi-scale metric learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010384767.4A CN111639679B (en) | 2020-05-09 | 2020-05-09 | Small sample learning method based on multi-scale metric learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111639679A true CN111639679A (en) | 2020-09-08 |
| CN111639679B CN111639679B (en) | 2022-03-04 |
Family
ID=72328562
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010384767.4A Active CN111639679B (en) | 2020-05-09 | 2020-05-09 | Small sample learning method based on multi-scale metric learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111639679B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112215280A (en) * | 2020-10-12 | 2021-01-12 | 西安交通大学 | Small sample image classification method based on meta-backbone network |
| CN112785479A (en) * | 2021-01-21 | 2021-05-11 | 南京信息工程大学 | Image invisible watermark universal detection method based on less-sample learning |
| CN112949740A (en) * | 2021-03-17 | 2021-06-11 | 重庆邮电大学 | Small sample image classification method based on multilevel measurement |
| CN113057657A (en) * | 2021-03-22 | 2021-07-02 | 华南理工大学 | Electroencephalogram emotion classification method based on multi-scale connectivity characteristics and element migration learning |
| CN113139536A (en) * | 2021-05-12 | 2021-07-20 | 哈尔滨工业大学（威海） | Text verification code identification method and equipment based on cross-domain meta learning and storage medium |
| CN113269734A (en) * | 2021-05-14 | 2021-08-17 | 成都市第三人民医院 | Tumor image detection method and device based on meta-learning feature fusion strategy |
| CN113344729A (en) * | 2021-06-04 | 2021-09-03 | 中国石油大学(华东) | Residual oil submergence digging method based on small sample learning |
| CN113642665A (en) * | 2021-08-24 | 2021-11-12 | 广州市香港科大霍英东研究院 | Relation network-based few-sample classification method and system |
| CN114202028A (en) * | 2021-12-13 | 2022-03-18 | 四川大学 | Rolling bearing life stage identification method based on MAMTL |
| CN114584471A (en) * | 2020-11-16 | 2022-06-03 | 中国移动通信有限公司研究院 | Model training method and device for network data analysis function based on federal learning |
| CN114612702A (en) * | 2022-01-24 | 2022-06-10 | 珠高智能科技(深圳)有限公司 | Image data annotation system and method based on deep learning |
| CN114757832A (en) * | 2022-06-14 | 2022-07-15 | 之江实验室 | Face super-resolution method and device based on cross convolution attention antagonistic learning |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105550426A (en) * | 2015-12-08 | 2016-05-04 | 东北大学 | Multi-scale binary tree blast furnace fault diagnosis method based on sample segmentation |
| US20160196384A1 (en) * | 2015-01-06 | 2016-07-07 | Siemens Aktiengesellschaft | Personalized whole-body circulation in medical imaging |
| CN107507170A (en) * | 2017-07-28 | 2017-12-22 | 中国民航大学 | A kind of airfield runway crack detection method based on multi-scale image information fusion |
| CN109376578A (en) * | 2018-08-27 | 2019-02-22 | 杭州电子科技大学 | A kind of small sample target identification method based on depth migration metric learning |
| CN109685135A (en) * | 2018-12-21 | 2019-04-26 | 电子科技大学 | A kind of few sample image classification method based on modified metric learning |
| CN109961089A (en) * | 2019-02-26 | 2019-07-02 | 中山大学 | Small sample and zero sample image classification method based on metric learning and meta learning |
| CN110020682A (en) * | 2019-03-29 | 2019-07-16 | 北京工商大学 | A kind of attention mechanism relationship comparison net model methodology based on small-sample learning |
| CN110276394A (en) * | 2019-06-21 | 2019-09-24 | 扬州大学 | Power equipment classification method based on deep learning under a kind of small sample |
| CN110414600A (en) * | 2019-07-27 | 2019-11-05 | 西安电子科技大学 | A kind of extraterrestrial target small sample recognition methods based on transfer learning |
| CN110490249A (en) * | 2019-08-16 | 2019-11-22 | 哈尔滨工业大学 | Structural Damage Identification based on attribute class relations and few sample meta learning |
| CN110569886A (en) * | 2019-08-20 | 2019-12-13 | 天津大学 | Image classification method for bidirectional channel attention element learning |
| CN110717554A (en) * | 2019-12-05 | 2020-01-21 | 广东虚拟现实科技有限公司 | Image recognition method, electronic device, and storage medium |
| CN110879989A (en) * | 2019-11-22 | 2020-03-13 | 四川九洲电器集团有限责任公司 | Ads-b signal target identification method based on small sample local machine learning model |
-
2020
- 2020-05-09 CN CN202010384767.4A patent/CN111639679B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160196384A1 (en) * | 2015-01-06 | 2016-07-07 | Siemens Aktiengesellschaft | Personalized whole-body circulation in medical imaging |
| CN105550426A (en) * | 2015-12-08 | 2016-05-04 | 东北大学 | Multi-scale binary tree blast furnace fault diagnosis method based on sample segmentation |
| CN107507170A (en) * | 2017-07-28 | 2017-12-22 | 中国民航大学 | A kind of airfield runway crack detection method based on multi-scale image information fusion |
| CN109376578A (en) * | 2018-08-27 | 2019-02-22 | 杭州电子科技大学 | A kind of small sample target identification method based on depth migration metric learning |
| CN109685135A (en) * | 2018-12-21 | 2019-04-26 | 电子科技大学 | A kind of few sample image classification method based on modified metric learning |
| CN109961089A (en) * | 2019-02-26 | 2019-07-02 | 中山大学 | Small sample and zero sample image classification method based on metric learning and meta learning |
| CN110020682A (en) * | 2019-03-29 | 2019-07-16 | 北京工商大学 | A kind of attention mechanism relationship comparison net model methodology based on small-sample learning |
| CN110276394A (en) * | 2019-06-21 | 2019-09-24 | 扬州大学 | Power equipment classification method based on deep learning under a kind of small sample |
| CN110414600A (en) * | 2019-07-27 | 2019-11-05 | 西安电子科技大学 | A kind of extraterrestrial target small sample recognition methods based on transfer learning |
| CN110490249A (en) * | 2019-08-16 | 2019-11-22 | 哈尔滨工业大学 | Structural Damage Identification based on attribute class relations and few sample meta learning |
| CN110569886A (en) * | 2019-08-20 | 2019-12-13 | 天津大学 | Image classification method for bidirectional channel attention element learning |
| CN110879989A (en) * | 2019-11-22 | 2020-03-13 | 四川九洲电器集团有限责任公司 | Ads-b signal target identification method based on small sample local machine learning model |
| CN110717554A (en) * | 2019-12-05 | 2020-01-21 | 广东虚拟现实科技有限公司 | Image recognition method, electronic device, and storage medium |
Non-Patent Citations (7)
| Title |
|---|
| FLOOD SUNG等: "Learning to Compare: Relation Network for Few-Shot Learning", 《2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| KAI-HSIANG CHENG等: "Multi-label Few-shot Learning for Sound Event Recognition", 《2019 IEEE 21ST INTERNATIONAL WORKSHOP ON MULTIMEDIA SIGNAL PROCESSING (MMSP)》 * |
| YUEMING DINGDENG: "Multi-scale Relation Network for Few-Shot Learning Based on Meta-learning", 《COMPUTER VISION SYSTEMS》 * |
| 宋丽丽: "迁移度量学习行人再识别算法", 《计算机工程与应用》 * |
| 张凯: "基于度量学习的行人再识别算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 田霞: "基于元学习的少样本图像分类方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 董虎胜: "基于度量学习的行人再识别研究", 《中国博士学位论文全文数据库 信息科技辑》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112215280B (en) * | 2020-10-12 | 2022-03-15 | 西安交通大学 | Small sample image classification method based on meta-backbone network |
| CN112215280A (en) * | 2020-10-12 | 2021-01-12 | 西安交通大学 | Small sample image classification method based on meta-backbone network |
| CN114584471A (en) * | 2020-11-16 | 2022-06-03 | 中国移动通信有限公司研究院 | Model training method and device for network data analysis function based on federal learning |
| CN112785479A (en) * | 2021-01-21 | 2021-05-11 | 南京信息工程大学 | Image invisible watermark universal detection method based on less-sample learning |
| CN112785479B (en) * | 2021-01-21 | 2023-05-23 | 南京信息工程大学 | Image invisible watermark universal detection method based on few sample learning |
| CN112949740A (en) * | 2021-03-17 | 2021-06-11 | 重庆邮电大学 | Small sample image classification method based on multilevel measurement |
| CN113057657A (en) * | 2021-03-22 | 2021-07-02 | 华南理工大学 | Electroencephalogram emotion classification method based on multi-scale connectivity characteristics and element migration learning |
| CN113057657B (en) * | 2021-03-22 | 2022-09-13 | 华南理工大学 | Electroencephalogram emotion classification method based on multi-scale connectivity characteristics and element migration learning |
| CN113139536A (en) * | 2021-05-12 | 2021-07-20 | 哈尔滨工业大学（威海） | Text verification code identification method and equipment based on cross-domain meta learning and storage medium |
| CN113269734A (en) * | 2021-05-14 | 2021-08-17 | 成都市第三人民医院 | Tumor image detection method and device based on meta-learning feature fusion strategy |
| CN113344729A (en) * | 2021-06-04 | 2021-09-03 | 中国石油大学(华东) | Residual oil submergence digging method based on small sample learning |
| CN113642665A (en) * | 2021-08-24 | 2021-11-12 | 广州市香港科大霍英东研究院 | Relation network-based few-sample classification method and system |
| CN114202028A (en) * | 2021-12-13 | 2022-03-18 | 四川大学 | Rolling bearing life stage identification method based on MAMTL |
| CN114202028B (en) * | 2021-12-13 | 2023-04-28 | 四川大学 | MAMTL-based rolling bearing life stage identification method |
| CN114612702A (en) * | 2022-01-24 | 2022-06-10 | 珠高智能科技(深圳)有限公司 | Image data annotation system and method based on deep learning |
| CN114757832A (en) * | 2022-06-14 | 2022-07-15 | 之江实验室 | Face super-resolution method and device based on cross convolution attention antagonistic learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111639679B (en) | 2022-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111639679B (en) | Small sample learning method based on multi-scale metric learning | |
| CN107122375B (en) | Image subject identification method based on image features | |
| Liu et al. | Progressive neural architecture search | |
| Liu et al. | Learning spatio-temporal representations for action recognition: A genetic programming approach | |
| CN109558942B (en) | Neural network migration method based on shallow learning | |
| Zhou et al. | Deep supervised and convolutional generative stochastic network for protein secondary structure prediction | |
| CN108132968A (en) | Network text is associated with the Weakly supervised learning method of Semantic unit with image | |
| CN106067042B (en) | Polarization SAR classification method based on semi-supervised depth sparseness filtering network | |
| CN109670576B (en) | Multi-scale visual attention image description method | |
| Sudharshan et al. | Object recognition in images using convolutional neural network | |
| CN108875076B (en) | Rapid trademark image retrieval method based on Attention mechanism and convolutional neural network | |
| CN110516095A (en) | Weakly supervised depth Hash social activity image search method and system based on semanteme migration | |
| Wang et al. | Modulated convolutional networks | |
| Qi et al. | Personalized sketch-based image retrieval by convolutional neural network and deep transfer learning | |
| CN108228674A (en) | A kind of information processing method and device based on DKT | |
| CN109325513A (en) | A kind of image classification network training method based on magnanimity list class single image | |
| CN110263855A (en) | A method of it is projected using cobasis capsule and carries out image classification | |
| Tavanaei et al. | Towards recognition of protein function based on its structure using deep convolutional networks | |
| Castellano et al. | Integrating contextual knowledge to visual features for fine art classification | |
| CN114329031A (en) | Fine-grained bird image retrieval method based on graph neural network and deep hash | |
| CN110188621A (en) | A kind of three-dimensional face expression recognition methods based on SSF-IL-CNN | |
| CN110197213A (en) | Image matching method, device and equipment neural network based | |
| Thiodorus et al. | Convolutional neural network with transfer learning for classification of food types in tray box images | |
| Gkoutos et al. | Chemical machine vision: automated extraction of chemical metadata from raster images | |
| CN114004357A (en) | Neural network training method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |