CN114511739A - Task-adaptive small sample image classification method based on meta-migration learning - Google Patents

Task-adaptive small sample image classification method based on meta-migration learning Download PDF

Info

Publication number
CN114511739A
CN114511739A CN202210089412.1A CN202210089412A CN114511739A CN 114511739 A CN114511739 A CN 114511739A CN 202210089412 A CN202210089412 A CN 202210089412A CN 114511739 A CN114511739 A CN 114511739A
Authority
CN
China
Prior art keywords
meta
task
learning
samples
learner
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210089412.1A
Other languages
Chinese (zh)
Inventor
初妍
谢天文
莫士奇
李松
时洁
曹宇辰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Engineering University
Original Assignee
Harbin Engineering University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Engineering University filed Critical Harbin Engineering University
Priority to CN202210089412.1A priority Critical patent/CN114511739A/en
Publication of CN114511739A publication Critical patent/CN114511739A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention belongs to the technical field of computers, and particularly relates to a task self-adaptive small sample image classification method based on meta-migration learning. The method makes up the problem that the characteristics of the MAML model extracted by adopting a 4Conv shallow network are insufficient by combining element migration learning; trainable parameters are added to learn the use of balance element knowledge in each task, and the problems of unbalanced task, unbalanced category and unbalanced distribution of small sample learning in a real scene are solved. The invention selects samples with low accuracy in each task, and recombines the data of the samples to make the samples become more difficult tasks, so that the accuracy of the model is improved in the process of learning the more difficult tasks by the meta-learner. The difficult task mining algorithm provided by the invention collects samples with poor classification effect on line to form difficult tasks, so that a learner can learn the difficult tasks more quickly and with better effect.

Description

Task-adaptive small sample image classification method based on meta-migration learning
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a task self-adaptive small sample image classification method based on meta-migration learning.
Background
Deep learning has had great success in various recognition tasks, however, the parameters of deep neural networks are numerous and require sufficiently labeled samples to train models, which severely limits their scalability. For many rare classes, it is not feasible to collect a large number of training samples, on the contrary, people often need to see some examples of fresh things to quickly identify a new object class, and as inspired by the Learning ability of human beings, researchers hope that machine Learning models can also quickly learn under the condition of few training samples, and the key of having the quick Learning ability is to acquire a priori knowledge, that is, a large number of tasks are needed to train the models, so that the models continuously summarize the priori knowledge in the process of quickly Learning new knowledge, and thus, the machine Learning models can be helped to make correct classification only by a small number of samples when encountering unidentified classes, that is, the concept of small sample Learning (Few-Shot-Learning). In the small sample learning problem, a training set is composed of a group of base classes each of which has enough training samples and a group of new classes each of which has only a small number of labeled samples (shots), and the small sample learning aims to achieve better generalization performance by transmitting the knowledge of the base classes through the training of a small number of new class samples of a model.
Meta-learning is applied to the field of small sample learning as a framework, and how to adapt to a new task with only a few labeled samples is learned by utilizing a large number of similar tasks. However, meta-learning uses a shallow network as a feature extractor to avoid overfitting of the model, resulting in insufficient extraction of features. In addition, the existing meta-learning methods for small sample classification assume that the number of instances of each task and category is fixed, even if the number of instances of each task and category is greatly different, meta-learners still learn by using meta-knowledge equally in all tasks, and they do not consider that the distribution difference of tasks is not seen, and the meta-knowledge learned on a training set may not be useful, which constitute the problems of classification imbalance, task imbalance and distribution imbalance in small sample learning. At present, research aiming at learning of unbalanced small samples is less, and how to handle the small sample learning problem closer to a real scene is one of the problems to be solved urgently at present.
Disclosure of Invention
The invention aims to solve the problems that the feature extraction is insufficient when a shallow network is adopted to extract features in a meta-learning method and the existing small sample learning method does not consider imbalance in a real scene, and provides a task self-adaptive small sample image classification method based on meta-migration learning.
A task self-adaptive small sample image classification method based on meta-migration learning comprises the following steps:
step 1: acquiring a large-scale image data set, pre-training a deep network by using samples in the large-scale image data set, and outputting weight parameter vectors theta and theta of a group of feature extractors and classifiers;
step 2: freezing the weight parameter vector theta of the feature extractor, and resetting the weight parameter vector theta of the classifier;
and step 3: acquiring a small sample image data set to be classified, and dividing the small sample image data set into a meta-training set and a meta-testing set; the meta-training set comprises a support set and a query set, wherein the support set is used for updating parameters of the basic learner, and the query set is used for calculating a loss function and updating the meta-learner;
and 4, step 4: initializing parameters of a meta learner and a basic learner, and setting the maximum iteration times; the basic learner is used for rapidly learning new knowledge from a new task, and the meta learner is used for summarizing experiences in all learned knowledge; the small sample learning takes tasks as a unit, and each task comprises N classes;
and 5: randomly extracting samples from the meta-training set to form a training task, then randomly extracting the training task to train a basic learner, continuously updating and optimizing meta-migration learning parameters and balance parameters, and obtaining a class in which the samples with low identification accuracy are located as a difficult class in the step;
step 6: randomly sampling from the difficult class set to form a difficult task;
and 7: randomly extracting difficult tasks to train a basic learner, and continuously updating optimized learner parameters, meta migration learning parameters and balance parameters until the loss is lower than a threshold value;
and 8: judging whether the maximum iteration times is reached; if the maximum iteration times are not reached, clearing the difficult class set, and returning to the step 5 for the next iteration; if the maximum iteration times are reached, outputting the element migration learning parameters and the balance parameters after the iteration is finished to obtain a trained basic learner;
and step 9: and inputting the meta-test set into a trained basic learning device to obtain a small sample image classification result.
Further, when pre-training in step 1, the loss function is:
Figure BDA0003488567620000021
wherein D is sample data; f. of[Θ；θ]Representing a model equation comprising weight parameter vectors Θ and θ of the feature extractor and the classifier; x, y denote the specimen and label, respectively;
the updating formula of the weight parameter vectors theta and theta of the feature extractor and the classifier is as follows:
Figure BDA0003488567620000022
further, the collection process of the difficulty classes in the step 5 is as follows:
step 5.1: acquiring a support set and a query set of a current input task;
step 5.2: calculating a loss function supporting centralized samples and updating a basic learner parameter [ theta ]; theta]Meta migration learning parameters
Figure BDA0003488567620000023
Equilibrium parameter omegani,γi,zi；
(Θ,θ0)＝g(zi)(Θ,θ)
Where the subscript i denotes the index of the task, i 1, 2.., N; g (·) is a non-negative activation function, and the above equation is an initialization equation of the model by balancing the variable zτTo determine how much meta-knowledge to use;
Figure BDA0003488567620000031
wherein mu is the learning rate of the meta migration learning parameter; l isT(query)A loss function that is a sample of the query set;
Figure BDA0003488567620000032
wherein f (·) max (0, min (·,1)) is a clipping function; alpha is the task learning rate; series of scalars
Figure BDA0003488567620000033
Processing the class imbalance as a coefficient of the per-task gradient descent step loss;
step 5.3: and extracting samples from the query set, calculating the classification accuracy of the samples, arranging the samples in ascending order according to the accuracy, and taking the class where the m lowest samples are located as a difficult class, wherein m is a preset constant.
The invention has the beneficial effects that:
the method makes up the problem that the characteristics of the MAML model extracted by adopting a 4Conv shallow network are insufficient by combining element migration learning; trainable parameters are added to learn the use of balance element knowledge in each task, and the problems of unbalanced task, unbalanced category and unbalanced distribution of small sample learning in a real scene are solved. The invention selects samples with low accuracy in each task, and recombines the data of the samples to make the samples become more difficult tasks, so that the accuracy of the model is improved in the process of learning the more difficult tasks by the meta-learner. The difficult task mining algorithm provided by the invention collects samples with poor classification effect on line to form difficult tasks, so that a learner can learn the difficult tasks more quickly and with better effect.
Drawings
FIG. 1 is a general block diagram of a model in the present invention.
FIG. 2 is a flow chart of the training of the model of the present invention.
FIG. 3 is a block diagram of a feature extractor ResNet-12 of the present invention.
Fig. 4 is a structural diagram of an inference network in the present invention.
FIG. 5 is a pseudo code diagram of a specific algorithm in the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
In order to solve the problems that the feature extraction is insufficient when the feature extraction method adopts a shallow network to extract features and the existing small sample learning method does not consider imbalance in a real scene, the invention provides a task-adaptive small sample image classification method (MT-TAML) based on meta-migration learning, which adopts the following technical scheme:
a task-adaptive small sample image classification method based on meta-migration learning is disclosed, wherein a model is generally shown in FIG. 1, and a model training process is shown in FIG. 2, and comprises the following steps:
step 1: acquiring a large-scale image data set (such as ImageNet), pre-training a deep network by using samples in the large-scale image data set, and outputting weight parameter vectors theta and theta of a group of feature extractors and classifiers;
in pre-training, the loss function is:
Figure BDA0003488567620000041
wherein D is sample data; f. of[Θ；θ]Representing a model equation comprising weight parameter vectors Θ and θ of the feature extractor and the classifier; x, y denote the specimen and label, respectively;
the feature extractor employs ResNet-12, as shown in FIG. 3; the updating formula of the weight parameter vectors theta and theta of the feature extractor and the classifier is as follows:
Figure BDA0003488567620000042
step 2: freezing the weight parameter vector theta of the feature extractor, and resetting the weight parameter vector theta of the classifier;
and step 3: acquiring a small sample image data set to be classified, and dividing the small sample image data set into a meta-training set and a meta-testing set; the meta-training set comprises a support set and a query set, wherein the support set is used for updating parameters of the basic learner, and the query set is used for calculating a loss function and updating the meta-learner;
and 4, step 4: initializing parameters of a meta learner and a basic learner, and setting the maximum iteration times; the basic learner is used for rapidly learning new knowledge from a new task, and the meta learner is used for summarizing experiences in all learned knowledge; the small sample learning takes tasks as a unit, and each task comprises N classes;
and 5: randomly extracting samples from the meta-training set to form a training task, then randomly extracting the training task to train a basic learner, continuously updating and optimizing meta-migration learning parameters and balance parameters, wherein a reasoning network of the balance parameters is shown in FIG. 4, and the class where the samples with low identification accuracy are located is obtained in the step and is used as a difficult class;
the collection procedure for the difficult classes is:
step 5.1: acquiring a support set and a query set of a current input task;
step 5.2: calculation branchHolding the loss function of the concentrated sample and updating the parameters [ theta ] of the basic learner; theta]Meta migration learning parameters
Figure BDA0003488567620000043
Balance parameter omegani,γi,zi；
(Θ,θ0)＝g(zi)(Θ,θ)
Where the subscript i denotes the index of the task, i 1, 2.., N; g (·) is a non-negative activation function, and the above equation is an initialization equation of the model by balancing the variable zτTo determine how much meta-knowledge to use;
Figure BDA0003488567620000044
wherein mu is the learning rate of the meta migration learning parameter; l isT(query)A loss function that is a sample of the query set;
Figure BDA0003488567620000051
wherein f (·) max (0, min (·,1)) is a clipping function; alpha is the task learning rate; series of scalars
Figure BDA0003488567620000052
Processing the class imbalance as a coefficient of the per-task gradient descent step loss;
step 5.3: extracting samples from the query set, calculating the classification accuracy of the samples, arranging the samples in ascending order according to the accuracy, taking the class where the m lowest samples are as the difficult class, wherein m is a preset constant; the specific algorithm of the task self-adaptive small sample image classification method based on the element migration learning is shown in FIG. 5;
step 6: randomly sampling from the difficult class set to form a difficult task;
and 7: randomly extracting difficult tasks to train a basic learner, and continuously updating optimized learner parameters, meta migration learning parameters and balance parameters until the loss is lower than a threshold value;
and 8: judging whether the maximum iteration times is reached; if the maximum iteration times are not reached, clearing the difficult class set, and returning to the step 5 for next iteration; if the maximum iteration times are reached, outputting the element migration learning parameters and the balance parameters after the iteration is finished to obtain a trained basic learner;
and step 9: and inputting the meta-test set into a trained basic learner to obtain a small sample image classification result.
The invention provides a task self-adaptive small sample learning algorithm (MT-TAML) based on element migration learning on an MAML basic model, and the problem of insufficient characteristics of the MAML model by adopting a 4Conv shallow network is solved by combining element migration learning; trainable parameters are added to learn the use of balance element knowledge in each task, and the problems of unbalanced task, unbalanced category and unbalanced distribution of small sample learning in a real scene are solved.
The difficult task mining algorithm provided by the invention collects samples with poor classification effect on line to form difficult tasks, so that a learner can learn the difficult tasks more quickly and with better effect. Traditional meta-batch processing consists of randomly sampled tasks, where randomness implies random difficulties. In the algorithm, samples with low accuracy in each task are intentionally selected, and data of the samples are recombined to form more difficult tasks, so that the accuracy of the model is improved in the process of learning the more difficult tasks by the meta-learner.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (3)

1. A task self-adaptive small sample image classification method based on meta-migration learning is characterized by comprising the following steps:
step 1: acquiring a large-scale image data set, pre-training a deep network by using samples in the large-scale image data set, and outputting weight parameter vectors theta and theta of a group of feature extractors and classifiers;
step 2: freezing the weight parameter vector theta of the feature extractor, and resetting the weight parameter vector theta of the classifier;
and step 3: acquiring a small sample image data set to be classified, and dividing the small sample image data set into a meta-training set and a meta-testing set; the meta-training set comprises a support set and a query set, wherein the support set is used for updating parameters of the basic learner, and the query set is used for calculating a loss function and updating the meta-learner;
and 4, step 4: initializing parameters of a meta learner and a basic learner, and setting the maximum iteration times; the basic learner is used for rapidly learning new knowledge from a new task, and the meta learner is used for summarizing experiences in all learned knowledge; the small sample learning takes tasks as a unit, and each task comprises N classes;
and 5: randomly extracting samples from the meta-training set to form a training task, then randomly extracting the training task to train a basic learner, continuously updating and optimizing meta-migration learning parameters and balance parameters, and obtaining a class in which the samples with low identification accuracy are located as a difficult class in the step;
step 6: randomly sampling from the difficult class set to form a difficult task;
and 7: randomly extracting difficult tasks to train a basic learner, and continuously updating optimized learner parameters, meta migration learning parameters and balance parameters until the loss is lower than a threshold value;
and 8: judging whether the maximum iteration times is reached; if the maximum iteration times are not reached, clearing the difficult class set, and returning to the step 5 for next iteration; if the maximum iteration times are reached, outputting the element migration learning parameters and the balance parameters after the iteration is finished to obtain a trained basic learner;
and step 9: and inputting the meta-test set into a trained basic learning device to obtain a small sample image classification result.
2. The task-adaptive small sample image classification method based on meta-migration learning according to claim 1, characterized in that: during the pre-training in step 1, the loss function is:
Figure FDA0003488567610000011
wherein D is sample data; f. of[Θ；θ]Representing a model equation containing weight parameter vectors theta and theta of the feature extractor and the classifier; x, y denote the specimen and label, respectively;
the updating formula of the weight parameter vectors theta and theta of the feature extractor and the classifier is as follows:
Figure FDA0003488567610000012
3. the task-adaptive small sample image classification method based on meta-migration learning according to claim 1, characterized in that: the collection process of the difficulty classes in the step 5 is as follows:
step 5.1: acquiring a support set and a query set of a current input task;
step 5.2: calculating a loss function supporting the centralized sample and updating a basic learner parameter [ theta ]; theta]Meta migration learning parameters
Figure FDA0003488567610000021
Balance parameter omegani,γi,zi；
(Θ,θ0)＝g(zi)(Θ,θ)
Wherein, subscript i represents an index of the task, i ═ 1, 2., N; g (-) is a nonnegative activation function, the above equation is the initialization equation of the model, by balancing the variable zτTo determine how much meta-knowledge to use;
Figure FDA0003488567610000022
wherein mu is the learning rate of the meta migration learning parameter; l isT(query)A loss function that is a sample of the query set;
Figure FDA0003488567610000023
wherein f (·) max (0, min (·,1)) is a clipping function; alpha is the task learning rate; series of scalars
Figure FDA0003488567610000024
Processing the class imbalance as a coefficient of the per-task gradient descent step loss;
step 5.3: and extracting samples from the query set, calculating the classification accuracy of the samples, arranging the samples in ascending order according to the accuracy, and taking the class where the m lowest samples are located as a difficult class, wherein m is a preset constant.
CN202210089412.1A 2022-01-25 2022-01-25 Task-adaptive small sample image classification method based on meta-migration learning Pending CN114511739A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210089412.1A CN114511739A (en) 2022-01-25 2022-01-25 Task-adaptive small sample image classification method based on meta-migration learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210089412.1A CN114511739A (en) 2022-01-25 2022-01-25 Task-adaptive small sample image classification method based on meta-migration learning

Publications (1)

Publication Number Publication Date
CN114511739A true CN114511739A (en) 2022-05-17

Family

ID=81550188

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210089412.1A Pending CN114511739A (en) 2022-01-25 2022-01-25 Task-adaptive small sample image classification method based on meta-migration learning

Country Status (1)

Country Link
CN (1) CN114511739A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115409124A (en) * 2022-09-19 2022-11-29 小语智能信息科技(云南)有限公司 Small sample sensitive information identification method based on fine-tuning prototype network
CN116109627A (en) * 2023-04-10 2023-05-12 广东省科技基础条件平台中心 Defect detection method, device and medium based on migration learning and small sample learning
WO2024082374A1 (en) * 2022-10-19 2024-04-25 电子科技大学长三角研究院(衢州) Few-shot radar target recognition method based on hierarchical meta transfer

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115409124A (en) * 2022-09-19 2022-11-29 小语智能信息科技(云南)有限公司 Small sample sensitive information identification method based on fine-tuning prototype network
WO2024082374A1 (en) * 2022-10-19 2024-04-25 电子科技大学长三角研究院(衢州) Few-shot radar target recognition method based on hierarchical meta transfer
CN116109627A (en) * 2023-04-10 2023-05-12 广东省科技基础条件平台中心 Defect detection method, device and medium based on migration learning and small sample learning
CN116109627B (en) * 2023-04-10 2023-08-01 广东省科技基础条件平台中心 Defect detection method, device and medium based on migration learning and small sample learning

Similar Documents

Publication Publication Date Title
CN114511739A (en) Task-adaptive small sample image classification method based on meta-migration learning
CN109948561B (en) The method and system that unsupervised image/video pedestrian based on migration network identifies again
WO2022037233A1 (en) Small sample visual target identification method based on self-supervised knowledge transfer
CN111639679B (en) Small sample learning method based on multi-scale metric learning
CN111985581B (en) Sample-level attention network-based few-sample learning method
CN110852168A (en) Pedestrian re-recognition model construction method and device based on neural framework search
CN109299342A (en) A kind of cross-module state search method based on circulation production confrontation network
CN107330355B (en) Deep pedestrian re-identification method based on positive sample balance constraint
CN108921107A (en) Pedestrian's recognition methods again based on sequence loss and Siamese network
CN111292195A (en) Risk account identification method and device
CN114332568A (en) Training method, system, equipment and storage medium of domain adaptive image classification network
CN103886030B (en) Cost-sensitive decision-making tree based physical information fusion system data classification method
CN110532564A (en) A kind of application layer protocol online recognition method based on CNN and LSTM mixed model
CN110633667B (en) Action prediction method based on multitask random forest
CN110287985A (en) A kind of deep neural network image-recognizing method based on the primary topology with Mutation Particle Swarm Optimizer
CN115130538A (en) Training method of text classification model, text processing method, equipment and medium
CN104598898B (en) A kind of Aerial Images system for rapidly identifying and its method for quickly identifying based on multitask topology learning
CN112215278A (en) Multi-dimensional data feature selection method combining genetic algorithm and dragonfly algorithm
CN114897085A (en) Clustering method based on closed subgraph link prediction and computer equipment
CN110414431A (en) Face identification method and system based on elastic context relation loss function
CN110795410A (en) Multi-field text classification method
CN114743133A (en) Lightweight small sample video classification and identification method and system
CN111737688B (en) Attack defense system based on user portrait
CN113392868A (en) Model training method, related device, equipment and storage medium
CN107909090A (en) Learn semi-supervised music-book on pianoforte difficulty recognition methods based on estimating

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination