CN113688944B - Image identification method based on meta-learning - Google Patents

Image identification method based on meta-learning Download PDF

Info

Publication number
CN113688944B
CN113688944B CN202111149647.7A CN202111149647A CN113688944B CN 113688944 B CN113688944 B CN 113688944B CN 202111149647 A CN202111149647 A CN 202111149647A CN 113688944 B CN113688944 B CN 113688944B
Authority
CN
China
Prior art keywords
meta
training
training set
sub
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111149647.7A
Other languages
Chinese (zh)
Other versions
CN113688944A (en
Inventor
张鸿杰
盛谦
蒋斌
郭延文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing L Zone Intelligent Technology Co ltd
Original Assignee
Nanjing L Zone Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing L Zone Intelligent Technology Co ltd filed Critical Nanjing L Zone Intelligent Technology Co ltd
Priority to CN202111149647.7A priority Critical patent/CN113688944B/en
Publication of CN113688944A publication Critical patent/CN113688944A/en
Application granted granted Critical
Publication of CN113688944B publication Critical patent/CN113688944B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The invention provides an image identification method based on meta-learning, which comprises the following steps: step 1, enhancing training data: generating a virtual open set sample for the images in each training set based on the generated confrontation network; step 2, dividing training data: combining the generated open set sample, dividing the training data into a meta training set and a meta testing set according to different domain distributions; step 3, meta-training: calculating the gradient of the model parameters on the meta-training set and modifying the parameters; step 4, meta-testing: evaluating the best model learned in step 3 on the meta-test set; step 5, meta-optimization: combining the loss function optimization models of the step 3 and the step 4; step 6, element iteration: and (5) repeating the steps 2 to 5 to carry out iterative optimization until an image recognition model with good generalization capability is obtained. Compared with the prior art, the method and the device can improve the generalization capability of the model to the domain distribution change on the training set with a plurality of different domain distributions.

Description

Image identification method based on meta-learning
Technical Field
The invention relates to the technical field of computer vision, in particular to an image identification method based on meta-learning.
Background
At present, most image recognition methods are proposed based on the assumption that the closed set and the domain distribution are the same. Wherein the closed set assumption means that the class distribution of the training set completely covers all classes appearing in the test set. However, this assumption does not hold true in real application scenarios. The recognition system is highly likely to face risks from outside the distribution of the training classes, and the closed set assumption forces the recognition system to effectively recognize these samples from unknown classes, which may cause security problems in practical applications.
A more common case is open set assumptions, i.e., the test set contains classes that have not appeared in the training set. This requires that the recognition system be able to accurately recognize both known classes that have appeared during the training process and to respond correctly to unknown classes that have not appeared. The assumption that the domain distribution is the same means that the domain distribution of the training set and the domain distribution of the test set are the same, that is, the image styles of the training set and the test set are the same, which also does not hold true in real application scenarios. The recognition system faces a variety of image styles, and the training process cannot exhaust all possible image styles. However, such a domain distribution difference will cause a serious decrease in the accuracy of the recognition system, forcing the recognition system to have a stronger generalization capability to cope with the influence of the domain distribution difference.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problem of open set and domain distribution difference of image recognition in a real application scene, an image recognition method based on meta-learning is provided. The invention trains an image recognition model on a plurality of training set data with class labels, and the domain distribution of each training set is different, so as to simulate the domain distribution change of a test set and the training set in the real world. In the image identification method based on meta-learning, a virtual open set sample is generated for each training set data by generating a confrontation network, and overfitting to any category distribution and domain distribution is reduced by a meta-learning optimization strategy based on the open set samples and original training data, so that the generalization capability of a model is improved.
The technical scheme is as follows: the invention discloses an image identification method based on meta-learning, which is characterized in that the generalization capability of a model to category distribution change and domain distribution change is improved through a meta-learning optimization strategy, and comprises the following steps:
step 1, generating a virtual open set sample for images in each training set based on a generated confrontation network, wherein the training set comprises a plurality of sub-training sets with class labels, the class labels contained in each sub-training set are completely overlapped, but the domain distribution of each sub-training set is different, namely the image style in each sub-training set is different; through the steps, the training data can be enhanced;
step 2, combining the virtual open set samples, dividing the training set into a meta-training set and a meta-testing set according to different domain distributions, wherein the classes in the meta-training set completely cover all the classes in the meta-testing set, namely the meta-training set comprises known classes with class labels and unknown classes formed by the virtual open set samples; through the steps, the division of training data is realized;
step 3, calculating the gradient of model parameters on the meta-training set and modifying the parameters to obtain the current optimal model; through the steps, meta-training is realized;
step 4, evaluating the current optimal model obtained in the step 3 on the meta test set; through the steps, the meta test is realized;
step 5, optimizing the current optimal model by combining all the loss functions in the step 3 and the step 4; through the steps, meta-optimization is realized;
and 6, repeating the steps 2 to 5 to carry out iterative optimization until a final optimal model is obtained. Through the steps, the element iteration is realized;
further, in one implementation, the step 1 includes:
step 1-1, projecting the original images in the training set to a feature space z = E (x) by using an encoder;
step 1-2, reversely generating an image G (z) from the feature space by using a generator;
step 1-3, jointly training the encoder, generator and discriminator by cross-optimizing two loss functions according to the following formula:
Figure BDA0003286476450000021
Figure BDA0003286476450000022
wherein L is dis Representing the discrimination loss function, L recons Representing the loss function of image reconstruction, S i Representing the ith sub-training set of the plurality S of training sets,
Figure BDA0003286476450000023
representing the images in the ith sub-training set, D representing the discriminator, G representing the generator, and E representing the encoder; in the invention, through the steps 1-3, the generated reconstructed image can be as vivid as possible as an original image, and the identifier can not distinguish the authenticity of the original image;
step 1-4, according to the following formula, the ith sub-training set S i Training classifier
Figure BDA0003286476450000024
Figure BDA0003286476450000031
Where k represents the number of known classes in the training set, L c A function representing the loss of classification is represented,
Figure BDA0003286476450000032
represents the ith sub-training set S i Image of (1)
Figure BDA0003286476450000033
The class labels of (a), the class labels containing only class labels of k known classes of the training set;
step 1-5, using the classifier
Figure BDA0003286476450000034
Approximating decision boundaries of known class data and unknown class data, generating each trainletExercise and Collection S i Corresponding virtual open set sample
Figure BDA0003286476450000035
Generating the virtual open set sample of each
Figure BDA0003286476450000036
Are all integrated into each virtual open set sample
Figure BDA0003286476450000037
Corresponding sub-training set S i And then, an enhanced training set is obtained, expressed as S = { S = { [ S ] + ,S - In which S is + Representing known class data, S - Representing the generated virtual open set sample.
Further, in one implementation, the steps 1-5 include:
inputting the image G (z) to a classifier
Figure BDA0003286476450000038
When the classifier is used
Figure BDA0003286476450000039
When the output confidence degrees of all the categories are low, and the low confidence degree is low relative to the output confidence degree of zero, determining that the image belongs to the unknown data;
from the ith sub-training set S i In randomly selecting a seed image x i As input, the following formula is minimized by the gradient descent method:
Figure BDA00032864764500000310
wherein z is * Representing the feature space vector to be found, the first term | z-E (x) i ) II denotes the feature space vector z to be found * It is desirable to match the seed image x as much as possible i Similar style, second item
Figure BDA00032864764500000311
Representing a log-likelihood assuming that the output confidence of an unknown class is zero, and by minimizing the log-likelihood, the output confidence of all the known classes is pushed down;
finding z by a fixed number of steps * And to z * Decoding using generator G, i.e. for each sub-training set S i Generating a virtual open set sample, the generated virtual open set sample being represented as
Figure BDA00032864764500000312
Each of the sub-training sets S i All have a sub-training set S i Corresponding virtual open set sample
Figure BDA00032864764500000313
All the sub-training sets S i Corresponding virtual open set sample
Figure BDA00032864764500000314
I.e. as a meta-training set
Figure BDA00032864764500000315
Virtual open set sample in (1)
Figure BDA00032864764500000316
All the generated virtual open set samples
Figure BDA00032864764500000317
Are all integrated into respective corresponding sub-training sets S i Thereafter, an enhanced training set S is obtained, which is derived from the original known class data S + And the generated open set sample S - And (4) forming.
Specifically, in the present invention, the fixed number of steps is set according to different image tasks in an experiment.
Further, in one implementation, the step 2 includes:
dividing the enhanced training set S into N according to different domain distribution s Training set of individual units
Figure BDA00032864764500000318
And N t Test set of individuals
Figure BDA00032864764500000319
The meta test set
Figure BDA0003286476450000041
Number N of t Is set to 1; the meta training set
Figure BDA0003286476450000042
For training models, the set of meta-tests
Figure BDA0003286476450000043
For testing the trained model. In the present invention, the step 2 is to simulate the training and testing set in the real scene, i.e. simulate the domain distribution difference in the real scene.
Further, in one implementation, the step 3 includes:
step 3-1, using the meta training set generated in step 1
Figure BDA0003286476450000044
Virtual open set sample in (1)
Figure BDA0003286476450000045
Training an image recognition model F, said
Figure BDA0003286476450000046
Presentation element training set
Figure BDA0003286476450000047
The formula of the process is as follows:
Figure BDA0003286476450000048
wherein L is o Representing virtual open set examples
Figure BDA0003286476450000049
θ represents a parameter of the image recognition model F;
step 3-2, training the meta training set
Figure BDA00032864764500000410
Known class data of
Figure BDA00032864764500000411
Training is carried out, and the formula of the process is expressed as follows:
L c ＝-log((F(θ；x)) y )；
wherein L is c Application of representation to meta-training set
Figure BDA00032864764500000412
Known class data in
Figure BDA00032864764500000413
Cross entropy loss of (d);
step 3-3, the overall loss function is expressed as:
Figure BDA00032864764500000414
step 3-4, calculating all the meta training sets according to the following formula
Figure BDA00032864764500000415
Loss of
Figure BDA00032864764500000420
Figure BDA00032864764500000416
Wherein, N s Presentation element training set
Figure BDA00032864764500000417
The number of (2);
step 3-5, calculating the gradient of the parameter theta according to the following formula:
Figure BDA00032864764500000418
and 3-6, modifying the parameters of the model F according to the following formula:
Figure BDA00032864764500000419
wherein α is the step size of the meta training;
and obtaining the model after the whole meta-training parameter is updated, namely the current optimal model.
Further, in an implementation manner, the step 4 includes:
step 4-1, according to the following formula, in the meta-test set
Figure BDA0003286476450000051
The adaptive parameter loss L (theta') is calculated:
Figure BDA0003286476450000052
wherein the meta test set
Figure BDA0003286476450000053
The loss function L (theta') of (C) uses a training set of secondary elements
Figure BDA0003286476450000054
Calculating the parameter theta' of middle learning;
step 4-2, calculating all the meta-test sets according to the following formula
Figure BDA0003286476450000055
Loss of (c):
Figure BDA0003286476450000056
further, in one implementation, the step 5 includes:
in meta-optimization, meta-training and meta-testing are simultaneously optimized by gradients computed from a joint loss function, the process of the optimization being expressed as the following formula:
Figure BDA0003286476450000057
where β is a trade-off parameter between meta-training and meta-testing.
Further, in one implementation, the step 6 includes:
and repeating the step 2 to the step 5, and obtaining a final optimal model, namely the image recognition model F with good generalization capability when the model optimized in the step 5 is optimized to be converged on the training set S.
The method uses the generated countermeasure network to generate open set samples for training set data distributed in a plurality of different domains, and improves the generalization capability of the model to domain distribution change and category distribution change in image recognition through a meta-learning optimization strategy based on the generated open set samples.
Compared with the prior art, the invention at least comprises the following beneficial effects:
(1) And the model acquires knowledge for identifying unknown classes by using the generated open set sample, and the open set identification knowledge is transferred to an image identification environment with unknown domain distribution through meta-learning.
(2) And the generalization capability of the model for coping with the domain distribution change is improved on a training set distributed in a plurality of different domains based on the meta-learning method.
(3) All models (including an encoder, a generator, a discriminator and an image recognition model) in the invention are not limited to specific structures, and suitable models can be selected according to specific requirements of image recognition tasks.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings required to be used in the embodiments will be briefly described below, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic workflow diagram of an image recognition method based on meta-learning according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of approximate image generation in an image recognition method based on meta learning according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating the generation of open sample in an image recognition method based on meta-learning according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an open set sample in an image recognition method based on meta learning according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, the present invention is described in detail with reference to the accompanying drawings and the detailed description thereof. In different embodiments, the deep learning model is selected, and different models can be selected according to the situation of a specific image recognition task. All other embodiments obtained by a person skilled in the art without making any inventive step are within the scope of the present invention.
The embodiment of the invention discloses an image identification method based on meta learning, which is applied to an image identification scene with unknown images and image domain distribution, namely, different image styles.
The process of the present invention is shown in FIG. 1. First, a plurality of open set examples approximating the domain distribution of the training set, such as "minus" in fig. 1, are generated on a plurality of training set data by generating a confrontation network, and used as an unknown class to learn knowledge for identifying the unknown class. And then, dividing the enhanced training set data into a plurality of virtual training sets and test sets according to different domain distributions, and using the virtual training sets and the test sets to simulate the domain distribution change in a real application scene. And finally, based on a meta-learning optimization strategy, learning a more robust feature space in an iterative optimization mode, and reducing overfitting of the model to any specific training set domain distribution and category distribution, thereby improving the generalization capability of the model in image recognition.
Specifically, as shown in fig. 1, the present embodiment provides an image recognition method based on meta-learning, which is characterized in that the generalization capability of a model to class distribution change and domain distribution change is enhanced by a meta-learning optimization strategy, and includes the following steps:
step 1, generating a virtual open set sample for images in each training set based on a generated confrontation network, wherein the training set comprises a plurality of sub-training sets with class labels, the class labels contained in each sub-training set are completely overlapped, but the domain distribution of each sub-training set is different, namely the image style in each sub-training set is different; through the steps, the training data can be enhanced;
step 2, combining the virtual open set samples, dividing the training set into a meta training set and a meta test set according to different domain distributions, wherein the classes in the meta training set completely cover all the classes in the meta test set, namely the meta training set comprises known classes with class labels and unknown classes formed by the virtual open set samples; through the steps, the division of training data is realized;
step 3, calculating the gradient of model parameters on the meta-training set and modifying the parameters to obtain the current optimal model; through the steps, meta-training is realized;
step 4, evaluating the current optimal model obtained in the step 3 on the meta-test set; through the steps, the meta test is realized;
step 5, optimizing the current optimal model by combining all the loss functions in the step 3 and the step 4; through the steps, meta-optimization is realized;
and 6, repeating the steps 2 to 5 for iterative optimization until a final optimal model is obtained. Through the steps, the element iteration is realized;
in the image recognition method based on meta learning according to this embodiment, the step 1 includes:
step 1-1, projecting the original images in the training set to a feature space z = E (x) by using an encoder;
step 1-2, reversely generating an image G (z) from the feature space by using a generator;
step 1-3, jointly training the encoder, generator and discriminator by cross-optimizing two loss functions according to the following formula:
Figure BDA0003286476450000071
Figure BDA0003286476450000081
wherein L is dis Represents the discrimination loss function, L recons Representing the loss function of image reconstruction, S i Representing the ith sub-training set of the plurality S of training sets,
Figure BDA0003286476450000082
representing the images in the ith sub-training set, D representing the discriminator, G representing the generator, and E representing the encoder; in the invention, through the steps 1-3, the generated reconstructed image can be as vivid as possible as an original image, and a discriminator cannot distinguish the true image from the false image; specifically, in this embodiment, a flowchart of the process is shown in fig. 2;
step 1-4, according to the following formula, the ith sub-training set S i Training classifier
Figure BDA0003286476450000083
Figure BDA0003286476450000084
Where k represents the number of known classes in the training set, L c A function representing the loss of classification is represented,
Figure BDA0003286476450000085
represents the ith sub-training set S i Image of (1)
Figure BDA0003286476450000086
The class labels of (1), the class labels only containing class labels of k known classes of the training set S;
step 1-5, using the classifier
Figure BDA0003286476450000087
Approximating decision boundaries of the known class data and the unknown class data, generating each sub-training set S i Corresponding virtual open set sample
Figure BDA0003286476450000088
Generating the virtual open set sample of each
Figure BDA0003286476450000089
Are all integrated into each virtual open set sample
Figure BDA00032864764500000810
Corresponding sub-training set S i After that, an enhanced training set is obtained, denoted as S = { S = } + ,S - In which S is + Representing known class data, S - Representing the generated virtual open set sample.
In the image recognition method based on meta learning according to this embodiment, the steps 1 to 5 include:
inputting the image G (z) to a classifier
Figure BDA00032864764500000811
When the classifier is used
Figure BDA00032864764500000812
When the output confidence coefficients of all the categories are lower, and the lower confidence coefficient is lower relative to the output confidence coefficient of zero, determining that the image belongs to the unknown data;
from the ith sub-training set S i In randomly selecting a seed image x i As input, the following formula is minimized by the gradient descent method:
Figure BDA00032864764500000813
wherein z is * Representing the feature space vector to be found, the first term | z-E (x) i ) II denotes the feature space vector z to be found * It is desirable to match the seed image x as much as possible i Similar style, second item
Figure BDA00032864764500000814
Representing a log-likelihood assuming that the output confidence of an unknown class is zero, and by minimizing the log-likelihood, the output confidence of all the known classes is pushed down;
finding z by a fixed number of steps * And to z * Decoding using generator G, i.e. for each sub-training set S i Generating a virtual open set sample, the generated virtual open set sample being represented as
Figure BDA00032864764500000815
Each of the sub-training sets S i All have a sub-training set S i Corresponding virtual open set sample
Figure BDA0003286476450000091
All the sub-training sets S i Corresponding virtual open set sample
Figure BDA0003286476450000092
I.e. as a meta training set
Figure BDA0003286476450000093
Virtual open set sample in (1)
Figure BDA0003286476450000094
All the generated virtual open set samples
Figure BDA0003286476450000095
Are all integrated into the respective corresponding sub-training sets S i Thereafter, an enhanced training set S is obtained, which is derived from the original known class data S + And the generated open set sample S - And (4) forming. Specifically, in the present invention, the fixed number of steps is set according to different image tasks in an experiment. A flow chart of this process is shown in fig. 3. Fig. 4 shows a schematic diagram of some open set generation examples on a Digits-DG image dataset, which is a classical domain generalization task experiment dataset, including a MNIST image dataset, a MNIST-M image dataset, a SVHN image dataset, and a USPS image dataset.
In the image recognition method based on meta learning according to this embodiment, the step 2 includes:
dividing the enhanced training set S into N according to different domain distribution s Training set of individual units
Figure BDA0003286476450000096
And N t Test set of individuals
Figure BDA0003286476450000097
The meta test set
Figure BDA0003286476450000098
Number N of t Is set to 1; the meta training set
Figure BDA0003286476450000099
For training models, the set of meta-tests
Figure BDA00032864764500000910
For testing the trained model. Specifically, in the present embodiment, N s Is set to be 3,N t Is set to 1. The step 2 is to simulate a training and testing set in a real scene, that is, to simulate a domain distribution difference in the real scene.
In the image recognition method based on meta learning according to this embodiment, the step 3 includes:
step 3-1, using the meta training set generated in step 1
Figure BDA00032864764500000911
Virtual open set sample in (1)
Figure BDA00032864764500000912
Training an image recognition model F, the formula of the process is as follows:
Figure BDA00032864764500000913
wherein L is o Representing virtual open set instances
Figure BDA00032864764500000914
θ represents a parameter of the image recognition model F;
step 3-2, training the meta training set
Figure BDA00032864764500000915
Known class data of
Figure BDA00032864764500000916
Training is carried out, and the formula of the process is expressed as follows:
L c ＝-log((F(θ；x)) y )；
wherein L is c Representation application to Meta-training set
Figure BDA00032864764500000917
Known class data of
Figure BDA00032864764500000918
Cross entropy loss of (d);
step 3-3, the overall loss function is expressed as:
Figure BDA00032864764500000919
step 3-4, calculating all the meta training sets according to the following formula
Figure BDA00032864764500000920
Loss of
Figure BDA00032864764500000921
Figure BDA0003286476450000101
Wherein N is s Presentation element training set
Figure BDA0003286476450000102
The number of (2);
step 3-5, calculating the gradient of the parameter theta according to the following formula:
Figure BDA0003286476450000103
and 3-6, modifying the parameters of the model F according to the following formula:
Figure BDA0003286476450000104
wherein α is the step size of the meta training;
and obtaining the model after the whole meta-training parameter is updated, namely the current optimal model.
In the image recognition method based on meta learning according to this embodiment, the step 4 includes:
step 4-1, according to the following formula, in the meta-test set
Figure BDA0003286476450000105
Upper-calculated adaptive parameter loss L (θ'):
Figure BDA0003286476450000106
wherein the meta test set
Figure BDA0003286476450000107
The loss function L (theta') of (C) uses a training set of secondary elements
Figure BDA0003286476450000108
Calculating the parameter theta' of middle learning;
step 4-2, calculating all the meta-test sets according to the following formula
Figure BDA0003286476450000109
Loss of (2):
Figure BDA00032864764500001010
in the image recognition method based on meta learning according to this embodiment, the step 5 includes:
in meta-optimization, meta-training and meta-testing are simultaneously optimized by gradients computed from a joint loss function, the process of the optimization being expressed as the following formula:
Figure BDA00032864764500001011
where β is a trade-off parameter between meta-training and meta-testing.
In the image recognition method based on meta learning according to this embodiment, the step 6 includes:
repeating the steps 2 to 5, and obtaining a final optimal model when the model optimized in the step 5 is optimized to be converged on the training set S, namely the image recognition model F with good generalization capability * 。
The method uses the generated countermeasure network to generate open set samples for training set data distributed in a plurality of different domains, and improves the generalization capability of the model to domain distribution change and category distribution change in image recognition through a meta-learning optimization strategy based on the generated open set samples.
Compared with the prior art, the invention at least comprises the following beneficial effects:
(1) And the model acquires knowledge for identifying unknown classes by using the generated open set sample, and the open set identification knowledge is transferred to an image identification environment with unknown domain distribution through meta-learning.
(2) And the generalization capability of the model for coping with the domain distribution change is improved on a training set distributed in a plurality of different domains based on the meta-learning method.
(3) All models (including an encoder, a generator, a discriminator and an image recognition model) in the invention are not limited to specific structures, and suitable models can be selected according to specific requirements of image recognition tasks.
In specific implementation, the present invention further provides a computer storage medium, where the computer storage medium may store a program, and the program may include some or all of the steps in the embodiments of the image recognition method based on meta learning provided by the present invention when executed. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), or the like.
Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented using software plus any required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
The same and similar parts among the various embodiments in this specification may be referred to each other. The above-described embodiments of the present invention do not limit the scope of the present invention.

Claims (7)

1. An image recognition method based on meta-learning is characterized by comprising the following steps:
step 1, generating a virtual open set sample for images in each training set based on a generated confrontation network, wherein the training set comprises a plurality of sub-training sets with class labels, the class labels contained in each sub-training set are completely overlapped, but the domain distribution of each sub-training set is different, namely the image style in each sub-training set is different;
the step 1 comprises the following steps:
step 1-1, projecting the original images in the training set to a feature space z = E (x) by using an encoder;
step 1-2, reversely generating an image G (z) from the feature space by using a generator;
step 1-3, jointly training the encoder, generator and discriminator by cross-optimizing two loss functions according to the following formula:
Figure FDA0003905623420000011
Figure FDA0003905623420000012
wherein L is dis Representing the discrimination loss function, L recons Representing the loss function of image reconstruction, S i Representing the ith sub-training set of the plurality S of training sets,
Figure FDA0003905623420000013
representing the images in the ith sub-training set, D representing the discriminator, G representing the generator, and E representing the encoder;
step 1-4, according to the following formula, the ith sub-training set S i Training classifier
Figure FDA0003905623420000014
Figure FDA0003905623420000015
Where k represents the number of known classes in the training set S, L c A function representing the loss of the classification is expressed,
Figure FDA0003905623420000016
represents the ith sub-training set S i Image of (1)
Figure FDA0003905623420000017
The class labels of (1), the class labels only containing class labels of k known classes of the training set S;
step 1-5, using the classifier
Figure FDA0003905623420000018
Approximating the decision boundary of the known class data and the unknown class data to generate each sub-training set S i Corresponding virtual open set sample
Figure FDA0003905623420000019
Generating the virtual open set sample of each
Figure FDA0003905623420000021
Are all integrated into each virtual open set sample
Figure FDA0003905623420000022
Corresponding sub-training set S i After that, an enhanced training set is obtained, denoted as S = { S = } + ,S - In which S is + Representing known class data, S - Representing the generated virtual open set sample;
step 2, combining the virtual open set samples, dividing the training set into a meta-training set and a meta-testing set according to different domain distributions, wherein the classes in the meta-training set completely cover all the classes in the meta-testing set, namely the meta-training set comprises known classes with class labels and unknown classes formed by the virtual open set samples;
step 3, calculating the gradient of model parameters on the meta-training set and modifying the parameters to obtain a current optimal model;
step 4, evaluating the current optimal model obtained in the step 3 on the meta test set;
step 5, optimizing the current optimal model by combining all the loss functions in the step 3 and the step 4;
and 6, repeating the steps 2 to 5 for iterative optimization until a final optimal model is obtained.
2. The method for image recognition based on meta-learning as claimed in claim 1, wherein the steps 1-5 comprise:
inputting the image G (z) to a classifier
Figure FDA0003905623420000023
When the classifier is used
Figure FDA0003905623420000024
When the output confidence coefficients of all the categories are lower, and the lower confidence coefficient is lower relative to the output confidence coefficient of zero, determining that the image belongs to the unknown data;
from the ith sub-training set S i In randomly selecting a seed image x i As input, the following formula is minimized by the gradient descent method:
Figure FDA0003905623420000025
wherein z is * The first term | | z-E (x) represents the feature space vector to be searched i ) | | represents a feature space vector z to be found * It is desirable to match the seed image x as much as possible i Similar style, second item
Figure FDA0003905623420000026
Figure FDA0003905623420000027
Representing a log-likelihood assuming that the output confidence of the unknown class is zero, and by minimizing the log-likelihood, the output confidence of all the known classes is pushed down;
finding z by a fixed number of steps * And to z * Decoding using generator G, i.e. for each sub-training set S i Generating a virtual open set sample, the generated virtual open set sample being represented as
Figure FDA0003905623420000031
Each of the sub-training sets S i All have a sub-training set S i Corresponding virtual open set sample
Figure FDA0003905623420000032
All the sub-training sets S i Corresponding virtual open set sample
Figure FDA0003905623420000033
I.e. as a meta-training set
Figure FDA0003905623420000034
Virtual open set sample in (1)
Figure FDA0003905623420000035
All the generated virtual open set samples
Figure FDA0003905623420000036
Are all integrated into respective corresponding sub-training sets S i Thereafter, an enhanced training set S is obtained, which is derived from the original known class data S + And the generated open set sample S - And (4) forming.
3. The method for image recognition based on meta learning according to claim 1, wherein the step 2 comprises:
dividing the enhanced training set S into N according to different domain distribution s Training set of individual units
Figure FDA0003905623420000037
And N t Test set of individuals
Figure FDA0003905623420000038
The meta test set
Figure FDA0003905623420000039
Number N of t Is set to 1; the meta training set
Figure FDA00039056234200000310
For training models, the meta-test set
Figure FDA00039056234200000311
For testing the trained model.
4. The method for image recognition based on meta learning according to claim 3, wherein the step 3 comprises:
step 3-1, using the meta training set generated in the step 1
Figure FDA00039056234200000312
Virtual open set sample in (1)
Figure FDA00039056234200000313
Training an image recognition model F, wherein the formula of the step is expressed as follows:
Figure FDA00039056234200000314
wherein L is o Representing virtual open set examples
Figure FDA00039056234200000315
Theta represents a parameter of the image recognition model F, and k is the number of class labels of known classes;
step 3-2, training the meta training set
Figure FDA00039056234200000316
Known class data of
Figure FDA00039056234200000317
Training is performed, and the formula of the step is expressed as follows:
L c ＝-log((F(θ；x)) y )；
wherein L is c Representation application to Meta-training set
Figure FDA00039056234200000318
Known class data of
Figure FDA00039056234200000319
Cross entropy loss of (F (theta; x)) y ；
Step 3-3, the entire loss function is expressed as:
Figure FDA0003905623420000041
step 3-4, calculating all the meta training sets according to the following formula
Figure FDA0003905623420000042
Loss of (2)
Figure FDA0003905623420000043
Figure FDA0003905623420000044
Wherein, N s Presentation element training set
Figure FDA0003905623420000045
The number of (2);
step 3-5, calculating the gradient of the parameter theta according to the following formula:
Figure FDA0003905623420000046
and 3-6, modifying the parameters of the model F according to the following formula:
Figure FDA0003905623420000047
wherein α is the step size of the meta training;
and obtaining a model after the whole meta-training parameter is updated, namely the current optimal model.
5. The method for image recognition based on meta learning according to claim 4, wherein the step 4 comprises:
step 4-1, according to the following formula, in the meta-test set
Figure FDA0003905623420000048
Upper-calculated adaptive parameter loss L (θ'):
Figure FDA0003905623420000049
wherein the meta test set
Figure FDA00039056234200000410
The loss function L (theta') of (C) uses a training set of secondary elements
Figure FDA00039056234200000411
Calculating the parameter theta' of middle learning;
step 4-2, calculating all the meta-test sets according to the following formula
Figure FDA00039056234200000412
Loss of (2):
Figure FDA00039056234200000413
6. the method for image recognition based on meta-learning according to claim 5, wherein the step 5 comprises:
in meta-optimization, meta-training and meta-testing are simultaneously optimized by gradients computed from a joint loss function, the process of the optimization being expressed as the following formula:
Figure FDA0003905623420000051
where β is a trade-off parameter between meta-training and meta-testing.
7. The method according to claim 6, wherein the step 6 comprises:
repeating the step 2 to the step 5, and obtaining a final optimal model, namely an image recognition model F with good generalization capability when the model optimized in the step 5 is optimized to be converged on the training set S * 。
CN202111149647.7A 2021-09-29 2021-09-29 Image identification method based on meta-learning Active CN113688944B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111149647.7A CN113688944B (en) 2021-09-29 2021-09-29 Image identification method based on meta-learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111149647.7A CN113688944B (en) 2021-09-29 2021-09-29 Image identification method based on meta-learning

Publications (2)

Publication Number Publication Date
CN113688944A CN113688944A (en) 2021-11-23
CN113688944B true CN113688944B (en) 2022-12-27

Family

ID=78587332

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111149647.7A Active CN113688944B (en) 2021-09-29 2021-09-29 Image identification method based on meta-learning

Country Status (1)

Country Link
CN (1) CN113688944B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106779086A (en) * 2016-11-28 2017-05-31 北京大学 A kind of integrated learning approach and device based on Active Learning and model beta pruning
CN110569982A (en) * 2019-08-07 2019-12-13 南京智谷人工智能研究院有限公司 Active sampling method based on meta-learning
CN111429893A (en) * 2020-03-12 2020-07-17 南京邮电大学 Many-to-many speaker conversion method based on Transitive STARGAN
CN112101184A (en) * 2020-09-11 2020-12-18 电子科技大学 Wireless cross-domain action identification method based on semi-supervised learning
CN112381098A (en) * 2020-11-19 2021-02-19 上海交通大学 Semi-supervised learning method and system based on self-learning in target segmentation field
CN112766323A (en) * 2020-12-30 2021-05-07 清华大学 Image identification method and device
CN113256019A (en) * 2021-06-15 2021-08-13 浙江中海达空间信息技术有限公司 Geological disaster hidden danger sensitivity prediction method based on unsupervised characterization learning

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111612047B (en) * 2020-04-29 2023-06-02 杭州电子科技大学 Zero sample image recognition method based on attribute feature vector and reversible generation model
CN112364894B (en) * 2020-10-23 2022-07-08 天津大学 Zero sample image classification method of countermeasure network based on meta-learning

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106779086A (en) * 2016-11-28 2017-05-31 北京大学 A kind of integrated learning approach and device based on Active Learning and model beta pruning
CN110569982A (en) * 2019-08-07 2019-12-13 南京智谷人工智能研究院有限公司 Active sampling method based on meta-learning
CN111429893A (en) * 2020-03-12 2020-07-17 南京邮电大学 Many-to-many speaker conversion method based on Transitive STARGAN
CN112101184A (en) * 2020-09-11 2020-12-18 电子科技大学 Wireless cross-domain action identification method based on semi-supervised learning
CN112381098A (en) * 2020-11-19 2021-02-19 上海交通大学 Semi-supervised learning method and system based on self-learning in target segmentation field
CN112766323A (en) * 2020-12-30 2021-05-07 清华大学 Image identification method and device
CN113256019A (en) * 2021-06-15 2021-08-13 浙江中海达空间信息技术有限公司 Geological disaster hidden danger sensitivity prediction method based on unsupervised characterization learning

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Few-Shot Open-Set Recognition using Meta-Learning;Bo Liu等;《2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）》;20200805;第8795-8804页 *
Low-Shot Learning from Imaginary Data;Yu-Xiong Wang等;《2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition》;20181216;第7278-7286页 *
基于改进深度度量学习算法的表面缺陷检测;王伟等;《计算机与现代化》;20210630(第6期);第61-68页 *
小样本学习研究综述;赵凯琳等;《软件学报》;20210228;第32卷(第2期);第2.2节 *

Also Published As

Publication number Publication date
CN113688944A (en) 2021-11-23

Similar Documents

Publication Publication Date Title
CN109492662B (en) Zero sample image classification method based on confrontation self-encoder model
CN103136504B (en) Face identification method and device
CN108765383B (en) Video description method based on deep migration learning
CN108230291B (en) Object recognition system training method, object recognition method, device and electronic equipment
CN113674140B (en) Physical countermeasure sample generation method and system
CN112115967B (en) Image increment learning method based on data protection
CN112364791B (en) Pedestrian re-identification method and system based on generation of confrontation network
JP2005202932A (en) Method of classifying data into a plurality of classes
CN113642621A (en) Zero sample image classification method based on generation countermeasure network
CN111754637A (en) Large-scale three-dimensional face synthesis system with suppressed sample similarity
CN114821204B (en) Meta-learning-based embedded semi-supervised learning image classification method and system
CN111126155B (en) Pedestrian re-identification method for generating countermeasure network based on semantic constraint
CN112241741A (en) Self-adaptive image attribute editing model and method based on classified countermeasure network
CN115393666A (en) Small sample expansion method and system based on prototype completion in image classification
CN113688944B (en) Image identification method based on meta-learning
CN110197226B (en) Unsupervised image translation method and system
CN111753684A (en) Pedestrian re-recognition method for generating by using target posture
CN115578593B (en) Domain adaptation method using residual attention module
CN116486493A (en) Living body detection method, device and equipment
CN114913588A (en) Face image restoration and recognition method applied to complex scene
CN112329735B (en) Training method of face recognition model and online education system
CN114943912A (en) Video face changing method, device and storage medium
CN114037056A (en) Method and device for generating neural network, computer equipment and storage medium
CN117237748B (en) Picture identification method and device based on multi-view contrast confidence
Polyak et al. Unsupervised generation of free-form and parameterized avatars

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant