CN116342906A - Cross-domain small sample image recognition method and system - Google Patents
Cross-domain small sample image recognition method and system Download PDFInfo
- Publication number
- CN116342906A CN116342906A CN202310258589.4A CN202310258589A CN116342906A CN 116342906 A CN116342906 A CN 116342906A CN 202310258589 A CN202310258589 A CN 202310258589A CN 116342906 A CN116342906 A CN 116342906A
- Authority
- CN
- China
- Prior art keywords
- sample
- small sample
- network
- samples
- target domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000012549 training Methods 0.000 claims abstract description 45
- 238000009826 distribution Methods 0.000 claims abstract description 37
- 238000006243 chemical reaction Methods 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 12
- 230000000630 rising effect Effects 0.000 claims abstract description 10
- 238000000605 extraction Methods 0.000 claims description 19
- 238000005259 measurement Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010428 oil painting Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a cross-domain small sample image recognition method and a system, comprising the following steps: determining a trained small sample recognition network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, and the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and the variable self-encoder comprises intermediate parameters which are used for gradient rising to reconstruct the target domain sample into a sample with a more complex style; the style conversion module is used for stylizing the source domain samples to obtain labeled training samples belonging to the target domain data distribution; and inputting the target domain sample to be identified into a trained small sample identification network to conduct prediction classification, so as to obtain an image identification result. According to the method, the target domain information is effectively introduced in the training stage, and the generalization capability of the model to the target domain is improved.
Description
Technical Field
The invention belongs to the field of computer vision and image processing, and particularly relates to a cross-domain small sample image recognition method and system.
Background
Currently, deep learning-based approaches have achieved excellent performance on many computer vision tasks. However, these powerful deep learning models often rely on large-scale data sets and long training processes, requiring significant human resources, time costs, and expensive computation costs. In this case, the problem of small sample image recognition is one of the key research directions in the field of deep learning at present. Compared with the traditional image recognition driven by a large number of samples, the small sample image recognition can realize accurate prediction classification on new samples under the condition of limited sample size, and is more in line with the real application scene. However, in more practical applications, it is difficult to collect samples from the same domain to accomplish a large number of small sample classification tasks, and when the training and test data distributions are inconsistent, i.e. the source domain data and the target domain data are distributed differently, there is a domain offset between the two, the model is more difficult to generalize, and in order to distinguish from the problem of traditional small sample learning, the small sample learning problem in this scenario is defined as "cross-domain small sample learning".
The prior approach is based on model tuning. The method is based on the idea of meta-learning learning to learn, and aims to enable a model to learn knowledge irrelevant to tasks during training, so that the model is not fitted to training data, and can be generalized to new tasks more quickly. For example, the idea of Meta-learning is presented in the article "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks, proceedings of the International Conference on Machine Learning (ICML), 2017.
The existing cross-domain small sample identification based on fine tuning has obvious limitation, the operation of fine tuning the model for each new task input is very complex, and especially for a model with a large scale, the parameter fine tuning consumes a large amount of computation cost and time, so the technology is not an essential solution.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a cross-domain small sample image recognition method and a system, which aim to solve the complexity that the existing cross-domain small sample recognition technology based on fine tuning needs to perform parameter fine tuning operation once for each input new task; and the existing small sample identification method can not solve the problems of domain gaps and poor generalization performance.
To achieve the above object, in a first aspect, the present invention provides a cross-domain small sample image recognition method, including the steps of:
determining a trained small sample recognition network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, wherein the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and comprises an intermediate parameter used for reconstructing the target domain sample into a sample with more complex style by gradient rising; the style conversion module is used for stylizing the source domain sample to obtain a labeled training sample belonging to the target domain data distribution; wherein different image styles correspond to different sample categories or data fields;
and inputting the target domain sample to be identified into a trained small sample identification network to conduct prediction classification, so as to obtain an image identification result.
In one possible embodiment, the small sample identification network comprises: the feature extraction module and the measurement matching module;
the characteristic extraction module is used for extracting characteristics of an input sample;
the measurement matching module is used for carrying out matching classification on samples of different categories.
In one possible implementation, the training process of the small sample recognition network is as follows:
when training the small sample recognition network F, for each input task T, circularly and randomly selecting N types of images from M types of images, determining K samples as a support set S for each type of images, determining Q samples as a query set Q, T= (S, Q) for each type of images, and obtaining a generated sample after inputting the images into the image generation network G
Will generate samples->
Inputting into a small sample recognition network to perform prediction recognition calculation to obtain loss L T And updating parameters of the small sample recognition network F, and updating intermediate parameters of the image generation network G to generate more complex target domain style samples, and continuously training the small sample recognition network in a circulating way until the trained small sample recognition network meets the requirements.
In one possible implementation, the style conversion module performs stylization on the stylized parameter output by the variation self-encoder as a style feature, so as to preserve the content feature of the original sample and generate a new sample with the target domain data distribution feature.
In one possible implementation manner, the metric matching module is configured to perform matching classification on the query set and the support set of the current task example, and specifically includes: and calculating class centers for the N class support set samples, calculating distances from each sample of the query set to the N class centers, and classifying each sample of the query set to the class center closest to the sample of the query set to finish the identification and classification of the sample of the query set.
In one possible implementation, the overall objective function of the small sample recognition network is:
wherein L is ω Representing a stylized query set
Calculated classification loss, alpha denotes parameters of the small sample recognition network, A denotes the stylized support set based +.>
And a classifier selected corresponding to the small sample identification network parameter alpha, wherein omega is an output result of the small sample identification network.
In a second aspect, the present invention provides a cross-domain small sample image recognition system, comprising:
the identification network determining unit is used for determining a trained small sample identification network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, wherein the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and comprises an intermediate parameter used for reconstructing the target domain sample into a sample with more complex style by gradient rising; the style conversion module is used for stylizing the source domain sample to obtain a labeled training sample belonging to the target domain data distribution; wherein different image styles correspond to different sample categories or data fields;
the sample recognition unit is used for inputting the target domain sample to be recognized into the trained small sample recognition network so as to conduct prediction classification and obtain an image recognition result.
In one possible embodiment, the small sample identification network comprises: the feature extraction module and the measurement matching module; the characteristic extraction module is used for extracting characteristics of an input sample; the measurement matching module is used for carrying out matching classification on samples of different categories, and specifically comprises the following steps: the method for matching and classifying the query set and the support set of the current task example specifically comprises the following steps: and calculating class centers for the N class support set samples, calculating distances from each sample of the query set to the N class centers, and classifying each sample of the query set to the class center closest to the sample of the query set to finish the identification and classification of the sample of the query set.
In one possible embodiment, the system further comprises: the recognition network training unit is used for circularly and randomly selecting N types of images from M types of images for each input task T when training the small sample recognition network F, determining K samples as a support set S for each type of images, determining Q samples as a query set Q for each type of images, and obtaining a generated sample after inputting the small sample recognition network F into the image generation network G
Will generate samples->
Inputting into a small sample recognition network to perform prediction recognition calculation to obtain loss L T And updating parameters of the small sample recognition network F, and updating intermediate parameters of the image generation network G to generate more complex target domain style samples, and continuously training the small sample recognition network in a circulating way until the trained small sample recognition network meets the requirements.
In one possible implementation, the style conversion module in the image generation network performs style-based on the style parameters output from the encoder by the variation as style characteristics, so as to preserve the content characteristics of the original samples and generate new samples with the target domain data distribution characteristics.
In a third aspect, the present application provides an electronic device, comprising: at least one memory for storing a program; at least one processor for executing a memory-stored program, which when executed is adapted to carry out the method described in the first aspect or any one of the possible implementations of the first aspect.
In a fourth aspect, the present application provides a computer readable storage medium storing a computer program which, when run on a processor, causes the processor to perform the method described in the first aspect or any one of the possible implementations of the first aspect.
In a fifth aspect, the present application provides a computer program product which, when run on a processor, causes the processor to perform the method described in the first aspect or any one of the possible implementations of the first aspect.
In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:
the invention provides a cross-domain small sample image recognition method and a cross-domain small sample image recognition system. Because the feature extraction network fully learns the distribution information about the target domain in the training stage, even if a domain gap exists between the test sample and the training sample, the network can effectively extract the related features of the category in the sample in the feature extraction stage, thereby being beneficial to distinguishing samples of different categories.
The invention provides a cross-domain small sample image recognition method and a cross-domain small sample image recognition system, which are characterized in that a matching measurement module is selected to train a small sample recognition model, and the similarity of features among samples of different categories is matched in a flexible measurement mode, so that the corresponding relation among the samples can be effectively found, and further the matching recognition is effectively carried out.
The invention provides a cross-domain small sample image recognition method and a system, and provides a countermeasure-based generation network model, which comprises a variation self-encoder module and a style conversion module, wherein target domain information can be effectively introduced in a training stage, the generalization capability of the model to a target domain is improved, and a more accurate recognition effect of the small sample recognition model on the target domain is achieved.
Drawings
FIG. 1 is a flowchart of a cross-domain small sample image recognition method provided by an embodiment of the invention;
FIG. 2 is a schematic diagram of a cross-domain small sample identification architecture provided by an embodiment of the present invention;
FIG. 3 is a stylized generation schematic diagram provided by an embodiment of the present invention;
fig. 4 is a schematic diagram of a cross-domain small sample image recognition system according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The term "and/or" herein is an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. The symbol "/" herein indicates that the associated object is or is a relationship, e.g., A/B indicates A or B.
The terms "first" and "second" and the like in the description and in the claims are used for distinguishing between different objects and not for describing a particular sequential order of objects. For example, the first response message and the second response message, etc. are used to distinguish between different response messages, and are not used to describe a particular order of response messages.
In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.
In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" means two or more, for example, a plurality of processing units means two or more processing units and the like; the plurality of elements means two or more elements and the like.
First, technical terms involved in the embodiments of the present application will be described.
(1) Stylized
The image stylization can also be called style migration, namely, migrating the style of an image with artistic characteristics to a common image, so that the original image has unique artistic styles, such as cartoon, oil painting, watercolor, ink and the like, while retaining the original content.
(2) Domain
Different fields refer to different distributions of sample data, e.g. different artistic styles of images have different distributions of data, i.e. images of different artistic styles belong to different fields.
Next, the technical solutions provided in the embodiments of the present application are described.
FIG. 1 is a flowchart of a cross-domain small sample image recognition method provided by an embodiment of the invention; as shown in fig. 1, the method comprises the following steps:
s101, determining a trained small sample identification network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, wherein the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and comprises an intermediate parameter used for reconstructing the target domain sample into a sample with more complex style by gradient rising; the style conversion module is used for stylizing the source domain sample to obtain a labeled training sample belonging to the target domain data distribution; wherein different image styles correspond to different sample categories or data fields;
s102, inputting the target domain sample to be identified into a trained small sample identification network to conduct prediction classification, and obtaining an image identification result.
Specifically, first, a pre-trained image generation network G and a small sample recognition network F are determined; the image generation network G includes: the variation self-encoder module and the style conversion module; the variation self-encoder module is used for extracting and reconstructing characteristic distribution of the target domain samples, the middle parameter epsilon is used for gradient rising to generate more difficult style parameters, and the style conversion module is used for stylizing the source domain samples to obtain labeled training samples belonging to the target domain data distribution; the small sample identification network F includes: the feature extraction module and the measurement matching module; the feature extraction module is used for extracting features of the input samples, and the metric matching module is used for carrying out matching classification on samples of different categories.
As shown in fig. 2, the architecture of the cross-domain small sample identification method and system provided by the invention comprises two parts:
(1) Image generation pre-training network
Comprising a variable self-encoder module>
And a style conversion module { F, g }; the variation self-encoder module is used for extracting and reconstructing the characteristic distribution of the target domain samples, the intermediate parameter epsilon is used for gradient rising to generate more difficult style parameters, and the style conversion module is used for stylizing the source domain samples to obtain labeled training samples belonging to the target domain data distribution.
(2) Small sample identification network T: the system comprises a feature extraction module and a measurement matching module; the feature extraction module is used for extracting features of the input samples, and the metric matching module is used for carrying out matching classification on samples of different categories.
In a specific embodiment, the invention provides a cross-domain small sample identification method, which comprises the following steps:
in another example, the invention provides a cross-domain small sample identification system comprising:
image generation pre-training module G: comprises a variation self-encoder unit and a style conversion unit; the variation self-encoder unit is used for extracting and reconstructing the characteristic distribution of the target domain samples, the intermediate parameter epsilon is used for gradient rising to generate more difficult style parameters, and the style conversion unit is used for stylizing the source domain samples to obtain labeled training samples belonging to the target domain data distribution.
Small sample recognition module T: the device comprises a feature extraction unit and a metric matching unit; the feature extraction unit is used for extracting features of the input samples, and the metric matching unit is used for carrying out matching classification on samples of different categories.
When training the small sample recognition module F, for each input task T, cyclically and randomly selecting N classes of images from M classes of images, determining K samples as a support set S for each class of images, and determining Q samples as a query set Q, i.e., t= (S, Q), for each class of images, wherein
The input module G is followed by a generation sample->
Inputting the generated sample into the small sample recognition module for prediction recognition calculation to obtain loss L T And updating parameters of the identification module F, and simultaneously updating intermediate parameters of the generation module G to generate a more difficult target domain style, thereby completing training of the small sample identification module F.
Inputting the new sample of the target domain to be identified into a trained small sample identification module to predict and classify the sample to be identified and output a corresponding identification result.
In an alternative example, the variation is derived from an encoder unit for computing a data distribution for a set of unlabeled unknown domain samples and reconstructing corresponding stylized parameters, including in particular mean and variance, by sampling anchor epsilon from the distribution.
In an optional example, the style conversion unit is configured to stylize an example, select a current example as a content feature, stylize a stylized parameter output from the encoder as a style feature, and output a sample retaining an original content feature, so that a label thereof is retained, and meanwhile, the sample has a data distribution feature of a target domain.
In an optional example, the feature extraction unit is configured to perform feature extraction, and specifically includes performing feature extraction on a sample obtained after the current example is stylized.
In an optional example, the metric matching unit is configured to perform matching classification on a query set and a support set of a current task example, and specifically includes: and calculating class centers for the N class support set samples, and calculating distances from each sample of the query set to the N class centers to finish the identification and classification of the sample of the query set.
In an alternative example, the overall objective function of the identification module is:
wherein L is T Representing a stylized query set
The calculated classification loss, alpha represents the parameters of the small sample identification module, A represents the weight based on the stylized support set +.>
And the classifier selected by the corresponding small sample identification module parameter alpha, wherein omega is the corresponding output result.
FIG. 3 is a schematic diagram of stylized generation according to an embodiment of the present invention, and as shown in FIG. 3, first, we first generate a small number of target domain samples X T Calculate the mean value of the characteristics space
Sum of variances->
To mitigate the impact on network training caused by sampling contingency, wherein +.>
Then calculate the statistics N (ψ, ζ) of the Gaussian distribution, where +.>
Task t= { S for each small sample T ,Q T We first sample a vector epsilon from the gaussian distribution 1 Input D vae Decoding to obtain vector->
As style feature input for the following AdaIN network; will support the collection image S T And query set image Q T Input E VGG And taking the obtained feature as another input of AdaIN, adaIN outputs some new style images, denoted +.>
These style images approximately follow the target domain distribution. According to the description of the previous sections, "ε 1 "i.e." style anchor "because the VAE network can reconstruct the distribution of a given M target domain images. The generated stylized image is further input into a task model to solve the problem of classifying small samples. In the invention, the relation net is selected as a task model, and other models for solving the problem of small samples are also applicable. To search for more styles that match the target domain distribution, we use a challenge-based approach to generate more difficult stylized samples and attempt to iteratively mine more invisible target domain distributions starting from the "style anchor". As shown in FIG. 3, we calculate the classification loss L on the query set of the task model T1 We obtain the pair epsilon 1 Feedback of->
And prepare a more difficult sample for the next iteration +.>
Then by minimizing L T Updating task model parameters, hopefully more accuratelyThe target domain images are classified even though the model has excellent generalization ability.
Fig. 4 is a schematic diagram of a cross-domain small sample image recognition system according to an embodiment of the present invention, as shown in fig. 4, including:
the recognition network training unit 410 is configured to, when training the small sample recognition network F, for each input task T, circularly and randomly select N types of images from M types of images, determine K samples as a support set S for each type of images, determine Q samples as a query set Q, t= (S, Q) for each type of images, and obtain a generated sample after inputting the generated sample into the image generation network G
Will generate samples->
Inputting into a small sample recognition network to perform prediction recognition calculation to obtain loss L T And updating parameters of the small sample recognition network F, and updating intermediate parameters of the image generation network G to generate more complex target domain style samples, and continuously training the small sample recognition network in a circulating way until the trained small sample recognition network meets the requirements.
An identification network determining unit 420, configured to determine a trained small sample identification network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, wherein the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and comprises an intermediate parameter used for reconstructing the target domain sample into a sample with more complex style by gradient rising; the style conversion module is used for stylizing the source domain sample to obtain a labeled training sample belonging to the target domain data distribution; wherein different image styles correspond to different sample categories or data fields;
the sample recognition unit 430 is configured to input the target domain sample to be recognized into a trained small sample recognition network to perform prediction classification, so as to obtain an image recognition result.
It should be understood that, the foregoing apparatus is used to perform the method in the foregoing embodiment, and corresponding program modules in the apparatus implement principles and technical effects similar to those described in the foregoing method, and reference may be made to corresponding processes in the foregoing method for the working process of the apparatus, which are not repeated herein.
Based on the method in the above embodiment, an embodiment of the present application provides an electronic device. The apparatus may include: at least one memory for storing programs and at least one processor for executing the programs stored by the memory. Wherein the processor is adapted to perform the method described in the above embodiments when the program stored in the memory is executed.
Based on the method in the above embodiment, the present application provides a computer-readable storage medium storing a computer program, which when executed on a processor, causes the processor to perform the method in the above embodiment.
Based on the methods in the above embodiments, the present application provides a computer program product, which when run on a processor causes the processor to perform the methods in the above embodiments.
It is to be appreciated that the processor in embodiments of the present application may be a central processing unit (centralprocessing unit, CPU), but may also be other general purpose processors, digital signal processors (digital signalprocessor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field programmable gate arrays (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor.
The method steps in the embodiments of the present application may be implemented by hardware, or may be implemented by a processor executing software instructions. The software instructions may be comprised of corresponding software modules that may be stored in random access memory (random access memory, RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable PROM (EPROM), electrically erasable programmable EPROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted across a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
It will be appreciated that the various numerical numbers referred to in the embodiments of the present application are merely for ease of description and are not intended to limit the scope of the embodiments of the present application.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (10)
1. The cross-domain small sample image recognition method is characterized by comprising the following steps of:
determining a trained small sample recognition network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, wherein the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and comprises an intermediate parameter used for reconstructing the target domain sample into a sample with more complex style by gradient rising; the style conversion module is used for stylizing the source domain sample to obtain a labeled training sample belonging to the target domain data distribution; wherein different image styles correspond to different sample categories or data fields;
and inputting the target domain sample to be identified into a trained small sample identification network to conduct prediction classification, so as to obtain an image identification result.
2. The method of claim 1, wherein the small sample identification network comprises: the feature extraction module and the measurement matching module;
the characteristic extraction module is used for extracting characteristics of an input sample;
the measurement matching module is used for carrying out matching classification on samples of different categories.
3. The method of claim 1, wherein the training process of the small sample recognition network is as follows:
when training the small sample recognition network F, for each input task T, circularly and randomly selecting N types of images from M types of images, determining K samples as a support set S for each type of images, determining Q samples as a query set Q, T= (S, Q) for each type of images, and obtaining a generated sample after inputting the images into the image generation network G
Will generate samples->
Inputting into a small sample recognition network to perform prediction recognition calculation to obtain loss L T And updating parameters of the small sample recognition network F, and updating intermediate parameters of the image generation network G to generate more complex target domain style samples, and continuously training the small sample recognition network in a circulating way until the trained small sample recognition network meets the requirements.
4. A method according to any one of claims 1 to 3, wherein the stylistic conversion module stylizes the stylized parameters output by the variational self-encoder as stylized features to preserve the content features of the original samples and generate new samples with target domain data distribution features.
5. The method according to claim 2, wherein the metric matching module is configured to perform matching classification on the query set and the support set of the current task instance, and specifically includes: and calculating class centers for the N class support set samples, calculating distances from each sample of the query set to the N class centers, and classifying each sample of the query set to the class center closest to the sample of the query set to finish the identification and classification of the sample of the query set.
6. A method according to any one of claims 1 to 3, characterized in that the overall objective function of the small sample recognition network is:
wherein L is T Representing a stylized query set
Calculated classification loss, alpha denotes parameters of the small sample recognition network, A denotes the stylized support set based +.>
And a classifier selected corresponding to the small sample identification network parameter alpha, wherein omega is an output result of the small sample identification network.
7. A cross-domain small sample image recognition system, comprising:
the identification network determining unit is used for determining a trained small sample identification network; the small sample recognition network is used for carrying out matching classification on samples of different categories; the training process of the small sample recognition network requires an image generation network, wherein the image generation network comprises: the system comprises a variable self-encoder module and a style conversion module, wherein the variable self-encoder module is used for extracting and reconstructing characteristic distribution of a target domain sample, and comprises an intermediate parameter used for reconstructing the target domain sample into a sample with more complex style by gradient rising; the style conversion module is used for stylizing the source domain sample to obtain a labeled training sample belonging to the target domain data distribution; wherein different image styles correspond to different sample categories or data fields;
the sample recognition unit is used for inputting the target domain sample to be recognized into the trained small sample recognition network so as to conduct prediction classification and obtain an image recognition result.
8. The system of claim 7, wherein the small sample recognition network comprises: the feature extraction module and the measurement matching module; the characteristic extraction module is used for extracting characteristics of an input sample; the measurement matching module is used for carrying out matching classification on samples of different categories, and specifically comprises the following steps: the method for matching and classifying the query set and the support set of the current task example specifically comprises the following steps: and calculating class centers for the N class support set samples, calculating distances from each sample of the query set to the N class centers, and classifying each sample of the query set to the class center closest to the sample of the query set to finish the identification and classification of the sample of the query set.
9. The system of claim 7, further comprising: the recognition network training unit is used for circularly and randomly selecting N types of images from M types of images for each input task T when training the small sample recognition network F, determining K samples as a support set S for each type of images, determining Q samples as a query set Q for each type of images, and obtaining a generated sample after inputting the small sample recognition network F into the image generation network G
Will generate samples->
Inputting into a small sample recognition network to perform prediction recognition calculation to obtain loss L T And updating parameters of the small sample recognition network F, and updating intermediate parameters of the image generation network G to generate more complex target domain style samples, and continuously training the small sample recognition network in a circulating way until the trained small sample recognition network meets the requirements.
10. The system according to any one of claims 7 to 9, wherein the stylistic conversion module in the image generation network stylizes the stylized parameters output from the encoder as stylized features to preserve content features of original samples and generate new samples with target domain data distribution features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310258589.4A CN116342906A (en) | 2023-03-14 | 2023-03-14 | Cross-domain small sample image recognition method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310258589.4A CN116342906A (en) | 2023-03-14 | 2023-03-14 | Cross-domain small sample image recognition method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116342906A true CN116342906A (en) | 2023-06-27 |
Family
ID=86885139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310258589.4A Pending CN116342906A (en) | 2023-03-14 | 2023-03-14 | Cross-domain small sample image recognition method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116342906A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117132804A (en) * | 2023-07-04 | 2023-11-28 | 中国矿业大学 | Hyperspectral image classification method based on causal cross-domain small sample learning |
-
2023
- 2023-03-14 CN CN202310258589.4A patent/CN116342906A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117132804A (en) * | 2023-07-04 | 2023-11-28 | 中国矿业大学 | Hyperspectral image classification method based on causal cross-domain small sample learning |
| CN117132804B (en) * | 2023-07-04 | 2024-04-05 | 中国矿业大学 | Hyperspectral image classification method based on causal cross-domain small sample learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rajeswar et al. | Adversarial generation of natural language | |
| Zhu et al. | Iou-uniform r-cnn: Breaking through the limitations of rpn | |
| CN109471938B (en) | Text classification method and terminal | |
| CN110069709B (en) | Intention recognition method, device, computer readable medium and electronic equipment | |
| Subramanian et al. | Adversarial generation of natural language | |
| Chen et al. | Contrastive neural architecture search with neural architecture comparators | |
| Huang et al. | Multi-graph fusion and learning for RGBT image saliency detection | |
| Wei et al. | Coarse-to-fine: A RNN-based hierarchical attention model for vehicle re-identification | |
| Galteri et al. | Spatio-temporal closed-loop object detection | |
| Cong et al. | Self-supervised online metric learning with low rank constraint for scene categorization | |
| Sagayam et al. | A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition | |
| Li et al. | Robust deep neural networks for road extraction from remote sensing images | |
| CN113989549A (en) | Semi-supervised learning image classification optimization method and system based on pseudo labels | |
| Kyrkou et al. | Imaging time-series for NILM | |
| CN110717090A (en) | Network public praise evaluation method and system for scenic spots and electronic equipment | |
| Cao et al. | Stacked residual recurrent neural network with word weight for text classification | |
| CN116342906A (en) | Cross-domain small sample image recognition method and system | |
| Speth et al. | Automated Label Noise Identification for Facial Attribute Recognition. | |
| CN110598869B (en) | Classification method and device based on sequence model and electronic equipment | |
| WO2020135054A1 (en) | Method, device and apparatus for video recommendation and storage medium | |
| Bielski et al. | Pay Attention to Virality: understanding popularity of social media videos with the attention mechanism | |
| Chen et al. | Data augmentation for environmental sound classification using diffusion probabilistic model with top-k selection discriminator | |
| CN116737939B (en) | Meta learning method, text classification device, electronic equipment and storage medium | |
| CN112364198A (en) | Cross-modal Hash retrieval method, terminal device and storage medium | |
| Cai et al. | Semantic and correlation disentangled graph convolutions for multilabel image recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |