CN112001398B - Domain adaptation method, device, apparatus, image processing method, and storage medium - Google Patents
Domain adaptation method, device, apparatus, image processing method, and storage medium Download PDFInfo
- Publication number
- CN112001398B CN112001398B CN202010869777.7A CN202010869777A CN112001398B CN 112001398 B CN112001398 B CN 112001398B CN 202010869777 A CN202010869777 A CN 202010869777A CN 112001398 B CN112001398 B CN 112001398B
- Authority
- CN
- China
- Prior art keywords
- image
- training
- training image
- fusion
- source domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 230000006978 adaptation Effects 0.000 title claims abstract description 42
- 238000003672 processing method Methods 0.000 title claims description 8
- 238000012549 training Methods 0.000 claims abstract description 533
- 238000012545 processing Methods 0.000 claims abstract description 157
- 230000004927 fusion Effects 0.000 claims abstract description 142
- 230000006870 function Effects 0.000 claims description 65
- 230000008569 process Effects 0.000 claims description 37
- 238000012512 characterization method Methods 0.000 claims description 25
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000009499 grossing Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 14
- 238000004891 communication Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 3
- 125000004122 cyclic group Chemical group 0.000 description 3
- 238000007499 fusion processing Methods 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000013508 migration Methods 0.000 description 2
- 230000005012 migration Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The domain adaptation method acquires a source domain training image and a target domain training image corresponding to the source domain training image after being converted into a target domain, and further fuses the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image, and effectively fuses pixel information of the source domain and pixel information of the target domain together through image fusion, so that the effect of aligning the pixels of the source domain and the target domain is achieved. Further, when the image processing model is trained, the fusion training image is used as a training sample, the reconstruction loss of the image processing model on the fusion training image is used as a loss function, and the image processing model is trained by taking the reconstruction loss as the loss function, so that the characteristics learned by the image processing model do not distinguish between the source domain and the target domain, and the characteristics have good adaptability to both the source domain and the target domain.
Description
Technical Field
The present invention relates to the field of domain adaptation technologies, and in particular, to a domain adaptation method, apparatus, device, image processing method, and storage medium.
Background
In real life, humans may have knowledge migration capabilities for some areas. Such as: if we can play the table tennis, we can play the tennis ball in analogy to learning; for another example, if we have played chess, we can be analogized to playing chess. It is desirable that the machine also have this capability, the ability to migrate knowledge. The domain adaptation method belongs to one of knowledge migration capabilities.
In machine learning, domain adaptation methods typically involve two (or more) domains, referred to as a source domain and a target domain, respectively. Typically, the data is tagged in the source domain and tagged in the target domain, and typically the tags of the source and target domains are shared. It is desirable to have a model trained in the source domain with a domain adaptation method that also works better in the target domain. If the source domain is the digital character of the printer and the target domain is the digital character of the handwriting, we hope that the character recognition model trained in the source domain has better effect on the target domain after using the domain adaptation method.
Disclosure of Invention
In view of the foregoing, the present application has been made to provide a domain adaptation method, apparatus, device, image processing method, and storage medium, so that an image processing model trained in a source domain can have a good recognition effect in a target domain through domain adaptation processing. The specific scheme is as follows:
A domain adaptation method, comprising:
acquiring a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted into a target domain;
fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
and training the image processing model by taking the fusion training image as a training sample and taking the reconstruction loss of the image processing model on the fusion training image as a loss function.
Preferably, the acquiring process of the target domain training image includes:
and converting the source domain training image into the target domain by utilizing the pre-trained image conversion model to obtain a corresponding target domain training image after converting the source domain training image into the target domain.
Preferably, the fusing the source domain training image and the corresponding target domain training image in the pixel space to obtain a fused training image includes:
the first image block of the target area in the source domain training image is scratched;
and acquiring a second image block of the target area in the target area training image, and filling the target area in the source area training image by using the second image block to obtain a fusion training image.
Preferably, the step of matting out the first image block of the target area in the source domain training image includes:
setting the pixel value of the image of the target area in the source domain training image to be zero to obtain a processed source domain training image;
the obtaining a second image block of the target area in the target area training image, filling the target area in the source area training image by using the second image block, and obtaining a fusion training image includes:
setting the pixel values of the images except the target area in the target area training image to be zero to obtain a processed target area training image;
and superposing the processed source domain training image and the processed target domain training image to obtain a fusion training image.
Preferably, the training the image processing model with the fused training image as a training sample and the reconstruction loss of the image processing model on the fused training image as a loss function includes:
taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image;
Determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
and training the image processing model by taking the reconstruction loss as a loss function.
Preferably, the loss function further comprises: a first classification penalty for the fused training image;
training the image processing model with the fused training image as a training sample, with a reconstruction penalty of the image processing model on the fused training image, and a first classification penalty of the fused training image as a penalty function, comprising:
taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image;
determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
using a first classification layer of the image processing model to classify the image based on the coding result to obtain a first classification result;
determining a first classification penalty for the fused training image based on the first classification result;
And training the image processing model by taking the reconstruction loss and the first classification loss as loss functions.
Preferably, the determining the first classification loss of the fusion training image based on the first classification result includes:
and determining a first classification loss of the fusion training image with label smoothing Lable smooth based on the first classification result and the classification label of the source domain training image corresponding to the fusion training image.
Preferably, the training sample further comprises: the source domain training image; the loss function further includes: a second classification penalty for the source domain training image;
training the image processing model with the fused training image and the source domain training image as training samples, with a reconstruction loss of the fused training image by the image processing model, and a second classification loss of the source domain training image as a loss function, comprising:
encoding the fusion training image and the source domain training image by using an encoding layer of an image processing model to obtain respective encoding results, and decoding the encoding results of the fusion training image by using a decoding layer to obtain a reconstructed image;
Determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
performing image classification based on the coding result of the source domain training image by using a second classification layer of the image processing model to obtain a second classification result;
determining a second classification loss of the source domain training image based on the second classification result;
and training the image processing model by taking the reconstruction loss and the second classification loss as loss functions.
An image processing method, comprising:
acquiring an image to be processed;
and processing the image to be processed by using the image processing model trained by the domain adaptation method to obtain an image processing result.
Preferably, the process of processing the image to be processed by using the image processing model includes:
and coding the image to be processed by using the image processing model to obtain hidden layer characterization features corresponding to the image to be processed.
A domain adaptation device, comprising:
the image acquisition unit is used for acquiring a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted into a target domain;
The image fusion unit is used for fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
the model training unit is used for taking the fusion training image as a training sample and taking the reconstruction loss of the image processing model on the fusion training image as a loss function to train the image processing model.
A domain adaptation device, comprising: a memory and a processor;
the memory is used for storing programs;
the processor is configured to execute the program to implement the steps of the domain adaptation method as described above.
A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the domain adaptation method as described above.
By means of the technical scheme, the domain adaptation method obtains the source domain training image and the corresponding target domain training image after the source domain training image is converted into the target domain, and further an image fusion strategy is adopted to fuse the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image, and through image fusion, the pixel information of the source domain and the pixel information of the target domain are effectively fused together, so that the effect of aligning the pixels of the source domain and the pixels of the target domain is achieved. Further, when the image processing model is trained, the fusion training image is used as a training sample, the reconstruction loss of the image processing model on the fusion training image is used as a loss function, and the image processing model is trained by taking the reconstruction loss as the loss function, so that the characteristics learned by the image processing model do not distinguish between the source domain and the target domain, the problem of characteristic alignment of the source domain and the target domain is solved, and the characteristics have good adaptability to the source domain and the target domain, namely, the image processing model also has good recognition effect in the target domain.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a domain adaptation method according to an embodiment of the present disclosure;
FIG. 2 illustrates a schematic diagram of an image conversion process;
FIG. 3 illustrates a schematic diagram of an image fusion process;
FIG. 4 illustrates a schematic diagram of a training process for an image processing model;
FIG. 5 illustrates a schematic diagram of a training process for another image processing model;
FIG. 6 illustrates a schematic diagram of a training process for yet another image processing model;
FIG. 7 illustrates a schematic diagram of a training process for yet another image processing model;
fig. 8 is a schematic structural diagram of a domain adaptation device according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a domain adaptation device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The application provides a domain adaptation scheme which can be used for domain adaptation problems of any computer vision, such as face recognition tasks, image classification tasks and the like. Through the domain adaptation scheme, the image processing model with good recognition effect in the target domain can be trained.
The scheme can be realized based on the terminal with the data processing capability, and the terminal can be a mobile phone, a computer, a server, a cloud terminal and the like.
Next, as described in connection with fig. 1, the domain adaptation method of the present application may include the following steps:
and step S100, acquiring a source domain training image, and converting the source domain training image into a target domain corresponding to the target domain training image.
Specifically, domain adaptation methods generally comprise two or more domains, referred to as a source domain and a target domain, respectively, wherein training images of the source domain are referred to herein simply as source domain training images. The source domain training image is typically annotated with information, while the target domain is not annotated with information.
And converting the source domain training image I into a target domain to obtain a corresponding target domain training image F (I). In the step, a source domain training image and a target domain training image are simultaneously acquired.
The target domain training image may be a target domain training image corresponding to a source domain training image obtained by converting the source domain training image into the target domain by using a pre-trained image conversion model. In addition, the source domain training images and corresponding target domain training images may be existing, such as directly acquired through a set of published training images, or the like.
For the above mentioned image conversion model, it may be a cyclic et al image conversion algorithm model. In order to enable the converted image to have higher definition quality, relay supervision can be added on the basis of the cyclic gram, feature maps of different layers in the middle of the cyclic gram model are used for generating conversion images with different resolutions, and then the conversion image with the highest resolution is selected from the conversion images to serve as the converted image, so that the conversion image with higher definition is obtained.
Referring to fig. 2, taking a heterogeneous face recognition task as an example, a source domain is a color RGB image, a target domain is an infrared NIR image as an example, and a source domain training image can be converted into a target domain training image through image conversion.
It should be noted that, the source domain training images and the target domain training images are in one-to-one correspondence, that is, for any source domain training image, there is one target domain training image corresponding to any source domain training image.
And step S110, fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image.
Specifically, in the embodiment of the present application, it is considered that the quality of the converted target domain training image may not be high, and when the target domain training image is directly used to train the image processing model, the model training effect may be poor, and the pixel information of the source domain training image cannot be utilized. Therefore, the embodiment of the application provides an image fusion strategy, namely, the source domain training image and the corresponding target domain training image are fused in a pixel space to obtain a fused training image. Through image fusion, the obtained fusion training image effectively fuses the pixel information of the source domain and the target domain together, and plays a role in aligning the pixels of the source domain and the target domain.
And step S120, training the image processing model by taking the fusion training image as a training sample and taking the reconstruction loss of the image processing model on the fusion training image as a loss function.
The image processing model can be a model which is already pre-trained by utilizing a source domain training image, or an untrained original model. The image processing model has the function of extracting hidden layer characterization features of an input training sample, and judges the performance of one image processing model, and mainly judges whether the extracted hidden layer characterization features are strong enough for the expression of the training sample. After extracting the hidden layer characterization feature of the training sample, the hidden layer characterization feature can be input into different task modules to obtain a corresponding task processing result, for example, the hidden layer characterization feature can be input into a classification module to obtain an image classification result of the training sample. In addition, the hidden layer characterization features can also be input into the coding layer, so that the reconstructed image and the like can be obtained through coding.
In the step, the fusion training image obtained in the previous step is used as a training sample to be input into an image processing model, and the image processing model is trained by taking the reconstruction loss of the image processing model on the fusion training image as a loss function.
The image processing model extracts hidden layer characterization features from the fusion training image, codes the fusion training image based on the extracted hidden layer characterization features to obtain a coded reconstructed image, further calculates the difference between the reconstructed image and the fusion training image, uses the difference as the reconstruction loss of the fusion training image, and trains the image processing model.
According to the domain adaptation method, the source domain training image is acquired, the corresponding target domain training image is converted from the source domain training image to the target domain, and then the image fusion strategy is adopted, the source domain training image and the corresponding target domain training image are fused in the pixel space to obtain the fusion training image, and through image fusion, the pixel information of the source domain and the pixel information of the target domain are effectively fused together, so that the effect of aligning the pixels of the source domain and the pixels of the target domain is achieved. Further, when the image processing model is trained, the fusion training image is used as a training sample, the reconstruction loss of the image processing model on the fusion training image is used as a loss function, and the image processing model is trained by taking the reconstruction loss as the loss function, so that the characteristics learned by the image processing model do not distinguish between the source domain and the target domain, the problem of characteristic alignment of the source domain and the target domain is solved, and the characteristics have good adaptability to the source domain and the target domain, namely, the image processing model also has good recognition effect in the target domain.
In some embodiments of the present application, the process of fusing the source domain training image and the corresponding target domain training image in the pixel space to obtain the fused training image is described in step S110.
In an alternative implementation manner, the image fusion process in the embodiment of the present application may be implemented as follows:
s1, the first image block of the target area in the source domain training image I is scratched.
The target area may be a preset area, such as an area with a size of 10×10 in the upper right corner of the image or other areas. In addition, the target area may be a randomly determined area that occupies a set proportion of the total area of the image, such as a randomly determined area that occupies 20% of the total area of the image.
And subtracting the first image corresponding to the target area from the source domain training image.
S2, acquiring a second image block of the target area in the target domain training image, and filling the target area in the source domain training image by using the second image block to obtain a fusion training image.
The size of the target domain training image obtained after the source domain training image is converted into the target domain is consistent with the size of the source domain training image. Therefore, in this step, the second image block corresponding to the target area in the target area training image may be obtained, and then the second image block is used to fill the target area in the source area training image, and the filled source area training image is used as the fusion training image.
Referring to fig. 3, a schematic diagram of a fused training image obtained by fusing a source domain training image and a target domain training image is illustrated.
As can be seen from fig. 3, the target area may be an area of the middle part of the right side in the image, and the pixel information of the source domain training image and the target domain training image may be fused into one image through the image fusion process.
Further optionally, for S1 above, the process of matting the first image block of the target area in the source domain training image may specifically include:
s11, setting the image pixel value of the target area in the source domain training image to be zero, and obtaining the processed source domain training image.
Specifically, the occlusion mask=0 can be added to the target area in the source domain training image I, and the mask=1 is added to the rest areas in the source domain training image I, that is, the image pixel value of the target area is set to zero.
On this basis, the step S2 of obtaining a second image block of the target region in the target region training image, and filling the target region in the source region training image with the second image block to obtain a fused training image may specifically include:
S21, setting the pixel values of the images except the target area in the target area training image to be zero, and obtaining the processed target area training image.
Specifically, the mask=1 may be added to the target area in the target area training image F (I), and the mask=0 may be added to the remaining areas, so as to obtain a processed target area training image.
S22, superposing the processed source domain training image and the processed target domain training image to obtain a fusion training image.
Specifically, the formula can be expressed as:
I fuse ＝I*mask+F(I)*(1-mask)
wherein I is fuse The method comprises the steps of representing a fusion training image, wherein I represents a source domain training image, F (I) represents a target domain training image corresponding to a target domain after the source domain training image I is converted, wherein the mask value comprises two parts, the mask value corresponding to the target region position is 0, and the mask value of other positions except the target region position is 1.
In some embodiments of the present application, the process of training the image processing model with the fused training image as a training sample and the reconstruction loss of the fused training image by the image processing model as a loss function in the step S120 is further described.
In connection with fig. 4, a schematic diagram of a training process for an image processing model is illustrated.
The implementation process of the step S120 may include the following steps:
s11, taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image.
The coding layer codes the fusion training image, and coded hidden layer characterization features can be obtained and used as a coding result.
The coding layer may use a ResNet50 structure. The decoding layer may use a multi-layer block stack of layers Deconvolution + InstanceNorm + Relu. Of course, the specific network structure of the coding layer and the decoding layer may also take other forms, which are not strictly limited in this application.
Fusion training image I in 3 x 112 dimensions fuse As input, 2048-dimensional hidden layer characterization feature vectors can be obtained after the encoding layer is processed, and then 2048-dimensional hidden layer characterization feature vector input can be usedAnd 3 x 112 dimension reconstructed images can be reconstructed by the deconvolution structure of the plurality of layers of the decoding layers.
S12, determining the reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image.
Specifically, the reconstruction loss of the fusion training image can be represented by L con Expressed, wherein the reconstruction loss can be expressed as:
wherein x is i Representing the size, y, of the ith pixel value in the fused training image i The i-th pixel value in the reconstructed image is represented, and n is the total pixel value number contained in the image.
S13, training the image processing model by taking the reconstruction loss as a loss function.
By training the image processing model by taking the reconstruction loss as a loss function, the characteristics learned by the image processing model do not distinguish the source domain from the target domain, the problem of characteristic alignment of the source domain and the target domain is solved, and the characteristics have good adaptability to both the source domain and the target domain, namely, the image processing model can have good recognition effect in the target domain.
On the basis of the above embodiment, a further training manner of the image processing model is described in the embodiment of the present application, and the loss function in the embodiment further increases the first classification loss L of the fused training image compared with the previous embodiment fuse Wherein the training samples remain unchanged as the fused training image. And the training process of the image processing model is to train by taking the fusion training image as a training sample, taking the reconstruction loss of the fusion training image by the image processing model and the first classification loss of the fusion training image as a loss function.
In connection with fig. 5, a schematic diagram of a training process of another image processing model is illustrated.
The training process of the image processing model may include:
s21, taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image.
S22, determining the reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image.
The steps S21 to S22 are identical to the steps S11 to S12, and are described in detail with reference to the foregoing, and are not repeated here.
S23, performing image classification based on the coding result by using a first classification layer of the image processing model to obtain a first classification result.
Specifically, the encoding result is input into a first classification layer, and the first classification layer classifies the fusion training image to obtain a first classification result.
S24, determining first classification loss of the fusion training image based on the first classification result.
Specifically, the classification label of the source domain training image can be used as the classification label of the corresponding fusion training image. Thus, the first classification loss L of the fused training image can be determined based on the first classification result and the classification label of the fused training image fuse 。
Wherein the first classification loss L fuse Can be expressed as:
where j represents the classified image category, p j Representing the probability that the training sample is predicted to belong to class j, q j Indicating a predetermined probability belonging to the j-th class.
Alternatively, qj may be 1 when a training sample belongs to the j-th class, otherwise q j Is 0.
Alternatively, consider that the fused training image is source domain trainingThe fusion result of the image and the target domain training image is not completely identical to the source domain training image, so that when the classification label of the source domain training image is used as the classification label of the fusion training image, the credibility of the classification label of the source domain training image cannot be completely ensured, and therefore, the first classification loss L with label smoothing Lable smooth can be selectively used in the embodiment fuse . That is, q j Instead of simply taking a 1 or 0, it is taken as follows:
where ε represents the dominant probability of the corresponding class of the source domain training image, which may generally be set to 0.1 or other small value between 0 and 1, K is the total number of classes of the classified image.
From this, it can be seen that when a certain training sample belongs to the j-th class, q j Taking 1-0.1=0.9 instead of directly taking 1, q when a certain training sample does not belong to the j-th class j Instead of taking 0 directly, 0.1/(K-1) is taken. That is, a little penalty is also made for the case where the classification is correct, so as to be suitable for the case where the classification label of the fusion training image is not completely trusted.
S25, training the image processing model by taking the reconstruction loss and the first classification loss as loss functions.
Specifically, in this embodiment, the loss function L includes two terms, i.e., L con And L fuse ：
L＝λL con +βL fuse
Wherein lambda and beta are super parameters of different loss weights, and can be set according to actual training conditions.
Compared with the previous embodiment, the loss function in the present embodiment increases the first classification loss of the fusion training image, so that the distinguishing capability of the features learned by the image processing model on the images of different source domains and target domains can be ensured.
On the basis of the embodiment, another embodiment of the image processing model is described in the embodimentIn the training mode, compared with the previous embodiment, the training samples in the present embodiment add the source domain training image, and the loss function further adds the second classification loss L of the source domain training image CE 。
On this basis, the training process of the image processing model can comprise two kinds of training processes, namely:
and the first training model is trained by taking the fusion training image and the source domain training image as training samples, taking the reconstruction loss of the image processing model on the fusion training image and the second classification loss of the source domain training image as a loss function.
And secondly, taking the fusion training image and the source domain training image as training samples, taking the reconstruction loss of the image processing model on the fusion training image, the first classification loss of the fusion training image and the second classification loss of the source domain training image as loss functions, and training the image processing model.
In contrast, the two training modes are distinguished by a loss function, and the second loss function increases the first classification loss of the fused training image compared with the first one.
Next, the two training modes are described separately.
For the first training mode, fig. 6 may be combined, which illustrates a schematic diagram of a training process of a further image processing model.
The training process of the image processing model may include:
s31, coding the fusion training image and the source domain training image by using a coding layer of an image processing model to obtain respective coding results, and decoding the coding results of the fusion training image by using a decoding layer to obtain a reconstructed image.
The same coding layer may be used for coding the fusion training image and the coding layer for coding the source domain training image, or two coding layers may be used, but the two coding layers share network parameters. The process of encoding the fusion training image and decoding to obtain the reconstructed image may be described with reference to the foregoing related description, which is not repeated here.
S32, determining the reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image.
Specifically, the process of determining the reconstruction loss of the fusion training image may be described with reference to the foregoing related description, which is not repeated herein.
S33, performing image classification based on the coding result of the source domain training image by using a second classification layer of the image processing model to obtain a second classification result.
Specifically, the encoding result of the source domain training image is input into a second classification layer, and the source domain training image is classified by the second classification layer to obtain a first classification result.
S34, determining second classification loss of the source domain training image based on the second classification result.
Specifically, the source domain training image carries a classification label, so that the second classification loss L of the source domain training image can be determined based on the second classification result and the classification label of the source domain training image CE 。
Wherein the second classification loss L CE Can be expressed as:
where j represents the classified image category, p j Representing the probability that the source domain training image is predicted to belong to the j-th class, y j To indicate a vector, when a certain source domain training image belongs to the j-th class, y j Take a value of 1, otherwise, y j The value is 0.
S35, using the reconstruction loss L con Said second classification loss L CE The image processing model is trained as a loss function.
Specifically, in this embodiment, the loss function L includes two terms, i.e., L con And L CE ：
L＝αL CE +λL con
Wherein, alpha and lambda are super parameters of different loss weights, which can be set according to actual training conditions.
Compared with the previous embodiment, the loss function in this embodiment increases the second classification loss of the source domain training image, so that the distinguishing capability of the features learned by the image processing model on different source domain training images can be ensured.
Further to the second training approach, fig. 7 may be combined, which illustrates a training process schematic of yet another image processing model.
The training process of the image processing model may include:
s41, respectively encoding the fusion training image and the source domain training image by using an encoding layer of an image processing model to obtain respective encoding results, and decoding the encoding results of the fusion training image by using a decoding layer to obtain a reconstructed image.
S42, determining the reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image.
Specifically, the above-mentioned S41-S42 are in one-to-one correspondence with the above-mentioned S31-S32, and detailed description thereof is omitted herein.
S43, performing image classification based on the coding result of the fusion training image by using a first classification layer of the image processing model to obtain a first classification result.
Specifically, the present step S43 corresponds to the previous step S23, and is described in detail with reference to the previous description, which is not repeated here.
S44, performing image classification based on the coding result of the source domain training image by using a second classification layer of the image processing model to obtain a second classification result.
Specifically, the present step S44 corresponds to the previous step S33, and the detailed description thereof is referred to above, and will not be repeated here.
S45, determining a first classification loss of the fusion training image based on the first classification result.
Specifically, the step S45 corresponds to the step S24, and is described in detail with reference to the foregoing descriptions, which are not repeated here.
S46, determining second classification loss of the source domain training image based on the second classification result.
Specifically, the present step S46 corresponds to the previous step S34, and the detailed description is referred to above in the related description, which is not repeated here.
S47, using the reconstruction loss L con Said first classification loss L fuse Said second classification loss L CE The image processing model is trained as a loss function.
Specifically, in the present embodiment, the loss function L includes three terms, i.e., L con 、L fuse And L CE ：
L＝αL CE +λL con +βL fuse
Wherein, alpha, lambda and beta are super parameters of different loss weights, which can be set according to actual training conditions.
Compared with the previous embodiment, the loss function in this embodiment includes the reconstruction loss of the fusion training image, the first classification loss of the fusion training image, and the second classification loss of the source domain training image, and by using the reconstruction loss, the features learned by the image processing model do not distinguish between the source domain and the target domain, so that the problem of alignment of the features of the source domain and the target domain is solved, and the features have better adaptability to both the source domain and the target domain, that is, the image processing model also has a good recognition effect in the target domain. Through fusing the first classification loss of the training image, the distinguishing capability of the features learned by the image processing model on images of different source domains and target domains can be ensured. The second classification loss of the source domain training image further improves the distinguishing capability of the features learned by the image processing model on different source domain images.
In some embodiments of the present application, an image processing method is further disclosed, where the image processing method uses the image processing model trained by the domain adaptation method of the foregoing embodiments to process the image to be processed, so as to obtain an image processing result.
The above has explained that the image processing model has the function of extracting hidden layer characterization features of an input image, and determines the performance of an image processing model, and mainly whether the extracted hidden layer characterization features are sufficiently strong for the expression of the input image. After extracting the hidden layer characterization feature of the input image, the hidden layer characterization feature can be input into different task modules to obtain a corresponding task processing result, for example, the hidden layer characterization feature can be input into a classification module to obtain an image classification result of the input image. In addition, the hidden layer characterization features can also be input into the coding layer, so that the reconstructed image and the like can be obtained through coding.
Therefore, in this embodiment, the image to be processed may be encoded by using the image processing model, so as to obtain the hidden layer characterization feature corresponding to the image to be processed, where the hidden layer characterization feature may be used as the image processing result. In addition, the image processing model can further utilize the hidden layer characterization feature to input different task models so as to obtain corresponding task processing results, and the corresponding task processing results can be used as image processing results, such as the hidden layer characterization feature is input into a classification module so as to obtain a classification result of an image to be processed.
In this embodiment, heterogeneous face recognition will be described as an example.
Heterogeneous face recognition refers to the process of matching (1:1) or retrieving (1:N) face images of two different domains (source and target).
For any two images of the source domain and the target domain, the images can be encoded through the image processing model to obtain the encoded hidden layer characterization features, and the similarity of the two images can be measured through the distance between the two hidden layer characterization features, such as Euclidean distance. For a 1:1 matching task, the distance can be directly compared with a set threshold, and when the distance is higher than the set threshold, the two images are considered to be not the same person, otherwise, the two images are considered to be the same person. For 1: n search tasks, namely, N images similar to the target image are required to be searched in an image set in a source domain or a target domain, the distance between the target image and each image in the image set can be calculated respectively, the first N images with smaller distance in the image set are selected to serve as N images similar to the target image according to the distance order.
The domain adaptation device provided in the embodiments of the present application will be described below, and the domain adaptation device described below and the domain adaptation method described above may be referred to correspondingly to each other.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a domain adaptation device according to an embodiment of the present application.
As shown in fig. 8, the apparatus may include:
an image obtaining unit 11, configured to obtain a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted to a target domain;
the image fusion unit 12 is configured to fuse the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
the model training unit 13 is configured to train the image processing model by using the fused training image as a training sample and using a reconstruction loss of the image processing model on the fused training image as a loss function.
Optionally, the process of acquiring the target domain training image by the image acquisition unit may include:
and converting the source domain training image into the target domain by utilizing the pre-trained image conversion model to obtain a corresponding target domain training image after converting the source domain training image into the target domain.
Optionally, the process of fusing the source domain training image and the corresponding target domain training image in the pixel space by the image fusion unit to obtain a fused training image may include:
The first image block of the target area in the source domain training image is scratched;
and acquiring a second image block of the target area in the target area training image, and filling the target area in the source area training image by using the second image block to obtain a fusion training image.
Optionally, the process of the image fusion unit to scratch the first image block of the target area in the source domain training image may include:
setting the pixel value of the image of the target area in the source domain training image to be zero to obtain a processed source domain training image;
on this basis, the process of obtaining the fused training image by the image fusion unit obtaining the second image block of the target area in the target area training image and filling the target area in the source area training image by using the second image block may include:
setting the pixel values of the images except the target area in the target area training image to be zero to obtain a processed target area training image;
and superposing the processed source domain training image and the processed target domain training image to obtain a fusion training image.
Optionally, the process of training the image processing model by using the fused training image as a training sample and using a loss of reconstruction of the fused training image by the image processing model as a loss function by the model training unit may include:
Taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image;
determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
and training the image processing model by taking the reconstruction loss as a loss function.
Optionally, the foregoing loss function may further include: a first classification penalty for the fused training image; on this basis, the model training unit may train the image processing model with the fused training image as a training sample, with a reconstruction loss of the fused training image by the image processing model, and a first classification loss of the fused training image as a loss function, and the training process may include:
taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image;
determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
Using a first classification layer of the image processing model to classify the image based on the coding result to obtain a first classification result;
determining a first classification penalty for the fused training image based on the first classification result;
and training the image processing model by taking the reconstruction loss and the first classification loss as loss functions.
Optionally, the process of determining the first classification loss of the fusion training image by the model training unit based on the first classification result may include:
and determining a first classification loss of the fusion training image with label smoothing Lable smooth based on the first classification result and the classification label of the source domain training image corresponding to the fusion training image.
Optionally, the training sample may further include: the source domain training image; the loss function may further include: the second classification of the source domain training image is lost. On this basis, the model training unit may train the image processing model by using the fused training image and the source domain training image as training samples, using a reconstruction loss of the image processing model on the fused training image, and a second classification loss of the source domain training image as a loss function, and the training process may include:
Encoding the fusion training image and the source domain training image by using an encoding layer of an image processing model to obtain respective encoding results, and decoding the encoding results of the fusion training image by using a decoding layer to obtain a reconstructed image;
determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
performing image classification based on the coding result of the source domain training image by using a second classification layer of the image processing model to obtain a second classification result;
determining a second classification loss of the source domain training image based on the second classification result;
and training the image processing model by taking the reconstruction loss and the second classification loss as loss functions.
The domain adaptation device provided by the embodiment of the application can be applied to domain adaptation equipment, such as a terminal: cell phones, computers, etc. Alternatively, fig. 9 shows a block diagram of a hardware structure of the domain adaptation device, and referring to fig. 9, the hardware structure of the domain adaptation device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
in the embodiment of the application, the number of the processor 1, the communication interface 2, the memory 3 and the communication bus 4 is at least one, and the processor 1, the communication interface 2 and the memory 3 complete communication with each other through the communication bus 4;
Processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention, etc.;
the memory 3 may comprise a high-speed RAM memory, and may further comprise a non-volatile memory (non-volatile memory) or the like, such as at least one magnetic disk memory;
wherein the memory stores a program, the processor is operable to invoke the program stored in the memory, the program operable to:
acquiring a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted into a target domain;
fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
and training the image processing model by taking the fusion training image as a training sample and taking the reconstruction loss of the image processing model on the fusion training image as a loss function.
Alternatively, the refinement function and the extension function of the program may be described with reference to the above.
The embodiment of the application also provides a storage medium, which may store a program adapted to be executed by a processor, the program being configured to:
Acquiring a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted into a target domain;
fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
and training the image processing model by taking the fusion training image as a training sample and taking the reconstruction loss of the image processing model on the fusion training image as a loss function.
Alternatively, the refinement function and the extension function of the program may be described with reference to the above.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and may be combined according to needs, and the same similar parts may be referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (13)
1. A domain adaptation method, comprising:
acquiring a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted into a target domain;
fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
training the image processing model by taking the fusion training image as a training sample and taking the reconstruction loss of the image processing model on the fusion training image as a loss function;
The training sample further comprises: the source domain training image; the loss function further includes: a second classification penalty for the source domain training image;
and training the image processing model by taking the fusion training image and the source domain training image as training samples and taking the reconstruction loss of the image processing model on the fusion training image and the second classification loss of the source domain training image as a loss function.
2. The method of claim 1, wherein the process of acquiring the target domain training image comprises:
and converting the source domain training image into the target domain by utilizing the pre-trained image conversion model to obtain a corresponding target domain training image after converting the source domain training image into the target domain.
3. The method of claim 1, wherein fusing the source domain training image and the corresponding target domain training image in pixel space to obtain a fused training image comprises:
the first image block of the target area in the source domain training image is scratched;
and acquiring a second image block of the target area in the target area training image, and filling the target area in the source area training image by using the second image block to obtain a fusion training image.
4. A method according to claim 3, wherein the matting the first image block of the target region in the source domain training image comprises:
setting the pixel value of the image of the target area in the source domain training image to be zero to obtain a processed source domain training image;
the obtaining a second image block of the target area in the target area training image, filling the target area in the source area training image by using the second image block, and obtaining a fusion training image includes:
setting the pixel values of the images except the target area in the target area training image to be zero to obtain a processed target area training image;
and superposing the processed source domain training image and the processed target domain training image to obtain a fusion training image.
5. The method of claim 1, wherein training the image processing model with the fused training image as a training sample and a loss of reconstruction of the fused training image by the image processing model as a loss function comprises:
taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image;
Determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
and training the image processing model by taking the reconstruction loss as a loss function.
6. The method of claim 1, wherein the loss function further comprises: a first classification penalty for the fused training image;
training the image processing model with the fused training image as a training sample, with a reconstruction penalty of the image processing model on the fused training image, and a first classification penalty of the fused training image as a penalty function, comprising:
taking the fusion training image as a training sample, coding the fusion training image by using a coding layer of an image processing model to obtain a coding result, and decoding the coding result by using a decoding layer to obtain a reconstructed image;
determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
using a first classification layer of the image processing model to classify the image based on the coding result to obtain a first classification result;
determining a first classification penalty for the fused training image based on the first classification result;
And training the image processing model by taking the reconstruction loss and the first classification loss as loss functions.
7. The method of claim 6, wherein the determining a first classification penalty for the fused training image based on the first classification result comprises:
and determining a first classification loss of the fusion training image with label smoothing Lable smooth based on the first classification result and the classification label of the source domain training image corresponding to the fusion training image.
8. The method of claim 1, wherein training the image processing model with the fused training image and the source domain training image as training samples, with a reconstruction penalty of the image processing model on the fused training image, and a second classification penalty of the source domain training image as a penalty function, comprises:
encoding the fusion training image and the source domain training image by using an encoding layer of an image processing model to obtain respective encoding results, and decoding the encoding results of the fusion training image by using a decoding layer to obtain a reconstructed image;
determining a reconstruction loss of the fusion training image based on the reconstruction image and the fusion training image;
Performing image classification based on the coding result of the source domain training image by using a second classification layer of the image processing model to obtain a second classification result;
determining a second classification loss of the source domain training image based on the second classification result;
and training the image processing model by taking the reconstruction loss and the second classification loss as loss functions.
9. An image processing method, comprising:
acquiring an image to be processed;
processing the image to be processed by using the image processing model trained by the domain adaptation method according to any one of claims 1-8 to obtain an image processing result.
10. The method of claim 9, wherein the processing of the image to be processed using the image processing model comprises:
and coding the image to be processed by using the image processing model to obtain hidden layer characterization features corresponding to the image to be processed.
11. A domain adaptation device, comprising:
the image acquisition unit is used for acquiring a source domain training image and a target domain training image corresponding to the source domain training image after the source domain training image is converted into a target domain;
The image fusion unit is used for fusing the source domain training image and the corresponding target domain training image in a pixel space to obtain a fused training image;
the model training unit is used for taking the fusion training image as a training sample, taking the reconstruction loss of the image processing model on the fusion training image as a loss function, and training the image processing model;
the training sample further comprises: the source domain training image; the loss function further includes: a second classification penalty for the source domain training image;
the model training unit takes the fusion training image and the source domain training image as training samples, and takes the reconstruction loss of the image processing model on the fusion training image and the second classification loss of the source domain training image as a loss function to train the image processing model.
12. A domain adaptation device, comprising: a memory and a processor;
the memory is used for storing programs;
the processor is configured to execute the program to implement the steps of the domain adaptation method according to any one of claims 1 to 8.
13. A storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the domain adaptation method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010869777.7A CN112001398B (en) | 2020-08-26 | 2020-08-26 | Domain adaptation method, device, apparatus, image processing method, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010869777.7A CN112001398B (en) | 2020-08-26 | 2020-08-26 | Domain adaptation method, device, apparatus, image processing method, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112001398A CN112001398A (en) | 2020-11-27 |
| CN112001398B true CN112001398B (en) | 2024-04-12 |
Family
ID=73471725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010869777.7A Active CN112001398B (en) | 2020-08-26 | 2020-08-26 | Domain adaptation method, device, apparatus, image processing method, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112001398B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255823B (en) * | 2021-06-15 | 2021-11-05 | 中国人民解放军国防科技大学 | Unsupervised domain adaptation method and unsupervised domain adaptation device |
| CN113642684A (en) * | 2021-10-18 | 2021-11-12 | 青岛根尖智能科技有限公司 | Robust cross-domain attitude estimation method and system based on image mixing mechanism |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108334904A (en) * | 2018-02-07 | 2018-07-27 | 深圳市唯特视科技有限公司 | A kind of multiple domain image conversion techniques based on unified generation confrontation network |
| CN109389174A (en) * | 2018-10-23 | 2019-02-26 | 四川大学 | A kind of crowd massing Sensitive Image Detection Method |
| CN109800677A (en) * | 2018-12-29 | 2019-05-24 | 西安交通大学 | A kind of cross-platform palm grain identification method |
| CN110197226A (en) * | 2019-05-30 | 2019-09-03 | 厦门大学 | A kind of unsupervised image interpretation method and system |
| CN110210561A (en) * | 2019-05-31 | 2019-09-06 | 北京市商汤科技开发有限公司 | Training method, object detection method and device, the storage medium of neural network |
| CN110414462A (en) * | 2019-08-02 | 2019-11-05 | 中科人工智能创新技术研究院(青岛)有限公司 | A kind of unsupervised cross-domain pedestrian recognition methods and system again |
| CN110516201A (en) * | 2019-08-20 | 2019-11-29 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and storage medium |
| WO2019228358A1 (en) * | 2018-05-31 | 2019-12-05 | 华为技术有限公司 | Deep neural network training method and apparatus |
| CN110634101A (en) * | 2019-09-06 | 2019-12-31 | 温州大学 | Unsupervised image-to-image conversion method based on random reconstruction |
| WO2020082748A1 (en) * | 2018-10-25 | 2020-04-30 | 腾讯科技（深圳）有限公司 | Detection model training method and apparatus and terminal device |
| CN111179277A (en) * | 2019-12-11 | 2020-05-19 | 中国科学院深圳先进技术研究院 | Unsupervised self-adaptive mammary gland lesion segmentation method |
| CN111353545A (en) * | 2020-03-09 | 2020-06-30 | 大连理工大学 | Plant disease and insect pest identification method based on sparse network migration |
| CN111444765A (en) * | 2020-02-24 | 2020-07-24 | 北京市商汤科技开发有限公司 | Image re-recognition method, training method of related model, related device and equipment |
| CN111476216A (en) * | 2020-05-26 | 2020-07-31 | 上海眼控科技股份有限公司 | Face recognition method and device, computer equipment and readable storage medium |
| CN111476805A (en) * | 2020-05-22 | 2020-07-31 | 南京大学 | Cross-source unsupervised domain adaptive segmentation model based on multiple constraints |
| CN111583201A (en) * | 2020-04-26 | 2020-08-25 | 浙江大学 | Transfer learning method for constructing super-resolution pathology microscope |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20200075344A (en) * | 2018-12-18 | 2020-06-26 | 삼성전자주식회사 | Detector, method of object detection, learning apparatus, and learning method for domain transformation |
-
2020
- 2020-08-26 CN CN202010869777.7A patent/CN112001398B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108334904A (en) * | 2018-02-07 | 2018-07-27 | 深圳市唯特视科技有限公司 | A kind of multiple domain image conversion techniques based on unified generation confrontation network |
| WO2019228358A1 (en) * | 2018-05-31 | 2019-12-05 | 华为技术有限公司 | Deep neural network training method and apparatus |
| CN109389174A (en) * | 2018-10-23 | 2019-02-26 | 四川大学 | A kind of crowd massing Sensitive Image Detection Method |
| WO2020082748A1 (en) * | 2018-10-25 | 2020-04-30 | 腾讯科技（深圳）有限公司 | Detection model training method and apparatus and terminal device |
| CN109800677A (en) * | 2018-12-29 | 2019-05-24 | 西安交通大学 | A kind of cross-platform palm grain identification method |
| CN110197226A (en) * | 2019-05-30 | 2019-09-03 | 厦门大学 | A kind of unsupervised image interpretation method and system |
| CN110210561A (en) * | 2019-05-31 | 2019-09-06 | 北京市商汤科技开发有限公司 | Training method, object detection method and device, the storage medium of neural network |
| CN110414462A (en) * | 2019-08-02 | 2019-11-05 | 中科人工智能创新技术研究院(青岛)有限公司 | A kind of unsupervised cross-domain pedestrian recognition methods and system again |
| CN110516201A (en) * | 2019-08-20 | 2019-11-29 | Oppo广东移动通信有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN110634101A (en) * | 2019-09-06 | 2019-12-31 | 温州大学 | Unsupervised image-to-image conversion method based on random reconstruction |
| CN111179277A (en) * | 2019-12-11 | 2020-05-19 | 中国科学院深圳先进技术研究院 | Unsupervised self-adaptive mammary gland lesion segmentation method |
| CN111444765A (en) * | 2020-02-24 | 2020-07-24 | 北京市商汤科技开发有限公司 | Image re-recognition method, training method of related model, related device and equipment |
| CN111353545A (en) * | 2020-03-09 | 2020-06-30 | 大连理工大学 | Plant disease and insect pest identification method based on sparse network migration |
| CN111583201A (en) * | 2020-04-26 | 2020-08-25 | 浙江大学 | Transfer learning method for constructing super-resolution pathology microscope |
| CN111476805A (en) * | 2020-05-22 | 2020-07-31 | 南京大学 | Cross-source unsupervised domain adaptive segmentation model based on multiple constraints |
| CN111476216A (en) * | 2020-05-26 | 2020-07-31 | 上海眼控科技股份有限公司 | Face recognition method and device, computer equipment and readable storage medium |
Non-Patent Citations (4)
| Title |
|---|
| Deep Reconstruction-Classification Networks for Unsupervised Domain Adaptation;Muhammad Ghifary等;《European Conference on Computer Vision》;597-613 * |
| Image to Image Translation for Domain Adaptation;Zak Murez等;《Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition》;4500-4509 * |
| 域自适应表情识别研究;胡玉晗;《中国优秀硕士学位论文全文数据库 信息科技辑》;第2020年卷(第7期);I138-890 * |
| 基于深度学习和域自适应的图像语义分割;张轶岳;《中国优秀硕士学位论文全文数据库 信息科技辑》;第2020年卷(第7期);I138-1275 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112001398A (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110458282B (en) | Multi-angle multi-mode fused image description generation method and system | |
| Zhang et al. | Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining | |
| US8605957B2 (en) | Face clustering device, face clustering method, and program | |
| CN112001398B (en) | Domain adaptation method, device, apparatus, image processing method, and storage medium | |
| CN108734210B (en) | Object detection method based on cross-modal multi-scale feature fusion | |
| CN111382555B (en) | Data processing method, medium, device and computing equipment | |
| CN106874826A (en) | Face key point-tracking method and device | |
| CN109598231A (en) | A kind of recognition methods of video watermark, device, equipment and storage medium | |
| Liu et al. | Deep active learning with contaminated tags for image aesthetics assessment | |
| CN111666922A (en) | Video matching method and device, computer equipment and storage medium | |
| CN110415184B (en) | Multi-modal image enhancement method based on orthogonal element space | |
| CN112036260A (en) | Expression recognition method and system for multi-scale sub-block aggregation in natural environment | |
| CN112200041A (en) | Video motion recognition method and device, storage medium and electronic equipment | |
| Han et al. | Class-aware feature aggregation network for video object detection | |
| WO2022166840A1 (en) | Face attribute editing model training method, face attribute editing method and device | |
| CN113949646B (en) | Web service QoS prediction method based on deep learning | |
| CN114926742A (en) | Loop detection and optimization method based on second-order attention mechanism | |
| CN114168773A (en) | Semi-supervised sketch image retrieval method based on pseudo label and reordering | |
| CN113128526B (en) | Image recognition method and device, electronic equipment and computer-readable storage medium | |
| CN112528077B (en) | Video face retrieval method and system based on video embedding | |
| CN112101135A (en) | Moving target detection method and device and terminal equipment | |
| You et al. | Pixel exclusion: Uncertainty-aware boundary discovery for active cross-domain semantic segmentation | |
| CN110659679A (en) | Image source identification method based on adaptive filtering and coupling coding | |
| CN115905838A (en) | Audio-visual auxiliary fine-grained tactile signal reconstruction method | |
| CN114445662A (en) | Robust image classification method and system based on label embedding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |