CN114332787A - Passive domain unsupervised domain self-adaptive vehicle re-identification method - Google Patents
Passive domain unsupervised domain self-adaptive vehicle re-identification method Download PDFInfo
- Publication number
- CN114332787A CN114332787A CN202111680087.8A CN202111680087A CN114332787A CN 114332787 A CN114332787 A CN 114332787A CN 202111680087 A CN202111680087 A CN 202111680087A CN 114332787 A CN114332787 A CN 114332787A
- Authority
- CN
- China
- Prior art keywords
- domain
- model
- source
- target
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013508 migration Methods 0.000 claims abstract description 14
- 230000005012 migration Effects 0.000 claims abstract description 14
- 238000013140 knowledge distillation Methods 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 18
- 230000003044 adaptive effect Effects 0.000 claims description 9
- 230000014759 maintenance of location Effects 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 229910052739 hydrogen Inorganic materials 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000004321 preservation Methods 0.000 claims description 3
- 239000004576 sand Substances 0.000 claims description 3
- 239000002131 composite material Substances 0.000 claims 1
- 238000010586 diagram Methods 0.000 claims 1
- 230000006978 adaptation Effects 0.000 description 6
- 238000013461 design Methods 0.000 description 2
- 238000004821 distillation Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000001010 compromised effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention provides a passive domain unsupervised domain self-adaptive vehicle re-recognition method. It no longer uses source domain data, but rather utilizes learned source domain knowledge implicit in the source domain model as a guide, thereby facilitating migration of the style of target domain data to the style of source domain data.
Description
Technical Field
The invention belongs to the technical field of machine learning and vehicle re-identification, and particularly relates to a passive domain unsupervised domain self-adaptive vehicle re-identification method.
Background
At present, most vehicle re-identification methods based on supervised learning achieve high precision, but the cost is extremely dependent on effective supervision labels, and the method cannot be well expanded to the unsupervised field. For this reason, some Unsupervised-based vehicle re-identification methods have come to mind, and here the main difference between the Unsupervised re-identification task and the currently focused Unsupervised Pre-training (Unsupervised Pre-training) task is emphasized: (1) the unsupervised pre-training task starts from network random initialization, and the unsupervised re-identification task starts from a pre-trained network; (2) the unsupervised pre-trained network can be applied to the downstream task only through the fine-tune, the unsupervised re-recognition task can be regarded as an unsupervised downstream task, and the network which is trained can be directly deployed. The unsupervised vehicle re-identification is divided into two directions of unsupervised domain adaptation and completely unsupervised.
The method is characterized in that the unsupervised domain self-adaptive task and the completely unsupervised task are similar in nature, the difference is that the unsupervised domain self-adaptive method additionally needs a source domain data set with a label, the model is pre-trained by using the source domain data, after the pre-training is finished, the source domain data is not used, and only the target domain data without the label is used for re-training to obtain a final model; while a completely unsupervised approach only requires training with unlabeled target domain data.
Currently, among the unsupervised vehicle re-identification tasks, the unsupervised domain adaptation task is currently the most interesting, while the completely unsupervised task, which does not rely on source domain data, is less studied.
Currently, a vehicle re-identification algorithm based on unsupervised domain adaptation usually needs a source domain model, and then utilizes labeled source domain data and unlabeled target domain data to perform a series of fine tuning, so that the model can have good performance on a target domain. However, as data privacy and security are increasingly valued by scientific research workers, the source domain data is not easy to acquire, and the source domain model is relatively easy to acquire.
In the conventional unsupervised domain adaptive method based on picture generation, it is generally necessary to promote the migration of the style of target domain data to the style of source domain data by using the source domain data as content and style guidance.
Disclosure of Invention
Considering that most vehicle re-identification methods based on supervised learning achieve higher precision in recent years, but depend heavily on effective supervised labels, how to deploy a vehicle re-identification model to a label-free data set and achieve better effect is a great difficulty. In addition, due to the limitations of data security and privacy, the source domain data acquisition is also difficult in practical applications.
In order to make up the blank and the defects of the prior art, the invention provides a passive domain unsupervised domain self-adaptive vehicle re-identification method. A generator is trained by using relationship keeping consistent loss and knowledge distillation loss through a source domain model and target domain data, so that a pseudo target sample with a source domain style is generated, and then the model is finely adjusted by using the pseudo target sample, so that the performance of the model is improved. It no longer uses source domain data, but rather utilizes learned source domain knowledge implicit in the source domain model as a guide, thereby facilitating migration of the style of target domain data to the style of source domain data.
Compared with other unsupervised domain self-adaption methods based on domain migration, the method has the greatest advantage that source domain data does not need to be accessed.
The invention specifically adopts the following technical scheme:
a method for re-identifying a vehicle in a passive domain in an unsupervised domain is characterized by comprising the following steps: in the vehicle re-identification process, a generator is trained by using relationship keeping consistent loss and knowledge distillation loss through a source domain model and target domain data, so that a pseudo target sample with a source domain style is generated, and then the model is finely adjusted by using the pseudo target sample, so that the performance of the model is improved.
Further, source domain data is not used, but rather source domain knowledge learned implicit in the source domain model is utilized as a guide, thereby facilitating migration of the style of the target domain data to the style of the source domain data.
And further, by using the synthetic image, generating a pseudo target sample by the synthetic image through the SPGAN for pre-training, and simultaneously receiving source domain data and target domain data to jointly train again, thereby completing the unsupervised domain self-adaptive task.
Further, two network models are designed through the synthesized image to be used for learning the direction similarity and the background similarity of the vehicle, and then the direction similarity and the camera similarity are subtracted from the vehicle ID similarity, so that the interference of the similar direction and the similar background to the models is reduced.
Further, the source domain model is obtained by a public pre-training model or by using a public data set with cross entropy loss and triplet loss pre-training; and the target domain model is obtained by loading the obtained source domain model parameters and utilizing target domain data to carry out fine adjustment.
Further, assume that only the feature extractor in the source model is adapted to the target domain; given a source model fS(. o) and an object model fT(. to train a generator for the passive knowledge migration module; setting the target image to x, given the generated image
Feature mapping of all source domain data output by source model
And probability distribution
To describe adaptation in the generated imageIn addition to the loss of knowledge distillation, a new relationship maintenance loss is introduced, which maintains the target image target model feature map fT(x) And the generated image source model feature map
Relative channel relationship between them;
the knowledge distillation loss is: in the passive knowledge migration module, a combination of source model and generator will be utilizedS(g (-) describing the target model fTThe knowledge adapted in (h) is regarded as a special case of knowledge distillation; extracting knowledge differences between the source domain data and the target domain data into a generator; using output from a model of a generating image feed source
And the target image feeds the output p (f) of the target modelT(x) Constitutes a loss of knowledge distillation:
the relationship retention penalty is: after successful knowledge distillation, the global features of the target image from the target model and the global features of the generated source domain style image from the source model should be similar, so a relationship preservation loss is used for constraint;
given source domain feature maps
And target domain feature map fT(x) It is first reshaped into a feature vector FSAnd FT，
Wherein D, H, W are feature map depth (number of channels), height, and width, respectively; then, their channel level autocorrelation, or gram matrix,
wherein G iss,GT∈RD×DAnd applying line L2 to normalize:
wherein [ i, ] represents the ith row in the matrix; finally, the relationship of the retention loss is defined as the mean square error MSE between the normalized glans matrices:
the total loss was:
through the two loss constraints, the source style image can be generated from the target image, so that the method is used for further fine adjustment of the model and the identification capability of the model is improved.
The present invention and its preferred aspects provide a new unsupervised domain adaptive vehicle re-identification framework for passive knowledge migration. Compared with other unsupervised domain adaptation methods based on domain migration, the method has the greatest advantage that source domain data does not need to be accessed. Only a model trained by source domain data needs to be obtained, and then the knowledge hidden in the source domain model can be specifically migrated to the target domain data by using the passive knowledge migration module.
Compared with the prior art, the invention and the preferred scheme thereof have the following technical advantages:
1. the existing method usually directly utilizes source domain data to pre-train a model, the source domain data is not used after pre-training, and only the target domain is used for training, so that real labels in the source domain data cannot be reasonably utilized.
The invention and the optimal scheme thereof apply the synthetic image in the field of vehicle re-identification, generate the synthetic image into a pseudo target sample through SPGAN for pre-training, and simultaneously can receive source domain data and target domain data for combined re-training, thereby completing the unsupervised domain self-adaptive task.
2. In the existing method, because the target image has no label information, the model can not be accurately distinguished for some interferences caused by the change of the vehicle direction or the view angle.
The invention and the preferable scheme thereof design two network models through the synthesized image to be used for learning the direction similarity and the background similarity of the vehicle, and then subtract the direction similarity and the camera similarity by the vehicle ID similarity so as to reduce the interference of the similar direction and the background to the models.
Drawings
FIG. 1 is a schematic flow chart of an overall method according to an embodiment of the present invention.
Detailed Description
In order to make the features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail as follows:
as shown in fig. 1, the method for re-identifying a vehicle in an unsupervised domain of a passive domain specifically includes the following design details:
(1) obtaining a source domain model and a target domain model: the source domain model may be obtained by downloading an open pre-trained model or pre-training with cross entropy loss and triplet loss using an open dataset, while the target domain model may be fine-tuned using target domain data by loading the obtained source domain model parameters.
(2) Source domain style image generation: according to the existing unsupervised domain adaptive working principle, the embodiment assumes that only the feature extractor in the source model is adapted to the targetA domain. Given a source model fS(. o) and an object model fT(..), the present embodiment trains a generator g (for the passive knowledge migration module). Since the training process is passive, the present embodiment will refer to the target image as x in the following for simplicity.
Given the generated image
Feature mapping of all source domain data output by source model
And probability distribution
To describe the adapted knowledge in the generated image, this embodiment introduces a new relationship maintenance penalty that maintains the target image target model feature map f, in addition to the conventional knowledge distillation penaltyT(x) And the generated image source model feature map
Relative channel relationship between them.
Knowledge distillation loss: in the passive knowledge migration module proposed by the present embodiment, a combination f of a source model and a generator is utilizedS(g (-) to describe the target model fTThe knowledge of adaptation in (can be seen as a special case of knowledge distillation. The purpose of this embodiment is to extract knowledge differences between source domain data and target domain data into the generator. In this case, the present embodiment utilizes the output of the model that generates the image feed source
And the target image feeds the output p (f) of the target modelT(x) Constitutes a loss of knowledge in distillation.
Relationship retention loss: after successful knowledge distillation, the present embodiment may think that the global features derived from the target image by the target model and the global features derived from the generated source domain style image by the source model should be similar, so the present embodiment utilizes the relationship preservation penalty to constrain. In the feature map fT(x) And
facilitating similar channel-level relationships therebetween helps achieve this goal.
In the past, the knowledge distillation work is usually restricted by maintaining the relation of batch level or pixel level. However, this way of constraining is not applicable to the current task. First, batch-level relationships do not provide good supervision over the task of generating each image, which can be detrimental to the effect of the generation. The validity of the pixel-level relationships is also compromised after global pooling. In contrast, channel-level relationships are computed on a per-image basis and are not affected by global pooling. Therefore, the channel level relationships are more suitable for calculating the relationship retention penalty.
Given source domain feature maps
And target domain feature map fT(x) This embodiment first needs to reshape it into a feature vector FSAnd FT，
Wherein D, H, W are feature map depth (number of channels), height, and width, respectively. Next, the present embodiment computes their channel level autocorrelation, or gram matrix,
wherein G iss,GT∈RD×D. As with other knowledge, the similarity of distillation remains lost, this example applies line L2 normalization,
wherein [ i, ] represents the ith row in the matrix. Finally, the present embodiment defines the relationship of the retention loss as the Mean Square Error (MSE) between the normalized gram matrices,
the total loss was:
through the two loss constraints, the embodiment can generate the source style image from the target image, so that the source style image is used for further fine adjustment of the model, and the model identification capability is improved.
The present invention is not limited to the above preferred embodiments, and various other passive domain unsupervised domain adaptive vehicle re-identification methods can be derived by anyone based on the teaching of the present invention.
Claims (6)
1. A method for re-identifying a vehicle in a passive domain in an unsupervised domain is characterized by comprising the following steps: in the vehicle re-identification process, a generator is trained by using relationship keeping consistent loss and knowledge distillation loss through a source domain model and target domain data, so that a pseudo target sample with a source domain style is generated, and then the model is finely adjusted by using the pseudo target sample, so that the performance of the model is improved.
2. The passive domain unsupervised domain adaptive vehicle re-identification method of claim 1, wherein: instead of using source domain data, source domain knowledge learned implicit in the source domain model is utilized as guidance to facilitate migration of the style of the target domain data to the style of the source domain data.
3. The passive domain unsupervised domain adaptive vehicle re-identification method of claim 2, wherein: and (3) applying the synthetic image, generating a pseudo target sample by the synthetic image through SPGAN for pre-training, and receiving the source domain data and the target domain data to jointly train again so as to complete the unsupervised domain self-adaptive task.
4. The passive domain unsupervised domain adaptive vehicle re-identification method of claim 3, wherein: two network models are designed through a composite image to be used for learning the direction similarity and the background similarity of the vehicle, and then the direction similarity and the camera similarity are subtracted from the vehicle ID similarity, so that the interference of the similar direction and the similar background to the models is reduced.
5. The passive domain unsupervised domain adaptive vehicle re-identification method of claim 4, wherein: the source domain model is obtained through an open pre-training model or through pre-training with cross entropy loss and triplet loss by utilizing an open data set; and the target domain model is obtained by loading the obtained source domain model parameters and utilizing target domain data to carry out fine adjustment.
6. The passive domain unsupervised domain adaptive vehicle re-identification method of claim 5, wherein:
assuming that only the feature extractor in the source model is adapted to the target domain; given a source model fS(. o) and an object model fT(. for passive knowledge migrationMoving the module to train a generator; setting the target image to x, given the generated image
Feature mapping of all source domain data output by source model
And probability distribution
In order to describe the adapted knowledge in the generated image, besides the loss of knowledge distillation, a new relationship maintenance loss is introduced, which maintains the target image target model characteristic diagram fT(x) And the generated image source model feature map
Relative channel relationship between them;
the knowledge distillation loss is: in the passive knowledge migration module, a combination of source model and generator will be utilizedS(g (-) describing the target model fTThe knowledge adapted in (h) is regarded as a special case of knowledge distillation; extracting knowledge differences between the source domain data and the target domain data into a generator; using output from a model of a generating image feed source
And the target image feeds the output p (f) of the target modelT(x) Constitutes a loss of knowledge distillation:
the relationship retention penalty is: after successful knowledge distillation, the global features of the target image from the target model and the global features of the generated source domain style image from the source model should be similar, so a relationship preservation loss is used for constraint;
given source domain feature maps
And target domain feature map fT(x) It is first reshaped into a feature vector FSAnd FT，
Wherein D, H, W are feature map depth (number of channels), height, and width, respectively; then, their channel level autocorrelation, or gram matrix,
wherein G iss，GT∈RD×DAnd applying line L2 to normalize:
wherein [ i ]: represents the ith row in the matrix; finally, the relationship of the retention loss is defined as the mean square error MSE between the normalized glans matrices:
the total loss was:
through the two loss constraints, the source style image can be generated from the target image, so that the method is used for further fine adjustment of the model and the identification capability of the model is improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111680087.8A CN114332787B (en) | 2021-12-30 | 2021-12-30 | Passive domain unsupervised domain self-adaptive vehicle re-identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111680087.8A CN114332787B (en) | 2021-12-30 | 2021-12-30 | Passive domain unsupervised domain self-adaptive vehicle re-identification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114332787A true CN114332787A (en) | 2022-04-12 |
| CN114332787B CN114332787B (en) | 2024-09-06 |
Family
ID=81022497
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111680087.8A Active CN114332787B (en) | 2021-12-30 | 2021-12-30 | Passive domain unsupervised domain self-adaptive vehicle re-identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114332787B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116543237A (en) * | 2023-06-27 | 2023-08-04 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Image classification method, system, equipment and medium for non-supervision domain adaptation of passive domain |
| CN116612445A (en) * | 2023-07-20 | 2023-08-18 | 华侨大学 | Unsupervised vehicle re-identification method based on self-adaptive clustering and difficult sample weighting |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110414368A (en) * | 2019-07-04 | 2019-11-05 | 华中科技大学 | A kind of unsupervised pedestrian recognition methods again of knowledge based distillation |
| CN112801019A (en) * | 2021-02-09 | 2021-05-14 | 福州大学 | Method and system for eliminating re-identification deviation of unsupervised vehicle based on synthetic data |
| US20210390686A1 (en) * | 2020-06-15 | 2021-12-16 | Dalian University Of Technology | Unsupervised content-preserved domain adaptation method for multiple ct lung texture recognition |
| US20210390355A1 (en) * | 2020-06-13 | 2021-12-16 | Zhejiang University | Image classification method based on reliable weighted optimal transport (rwot) |
-
2021
- 2021-12-30 CN CN202111680087.8A patent/CN114332787B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110414368A (en) * | 2019-07-04 | 2019-11-05 | 华中科技大学 | A kind of unsupervised pedestrian recognition methods again of knowledge based distillation |
| US20210390355A1 (en) * | 2020-06-13 | 2021-12-16 | Zhejiang University | Image classification method based on reliable weighted optimal transport (rwot) |
| US20210390686A1 (en) * | 2020-06-15 | 2021-12-16 | Dalian University Of Technology | Unsupervised content-preserved domain adaptation method for multiple ct lung texture recognition |
| CN112801019A (en) * | 2021-02-09 | 2021-05-14 | 福州大学 | Method and system for eliminating re-identification deviation of unsupervised vehicle based on synthetic data |
Non-Patent Citations (1)
| Title |
|---|
| 陈志鸿: ""关于视频图像多目标跟踪优化研究"", 《计算机仿真》, 30 September 2020 (2020-09-30) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116543237A (en) * | 2023-06-27 | 2023-08-04 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Image classification method, system, equipment and medium for non-supervision domain adaptation of passive domain |
| CN116543237B (en) * | 2023-06-27 | 2023-11-28 | 合肥综合性国家科学中心人工智能研究院(安徽省人工智能实验室) | Image classification method, system, equipment and medium for non-supervision domain adaptation of passive domain |
| CN116612445A (en) * | 2023-07-20 | 2023-08-18 | 华侨大学 | Unsupervised vehicle re-identification method based on self-adaptive clustering and difficult sample weighting |
| CN116612445B (en) * | 2023-07-20 | 2023-10-31 | 华侨大学 | Unsupervised vehicle re-identification method based on self-adaptive clustering and difficult sample weighting |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114332787B (en) | 2024-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114332787A (en) | Passive domain unsupervised domain self-adaptive vehicle re-identification method | |
| CN111783521B (en) | Pedestrian re-identification method based on low-rank prior guidance and based on domain invariant information separation | |
| CN107392131A (en) | A kind of action identification method based on skeleton nodal distance | |
| CN108268871A (en) | A kind of licence plate recognition method end to end and system based on convolutional neural networks | |
| CN112668648A (en) | Infrared and visible light fusion identification method based on symmetric fusion network | |
| CN111242061B (en) | Synthetic aperture radar ship target detection method based on attention mechanism | |
| CN106338733A (en) | Forward-looking sonar object tracking method based on frog-eye visual characteristic | |
| CN107092931B (en) | Method for identifying dairy cow individuals | |
| CN110659586A (en) | Cross-view gait recognition method based on identity maintenance cyclic generation type countermeasure network | |
| CN110097033A (en) | A kind of single sample face recognition method expanded based on feature | |
| CN112380374B (en) | Zero sample image classification method based on semantic expansion | |
| CN109359549A (en) | A kind of pedestrian detection method based on mixed Gaussian and HOG_LBP | |
| CN111291669A (en) | Two-channel depression angle human face fusion correction GAN network and human face fusion correction method | |
| CN112329832A (en) | Passive positioning target track data enhancement method and system based on deep convolution generation countermeasure network | |
| CN115205903B (en) | Pedestrian re-recognition method based on identity migration generation countermeasure network | |
| CN112017263B (en) | Intelligent test paper generation method and system based on deep learning | |
| Kwan et al. | Compressive vehicle tracking using deep learning | |
| CN114972904B (en) | Zero sample knowledge distillation method and system based on fighting against triplet loss | |
| CN101286236B (en) | Infrared object tracking method based on multi- characteristic image and average drifting | |
| CN114091510A (en) | Cross-domain vehicle weight identification method based on domain self-adaptation | |
| Sun et al. | Multi-AUV target recognition method based on GAN-meta learning | |
| CN113591545A (en) | Deep learning-based multistage feature extraction network pedestrian re-identification method | |
| CN112651329A (en) | Low-resolution ship classification method for generating countermeasure network through double-flow feature learning | |
| CN116665097A (en) | Self-adaptive target tracking method combining context awareness | |
| CN111507276A (en) | Construction site safety helmet detection method based on hidden layer enhancement features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |