CN115620338A - Method and device for re-identifying pedestrians who change clothes guided by black clothes and head images - Google Patents

Abstract

The invention discloses a method and device for re-recognition of pedestrians who change clothes guided by black clothes and head images. The method includes: firstly, using a newly designed method of occluding clothes to process the original image to obtain the corresponding black clothes image. The obtained black clothes image is placed in the black clothes branch for pre-training; then the framework is jointly learned, the original pedestrian image is put into the original branch and the pre-trained black clothes branch is used to guide the original branch learning; at the same time, the pedestrian head image Put into the head branch to obtain finer-grained pedestrian features. The present invention blocks the clothes of all pedestrians, thereby obtaining the image of black clothes, making the clothes of pedestrians uniform in color, so that the model pays more attention to the parts other than the color of the clothes, thereby improving the robustness of the model; and can effectively use the information, effectively reducing the information loss in the process of acquiring black clothes images, and improving the robustness of features.

Description

Method and device for re-identifying pedestrians who change clothes guided by black clothes and head images

technical field

The invention relates to the technical field of pedestrian re-identification, in particular to a method and device for re-identifying pedestrians who change clothes and are guided by black clothes and head images.

Background technique

The purpose of person re-identification is to solve the problem of person retrieval under different conditions, such as different cameras, different lights or different viewing angles. There are many methods for pedestrian re-identification research, such as lightweight networks, domain generalization, unsupervised learning and other sub-fields, all of which have achieved good results in recent years. But these methods generally assume that a person's clothing remains consistent over a long period of time.

But in the real world, people's clothes don't stay the same. For example, people always wear different clothes for a long time, and some suspects may evade tracking by changing their clothes in a short period of time. Therefore, a different version of the person re-ID problem is proposed, which is called long-term clothes-changing person re-ID, and has become a hot topic nowadays. In the long run, re-identification of people changing clothes is a hot issue nowadays. The core of solving clothes-changing person re-identification is to extract discriminative relevant features that are only related to identity. To remove distractors from clothes, researchers generally employ two general strategies.

The first is data strategy. A common approach is to build a large-scale dataset where each person should have multiple pictures with a large number of different clothes, and then force the model to learn clothes-independent features from these pictures. However, constructing such a clothes-changing dataset purely by human power is very arduous, almost impossible. Therefore, some researchers utilize GAN or other means to extend the original dataset.

The second is the feature separation strategy. A common operation is to separate clothing features from other identity features. By doing this, other features besides clothes can be used for identity judgment. For example, Yang et al. adopted the silhouette of pedestrians as a query and gallery, and utilized polar coordinates to better obtain pedestrian silhouette features. However, although learning from silhouettes can obtain clothing-independent features, it simultaneously discards a large part of clothing-independent features (such as the head). Besides, Hong et al. proposed to use appearance branch and shape branch to extract fine-grained features. However, this method is often affected by different colors of clothes and cannot extract more robust features that are not related to clothes.

The main problems of the prior art:

1. The existing re-identification methods for people changing clothes require a lot of image generation work and a lot of training time.

2. Existing re-identification methods for people changing clothes are often affected by different colors of clothes, and cannot extract more robust features that are not related to clothes.

3. Most of the existing re-identification methods for people changing clothes use the traditional convolutional neural network as the training network. The traditional convolutional neural network brings a certain loss due to downsampling and pooling.

4. Most of the existing re-identification methods for people changing clothes ignore the influence of head features on the overall judgment.

Contents of the invention

Aiming at the above-mentioned problems in the background technology, the present invention proposes a method and device for re-identifying pedestrians who change clothes guided by black clothes and head images, uses a non-GAN method to extract features that have nothing to do with clothes in the image, and proposes a new occlusion The clothes strategy makes the clothes of all pedestrians tend to be consistent, and forces the model to learn robust features that have nothing to do with clothes. The improved Transformer is used as the training network, and the head branch is designed separately to obtain the fine-grained head features of the original image.

In order to achieve the above object, the present invention adopts the following technical solutions:

On the one hand, the present invention proposes a method for re-identifying pedestrians who change clothes guided by black clothes and head images, including:

Step 1: Remove the clothes feature from the original pedestrian image to get a black clothed pedestrian image;

Step 2: Use the pre-trained HRNet to process the original pedestrian image to obtain the pedestrian head image in the original pedestrian image;

Step 3: Build a re-recognition network for pedestrians in clothes, which consists of three network branches, namely the original branch, the black branch, and the head branch, which are used to learn the original pedestrian image features, black clothes pedestrian image features, and pedestrian Head image features; the three network branches have the same backbone network structure, but do not share parameters;

Step 4: Input the black-clothed pedestrian image obtained in step 1 into the black-clothed branch for training, and obtain the pre-trained black-clothed branch;

Step 5: Train the original branch under the guidance of the pre-trained black branch to obtain the features related to pedestrians but not related to clothes in the original pedestrian image;

Step 6: Input the pedestrian head image obtained in step 2 into the head branch for learning, and combine the learned pedestrian head image features with the features related to pedestrians but not related to clothes obtained in step 5 to obtain pedestrian The overall features that are relevant but not related to clothes, complete the training of the re-identification network for people changing clothes;

Step 7: Perform re-identification of people changing clothes based on the trained re-identification network for people changing clothes.

Further, said step 1 includes:

Use the pre-trained human body analysis model to obtain the body part images of pedestrians in the original pedestrian image, and recombine the obtained body part images to obtain six parts: background, head, top, pants, arms and Legs, from which the pixels of the top and trousers images are extracted to form the clothes region; all pixels in the clothes region are set to zero, resulting in a black clothed pedestrian image.

Furthermore, the backbone networks of the three network branches are all imViT networks. In each imViT network, two outputs are used to extract global features and local features respectively. The backbone network consists of triplet loss on global features and local features and The ID loss is optimized separately, where the ID loss is the cross-entropy loss without label smoothing.

Further, said step 5 includes:

According to the knowledge distillation algorithm, the branch of the original image is trained under the guidance of the pre-trained black branch, and the mean square error loss is used to standardize the training of the original branch to train more features related to identity but not related to clothes.

Another aspect of the present invention proposes a device for re-identifying pedestrians who change clothes guided by black clothes and head images, including:

The image of the black clothes is derived from the module, which is used to delete the clothes feature from the original pedestrian image, and obtain the black clothes pedestrian image;

The head image obtains the module, is used to process the original pedestrian image using pre-trained HRNet, and obtains the pedestrian head image in the original pedestrian image;

Clothes-changing pedestrian re-identification network building module is used to build a clothing-changing pedestrian re-identification network. The network consists of three network branches, namely the original branch, the black branch, and the head branch, which are used to learn the original pedestrian image features, Image features of pedestrians in black clothes, image features of pedestrians' heads; the backbone network structure of the three network branches is the same, but do not share parameters;

The black clothing branch training module is used to input the black clothing pedestrian image obtained by the black clothing image module into the black clothing branch for training to obtain the pre-trained black clothing branch;

The original branch training module is used to train the original branch under the guidance of the pre-trained black branch, and obtain the features related to pedestrians but not related to clothes in the original pedestrian image;

The head branch training module is used to obtain the head image obtained by the head image module and input the head image into the head branch for learning, and the learned pedestrian head image features are related to the pedestrian but not related to the original branch training module. Combining the features that are not related to clothes, the overall features that are related to pedestrians but not related to clothes are obtained, and the training of re-identification network for pedestrians who change clothes is completed;

The re-identification module of people changing clothes is used for re-identifying people changing clothes based on the trained re-identification network for people changing clothes.

Further, the black clothes image obtaining module is specifically used for:

Use the pre-trained human body analysis model to obtain the body part images of pedestrians in the original pedestrian image, and recombine the obtained body part images to obtain six parts: background, head, top, pants, arms and Legs, from which the pixels of the top and trousers images are extracted to form the clothes region; all pixels in the clothes region are set to zero, resulting in a black clothed pedestrian image.

Furthermore, the backbone networks of the three network branches are all imViT networks. In each imViT network, two outputs are used to extract global features and local features respectively. The backbone network consists of triplet loss on global features and local features and The ID loss is optimized separately, where the ID loss is the cross-entropy loss without label smoothing.

Further, the original branch training module is specifically used for:

According to the knowledge distillation algorithm, the branch of the original image is trained under the guidance of the pre-trained black branch, and the mean square error loss is used to standardize the training of the original branch to train more features related to identity but not related to clothes.

Compared with the prior art, the present invention has the beneficial effects:

1. Compared with the method of using GAN to generate a large number of clothes-changing image expansion data sets, the present invention uses a non-GAN method. Use the human body semantic analysis model to obtain the top and trousers of the human body, and use the method of the present invention to block them. In this way, the model can learn features that have nothing to do with clothes more intensively without expanding the data set, which saves space. Another time saver.

2. Compared with other methods that separate clothing features and identity features, the present invention blocks all pedestrians' clothes to obtain black clothes images, which makes the pedestrians' clothes uniform in color, so that the model pays more attention to things other than the color of clothes part, thereby improving the robustness of the model.

3. Compared with learning features directly from the image after excluding clothes, the present invention utilizes the black branch to guide the original branch to learn clothes-independent identity features directly from the original RGB image, so that the information in the original image can be effectively used, Effectively reduce the information loss in the process of acquiring black clothes images and improve the robustness of features.

4. Compared with the method of directly extracting clothing-independent features from the original image, the present invention can extract more discriminative fine-grained features by adding special processing of the head image block, which has a good supplementary effect on the global features .

5. The test results on the PRCC data set show that the method of the present invention achieves excellent re-identification results for people changing clothes.

Description of drawings

Fig. 1 is a basic flowchart of a method for re-identifying pedestrians who change clothes guided by black clothes and head images according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a re-identification network architecture for clothes-changing pedestrians constructed in an embodiment of the present invention;

Fig. 3 is a comparison example of feature activation maps obtained in the embodiment of the present invention;

Fig. 4 is a schematic structural diagram of a device for re-identifying people who change clothes guided by black clothes and head images according to an embodiment of the present invention.

detailed description

The present invention will be further explained below in conjunction with accompanying drawing and specific embodiment:

As shown in Figure 1, a method for re-identification of pedestrians who change clothes guided by black clothes and head images, including:

Step 1: Delete the clothes feature from the original pedestrian image to get the black clothes pedestrian image (referred to as the black image);

Step 2: Use the pre-trained HRNet to process the original pedestrian image to obtain the pedestrian head image in the original pedestrian image;

Step 3: Build a re-recognition network for pedestrians who change clothes. The network consists of three network branches, namely the original branch, the black branch, and the head branch. As shown in Figure 2, they are used to learn the original pedestrian image features, black Clothes pedestrian image features, pedestrian head image features; the three network branches have the same backbone network structure, but do not share parameters to better utilize the discriminative feature space;

Step 4: Input the black-clothed pedestrian image obtained in step 1 into the black-clothed branch for training, and obtain the pre-trained black-clothed branch;

Step 5: Train the original branch under the guidance of the pre-trained black branch to obtain the features related to pedestrians but not related to clothes in the original pedestrian image;

Step 6: Input the pedestrian head image obtained in step 2 into the head branch for learning, and combine the learned pedestrian head image features with the features related to pedestrians but not related to clothes obtained in step 5 to obtain pedestrian The overall features that are relevant but not related to clothes, complete the training of the re-identification network for people changing clothes;

Step 7: Perform re-identification of people changing clothes based on the trained re-identification network for people changing clothes.

Further, said step 1 includes:

Use the pre-trained human body analysis model to obtain the body part images of pedestrians in the original pedestrian image, and recombine the obtained body part images to obtain six parts: background, head, top, pants, arms and Legs, from which the pixels of the top and trousers images are extracted to form the clothes region; all pixels in the clothes region are set to zero, resulting in a black clothed pedestrian image.

Furthermore, the backbone networks of the three network branches are all imViT networks. In each imViT network, two outputs are used to extract global features and local features respectively. The backbone network consists of triplet loss on global features and local features and The ID loss is optimized separately, where the ID loss is the cross-entropy loss without label smoothing.

Further, said step 5 includes:

According to the knowledge distillation algorithm, the branch of the original image is trained under the guidance of the pre-trained black branch, and the mean square error loss is used to standardize the training of the original branch to train more features related to identity but not related to clothes.

Specifically, the method includes:

1. Get the black image

In order to eliminate the influence of clothes in feature extraction, we remove the clothes features from the original image, and get black clothes pedestrian images. First, we use HRNet, which is a pre-trained human parsing model, to obtain images of body parts. The prediction result of this model divides the body into 20 parts, and since this model divides the 20 parts, we recombine them to get six parts: background, head, top, pants, arms and legs. We extract the top and pants from it, forming the clothes area. We only use two of them (top, pants).

First, for a given input batch sample x _i [i=1....B], where B is the batch size, x _i is actually an image, x _i ∈ R ^H×W×C , where H, W , C represent its height, width, channel number respectively. First, we denote the semantic graph of x _i parsed by the human body parsing model as s _i [i=1....B], s _i ∈ R ^1×H×W . The pixel values of s _i are defined as s _i ∈ {0,1,2,3,4,5} representing the six parts of the body respectively. Each pixel of _si is defined as 0, 1, 2, 3, 4 or 5, representing six parts of the body, respectively.

Second, get the pixels for the top and pants. Each pixel of _xi is defined as _vj with c values, for a total of W×H pixel vectors for each input sample _xi . And we represent all pixel vectors of tops and pants in _xi as:

B(clothes and pants)＝{v _j |v _j ＝x _i [s _i ＝＝2||s _i ＝＝3], i∈[1,B],j∈[1,N]}(1)

where N is the total number of pants and tops in each _xi and is different in each _xi , j represents the j _th pixel vector, s _i represents the semantic segmentation map, 2 represents the index of tops, and 3 represents the index of pants , _xi [s _i ==2||s _i ==3] represents the coat and trousers of the x _i pixel vector. Therefore, the original image x _i can be regarded as x _i =[v ₁ ,v ₂ ...v _c1 ,...,v _cn ,...v _last ], where [v _c1 ...v _cn ] belongs to Equation (1) obtains the pixels of the pants and the top.

Finally, the black image is obtained by setting the pixels of the pants and top to zero. Specifically, we set [v _c1 ...v _cn ]=0, then we can get x _i′ =[v ₁ ,v ₂ ...0,...v _last ], which is the corresponding black of x _i clothing image.

2. The backbone network of each branch

We choose imViT (see [He, Shuting, et al. "Transreid: Transformer-based object re-identification." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.1, 2, 4]) as the backbone of each of our branches network. In each imViT, there are two outputs for extracting global and local features. For local, we can choose how many local sub-features the local features in the network are composed of, and 4 local sub-features are selected in the experiment of the present invention. Therefore, for the i-th (i∈{0,1,2}) branch, we can get the local feature F _li and the global feature F _gi , where F _li = [F _li1 ,F _li2 ,F _{li3 ,F li4 ,} i ∈{0,1,2}].

The backbone network is optimized by triplet loss and ID loss on global and local features, respectively. ID loss is cross-entropy loss without label smoothing. As for the triplet loss L _tri is as follows:

L _tri ＝log(1+exp(||f _a -f _p || ₂ -||f _a -f _n || ₂ ))

where f _a represents anchors, f _p represents positive samples, and f _n represents negative samples. Therefore, each branch of our framework has two sets of loss functions, namely L _tri-gi , L _id-gi , and L _tri-li , L _id-li , where L _tri-gi , L _id-gi are respectively The triplet loss and ID loss corresponding to the global feature F _gi , L _tri-li and L _id-li are the triplet loss and ID loss corresponding to the local feature F _li , respectively.

3. The original branch directed by the black branch

The black branch can learn clothes-independent features. But some important discriminative information may be discarded in the process of obtaining the black clothes image, which is hidden in the original image. However, it is infeasible to directly extract clothes-independent identity features from raw images. According to the knowledge distillation algorithm, we train the original image branch under the guidance of the pre-trained black branch. Specifically, we employ a mean square error (MSE) loss to regularize the training of the original branch to train more identity-related but clothes-unrelated features. The mean square error loss L _mse is defined as

L _mse =L _mse-opg +L _mse-opl

Among them, F _l1 and F _g1 represent the local features and global features obtained by the black branch respectively, F _l2 and F _g2 respectively represent the local features and global features connected by the original branch guided by the black branch, L _mse-opg , L _{mse- opl} represents the mean square error loss corresponding to the global feature F _opg = [F _g1 , F _g2 ] and the local feature F _opl = [F _l1 , F _l2 ] respectively.

4. Extraction of head features

For the input raw image, we use a pre-trained HRNet to obtain the head image part. Put the obtained head image into imViT3 to obtain the merged local feature F _l3 and global feature F _g3 , where the former is formed by merging four local features F _l31 , F _l32 , F _l33 , and F _l34 . Similarly, the original branch guided by the black branch can get the connected local feature F _l2 and global feature F _g2 . To obtain more uncertain clothes-related features, we perform element-wise summation of F _l2 and F _l3 , F _g2 and F _g3 , which are defined as follows:

F _ohg =wF _g2 +(1-w)F _g3

F _ohl ＝wF _l2 +(1-w)F _l3

Among them, F _ohg and F _ohl respectively represent the global overall feature and local overall feature related to pedestrians but not related to clothes, w is the weight coefficient, w∈(0,1).

And the triplet loss and ID loss are applied in it, they are L _id-ohg , L _tri-ohg , L _id-ohl and L _tri-ohl , where L _id-ohg and L _tri-ohg represent F _ohg corresponding to The ID loss and triplet loss of , L _id-ohl and L _tri-ohl represent the ID loss and triplet loss corresponding to F _ohl respectively.

5. Joint training

和

上用ID损失和三元组损失(为了简洁起见，我们在图1的黑衣分支中省略了这两种损失)对其进行单独训练。然后我们得到预训练的黑衣分支。In training, there are two phases. First, we feed the black image into the black branch, and in the global feature

and

It is trained separately with ID loss and triplet loss (for brevity, we omit these two losses in the black branch in Fig. 1). Then we get the pretrained black branch.

Second, we fix the learned weights of the black branch and jointly train other branches. Therefore, the total loss function is defined as:

L _total (θ)＝l ₁ L _id (θ)+l ₂ L _tri (θ)+l ₃ L _mse (θ)

Among them, L _id (θ) represents the ID loss, L _tri (θ) represents the triplet loss, and L _mse (θ) represents the mean square error loss. l ₁ , l ₂ , l ₃ are trade-off parameters to balance the contribution of each loss. In the experiments of the present invention, l ₁ , l ₂ , and l ₃ are set to 0.25, 0.25, and 0.5, respectively.

For verifying effect of the present invention, carry out following experiment:

We tested the dataset of PRCC (PRCC includes 33698 pictures from 221 people, 3 different angles, and also provides silhouette sketch images of people, which is convenient for extracting people’s contour information) in different clothing settings and the same clothing settings. experiment. In the different clothes setting, the image from camera A is used as the candidate set gallery, while the image from camera C is used as the lookup set query. In the same clothes setting, the image of gallery is also from camera A, but the image of query is from camera B. The experimental results are shown in Table 1.

Table 1 Experimental results

J.Gilg,andG.Rigoll,“Lightweight multi-branch network for person re-identification,”in2021 IEEE International Conference on Image Processing(ICIP).IEEE,2021,pp.1129–1133.])。所有比较方法的结果都来自他们发表的论文。We made some comparisons between our method and some state-of-the-art methods on the PRCC dataset, including representative traditional person re-identification methods (PCB [Y.Sun, L.Zheng, Y.Yang, Q.Tian, and S. Wang, “Beyondpart models: Person retrieval with refined part pooling(and a strong convolutional baseline),” in Proceedings of the European conference on computer vision (ECCV), 2018, pp.480–496.], Zheng et al's method[ Z. Zheng, L. Zheng, and Y. Yang, "A discriminatively learned cnn embedding for personreidentification," ACM transactions on multimedia computing, communications, and applications (TOMM), vol.14, no.1, pp.1–20, 2017.], HPM [Y.Fu, Y.Wei, Y.Zhou, H.Shi, G.Huang, X.Wang, Z.Yao, and T.Huang, "Horizontal pyramid matching for person re-identification," in Proceedings of the AAAI conference on artificial intelligence, vol.33, no.01, 2019, pp.8295–8302.], HACNN [W.Li, X.Zhu, and S.Gong, "Harmonious attention network for person re- identification,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp.2285–2294.]) and re-identification method for people changing clothes (PRCC (sketch) [Q.Yang, A.Wu, and W.- S. Zheng, "Person re-identification by contour sketch under moderate clothing change," IEEE transactions on pattern analysis and machine intelligence, vol.43, no.6, pp.2029–2046, 2019.], GI-ReID (OSNet) [X.Jin, T.He, K.Zheng , Z.Yin, X.Shen, Z.Huang, R.Feng, J.Huang, Z.Chen, and X.-S.Hua, “Cloth-changing person re-identification from a single image with gait prediction and regularization, "in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp.14 278–14 287.], LightMBN [F. Herzog, X. Ji, T. Teepe, S.

J. Gilg, and G. Rigoll, “Lightweight multi-branch network for person re-identification,” in 2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021, pp.1129–1133.]). The results of all compared methods come from their published papers.

As can be seen from Table 1, the method of the present invention has achieved very good performance. On the same garment, although our method is slightly inferior to LightMBN, it surpasses almost all other methods. This shows that the method of the present invention has a better generalization ability in the re-identification of people with the same clothes. In the settings of different clothes, the method of the present invention shows obvious excellent performance. In mAP (mAP is used to evaluate the overall effect of the pedestrian re-identification algorithm; wherein AP refers to the average precision of a query sample, indicating that the model is in a certain sample The effect on the mAP is the average of all query samples AP, indicating the overall effect of the model on all query samples) exceeding LightMBN by 4.7%, and the accuracy of rank-1 (that is, R@1 in Table 1) exceeding 9.6% . This may be due to the fact that the method of the present invention is able to extract more identity features that are not related to clothes.

For an intuitive analysis, we randomly sample 3 images from PRCC and show the corresponding activation maps (CAMs) captured by the baseline and our method in Fig. 3. As can be seen from the first row, the activation regions of the baseline are concentrated on the clothes and the background, and many background and clothes regions are activated and used to recognize a person, which may confuse the recognition results. Also, the baseline pays less attention to the head. From the second row, it can be found that the activation points in our proposed method are more concentrated, and there are fewer background and clothes areas, which alleviates the impact of background and clothes on recognition. Furthermore, we also notice that our proposed model pays more attention to the shape of the head and body.

On the basis of the above embodiments, as shown in Figure 4, the present invention also proposes a device for re-identifying pedestrians who change clothes guided by black clothes and head images, including:

The image of the black clothes is derived from the module, which is used to delete the clothes feature from the original pedestrian image, and obtain the black clothes pedestrian image;

The head image obtains the module, is used to process the original pedestrian image using pre-trained HRNet, and obtains the pedestrian head image in the original pedestrian image;

Clothes-changing pedestrian re-identification network building module is used to build a clothing-changing pedestrian re-identification network. The network consists of three network branches, namely the original branch, the black branch, and the head branch, which are used to learn the original pedestrian image features, Image features of pedestrians in black clothes, image features of pedestrians' heads; the backbone network structure of the three network branches is the same, but do not share parameters;

The black clothing branch training module is used to input the black clothing pedestrian image obtained by the black clothing image module into the black clothing branch for training to obtain the pre-trained black clothing branch;

The original branch training module is used to train the original branch under the guidance of the pre-trained black branch, and obtain the features related to pedestrians but not related to clothes in the original pedestrian image;

The head branch training module is used to obtain the head image obtained by the head image module and input the head image into the head branch for learning, and the learned pedestrian head image features are related to the pedestrian but not related to the original branch training module. Combining the features that are not related to clothes, the overall features that are related to pedestrians but not related to clothes are obtained, and the training of re-identification network for pedestrians who change clothes is completed;

The re-identification module of people changing clothes is used for re-identifying people changing clothes based on the trained re-identification network for people changing clothes.

Further, the black clothes image obtaining module is specifically used for:

Use the pre-trained human body analysis model to obtain the body part images of pedestrians in the original pedestrian image, and recombine the obtained body part images to obtain six parts: background, head, top, pants, arms and Legs, from which the pixels of the top and trousers images are extracted to form the clothes region; all pixels in the clothes region are set to zero, resulting in a black clothed pedestrian image.

Furthermore, the backbone networks of the three network branches are all imViT networks. In each imViT network, two outputs are used to extract global features and local features respectively. The backbone network consists of triplet loss on global features and local features and The ID loss is optimized separately, where the ID loss is the cross-entropy loss without label smoothing.

Further, the original branch training module is specifically used for:

According to the knowledge distillation algorithm, the branch of the original image is trained under the guidance of the pre-trained black branch, and the mean square error loss is used to standardize the training of the original branch to train more features related to identity but not related to clothes.

In summary, compared with the method of using GAN to generate a large number of clothes-changing image expansion data sets, the present invention uses a non-GAN method. Use the human body semantic analysis model to obtain the top and trousers of the human body, and use the method of the present invention to block them. In this way, the model can learn features that have nothing to do with clothes more intensively without expanding the data set, which saves space. Another time saver. Compared with other methods that separate clothing features and identity features, the present invention blocks all pedestrians' clothes to obtain a black image, which makes the color of pedestrians' clothes uniform, so that the model pays more attention to parts other than the color of the clothes. Thereby improving the robustness of the model. Compared with learning features directly from the image after excluding clothes, the present invention uses the black branch to guide the original branch to learn clothes-independent identity features directly from the original RGB image, which can effectively use the information in the original image and effectively reduce The information is lost during the acquisition of the black clothes image, which improves the robustness of the features. Compared with the method of directly extracting clothing-independent features from the original image, the present invention can extract more discriminative fine-grained features by adding special processing of the head image block, which has a good supplementary effect on the global features. The test results on the PRCC data set show that the method of the present invention achieves excellent re-identification results for people changing clothes.

What is shown above is only a preferred embodiment of the present invention, and it should be pointed out that for those of ordinary skill in the art, some improvements and modifications can also be made without departing from the principle of the present invention. It should be regarded as the protection scope of the present invention.

Claims (8)

1. A method for re-identifying pedestrians who change clothes guided by black clothes and head images, characterized in that it comprises: Step 1: Remove the clothes feature from the original pedestrian image to get a black clothed pedestrian image; Step 2: Use the pre-trained HRNet to process the original pedestrian image to obtain the pedestrian head image in the original pedestrian image; Step 3: Build a re-recognition network for pedestrians in clothes, which consists of three network branches, namely the original branch, the black branch, and the head branch, which are used to learn the original pedestrian image features, black clothes pedestrian image features, and pedestrian Head image features; the three network branches have the same backbone network structure, but do not share parameters; Step 4: Input the black-clothed pedestrian image obtained in step 1 into the black-clothed branch for training, and obtain the pre-trained black-clothed branch; Step 5: Train the original branch under the guidance of the pre-trained black branch to obtain the features related to pedestrians but not related to clothes in the original pedestrian image; Step 6: Input the pedestrian head image obtained in step 2 into the head branch for learning, and combine the learned pedestrian head image features with the features related to pedestrians but not related to clothes obtained in step 5 to obtain pedestrian The overall features that are relevant but not related to clothes, complete the training of the re-identification network for people changing clothes; Step 7: Perform re-identification of people changing clothes based on the trained re-identification network for people changing clothes. 2. The method for re-identifying pedestrians who change clothes guided by black clothes and head images according to claim 1, wherein said step 1 comprises: Use the pre-trained human body analysis model to obtain the body part images of pedestrians in the original pedestrian image, and recombine the obtained body part images to obtain six parts: background, head, top, pants, arms and Legs, from which the pixels of the top and trousers images are extracted to form the clothes region; all pixels in the clothes region are set to zero, resulting in a black clothed pedestrian image. 3. The method for re-identifying pedestrians who change clothes under the guidance of black clothes and head images according to claim 1, wherein the three network branch backbone networks are all imViT networks, and in each imViT network, there are two The output is used to extract global features and local features, respectively, and the backbone network is optimized by triplet loss and ID loss on global features and local features, respectively, where ID loss is cross-entropy loss without label smoothing. 4. The re-identification method for people changing clothes guided by black clothes and head images according to claim 1, wherein said step 5 comprises: According to the knowledge distillation algorithm, the branch of the original image is trained under the guidance of the pre-trained black branch, and the mean square error loss is used to standardize the training of the original branch to train more features related to identity but not related to clothes. 5. A clothing-changing pedestrian re-identification device guided by black clothes and head images, characterized in that it includes: The image of the black clothes is derived from the module, which is used to delete the clothes feature from the original pedestrian image, and obtain the black clothes pedestrian image; The head image obtains the module, is used to process the original pedestrian image using pre-trained HRNet, and obtains the pedestrian head image in the original pedestrian image; Clothes-changing pedestrian re-identification network building module is used to build a clothing-changing pedestrian re-identification network. The network consists of three network branches, namely the original branch, the black branch, and the head branch, which are used to learn the original pedestrian image features, Image features of pedestrians in black clothes, image features of pedestrians' heads; the backbone network structure of the three network branches is the same, but do not share parameters; The black clothing branch training module is used to input the black clothing pedestrian image obtained by the black clothing image module into the black clothing branch for training to obtain the pre-trained black clothing branch; The original branch training module is used to train the original branch under the guidance of the pre-trained black branch, and obtain the features related to pedestrians but not related to clothes in the original pedestrian image; The head branch training module is used to obtain the head image obtained by the head image module and input the head image into the head branch for learning, and the learned pedestrian head image features are related to the pedestrian but not related to the original branch training module. Combining the features that are not related to clothes, the overall features that are related to pedestrians but not related to clothes are obtained, and the training of re-identification network for pedestrians who change clothes is completed; The re-identification module of people changing clothes is used for re-identifying people changing clothes based on the trained re-identification network for people changing clothes. 6. The clothing-changing pedestrian re-identification device guided by black clothes and head images according to claim 5, wherein the black clothes image deriving module is specifically used for: Use the pre-trained human body analysis model to obtain the body part images of pedestrians in the original pedestrian image, and recombine the obtained body part images to obtain six parts: background, head, top, pants, arms and Legs, from which the pixels of the top and trousers images are extracted to form the clothes region; all pixels in the clothes region are set to zero, resulting in a black clothed pedestrian image. 7. The clothing-changing pedestrian re-identification device guided by black clothes and head images according to claim 5, characterized in that, the three network branch backbone networks are all imViT networks, and in each imViT network, there are two The output is used to extract global features and local features, respectively, and the backbone network is optimized by triplet loss and ID loss on global features and local features, respectively, where ID loss is cross-entropy loss without label smoothing. 8. The clothing-changing pedestrian re-identification device guided by black clothes and head images according to claim 5, wherein the original branch training module is specifically used for: According to the knowledge distillation algorithm, the branch of the original image is trained under the guidance of the pre-trained black branch, and the mean square error loss is used to standardize the training of the original branch to train more features related to identity but not related to clothes.

Images