CN114580492B - Cross-domain pedestrian re-recognition method based on mutual learning - Google Patents

Cross-domain pedestrian re-recognition method based on mutual learning Download PDF

Info

Publication number
CN114580492B
CN114580492B CN202111468957.5A CN202111468957A CN114580492B CN 114580492 B CN114580492 B CN 114580492B CN 202111468957 A CN202111468957 A CN 202111468957A CN 114580492 B CN114580492 B CN 114580492B
Authority
CN
China
Prior art keywords
samples
model
training
target domain
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111468957.5A
Other languages
Chinese (zh)
Other versions
CN114580492A (en
Inventor
周忠
吴威
刁其帅
张磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202111468957.5A priority Critical patent/CN114580492B/en
Publication of CN114580492A publication Critical patent/CN114580492A/en
Application granted granted Critical
Publication of CN114580492B publication Critical patent/CN114580492B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24147Distances to closest patterns, e.g. nearest neighbour classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a cross-domain pedestrian re-identification method based on mutual learning, which comprises two parts of target domain information mining based on mutual learning and training strategies based on mutual learning, wherein the target domain information mining step by utilizing the mutual learning is as follows: (1) Training by using a source domain and a target domain to obtain two pre-training models; (2) Extracting features by using the two models, and excavating neighbor samples of pedestrians in the target domain; (3) generating a pseudo tag by Jaccard distance; the training strategy based on mutual learning comprises the following steps: (1) Each pre-training model selects samples for the peer-to-peer model for training; (2) Defining the isolation of the samples by utilizing KL divergence and selecting the samples; (3) calculating a rank matrix of the selected samples by utilizing KL divergence; (4) Each model is updated through a triplet constructed by the peer-to-peer model; (5) And (5) utilizing the updated two models to carry out target domain information mining, updating the pseudo tag, and carrying out training on the neural network again to finish pedestrian re-identification.

Description

一种基于互学习的跨域行人重识别方法A cross-domain person re-identification method based on mutual learning

技术领域Technical Field

本发明属于计算机视觉技术领域,具体设计了一种基于互学习的跨域行人重识别方法,可以用于安防监控、视频分析等场景。The present invention belongs to the field of computer vision technology, and specifically designs a cross-domain pedestrian re-identification method based on mutual learning, which can be used in scenarios such as security monitoring and video analysis.

背景技术Background technique

近年来,随着监控摄像头遍布世界各个角落,监控网络的覆盖面积正在全面提高,激增的海量数据进一步暴露了传统人工安防分析在时间效率和准确率上的缺陷。传统的安防分析系统主要依赖人工观看监控内容来发现问题,一方面由于大量视频无法及时快速查看,错过黄金应急时间,另一方面准确率不高。自从2014年深度学习引入到行人重识别领域中后,依靠大量标注的数据集和开源的网络结构,其准确率得到了很大的提高。但当在已有标注的数据集上进行训练的模型应用于新的无手工标注的场景时,其性能会大幅下降,这主要是由于不同场景的光照、背景和摄像头角度等导致的数据分布不一致所造成的。为了缓解这个问题,跨域行人重识别的主要任务便是如何将模型从源域有标注的数据中学到的知识和特征表达能力迁移到无标注的目标域数据上。由于跨域行人重识别可以给行人重识别的实际应用带来便利,因此成为工业界和学术界主要的研究方向。In recent years, as surveillance cameras are spread all over the world, the coverage of surveillance networks is improving comprehensively. The surge in massive data has further exposed the shortcomings of traditional manual security analysis in terms of time efficiency and accuracy. Traditional security analysis systems mainly rely on manual viewing of surveillance content to discover problems. On the one hand, due to the inability to quickly view a large number of videos in a timely manner, the golden emergency time is missed, and on the other hand, the accuracy is not high. Since deep learning was introduced into the field of pedestrian re-identification in 2014, its accuracy has been greatly improved by relying on a large number of labeled data sets and open source network structures. However, when the model trained on the existing labeled data set is applied to a new scene without manual annotation, its performance will drop significantly. This is mainly due to the inconsistent data distribution caused by the illumination, background and camera angle of different scenes. In order to alleviate this problem, the main task of cross-domain pedestrian re-identification is how to transfer the knowledge and feature expression capabilities learned by the model from the labeled data in the source domain to the unlabeled target domain data. Since cross-domain pedestrian re-identification can bring convenience to the practical application of pedestrian re-identification, it has become the main research direction in industry and academia.

目前无监督跨域行人重识方法受噪声伪标签、低质量样本、正负样本对距离分布差异等因素的影响,准确率相对于有监督行人重识别的准确率仍有较大差距,并且在实际场景中的应用较少,没有充分结合时间空间信息。At present, the unsupervised cross-domain pedestrian re-identification method is affected by factors such as noisy pseudo-labels, low-quality samples, and differences in the distance distribution of positive and negative samples. The accuracy is still far behind that of supervised pedestrian re-identification, and it is rarely used in actual scenarios and does not fully integrate temporal and spatial information.

发明内容Summary of the invention

本发明的技术解决问题:提供一种基于互学习的跨域行人重识别方法,有效的应用在没有行人身份标注的新场景下,有效减少训练模型中对标注的依赖,可以用于安防监控、视频分析等场景。The technology of the present invention solves the problem: providing a cross-domain pedestrian re-identification method based on mutual learning, which is effectively applied in new scenarios without pedestrian identity annotation, effectively reducing the dependence on annotation in the training model, and can be used in security monitoring, video analysis and other scenarios.

本发明的技术解决方案:针对以前方法中生成伪标签质量较低的问题,基于互学习进行目标域信息挖掘,结合两个模型的特征表达能力得到质量更高的伪标签。首先每个模型根据k-reciprocal nearest neighbor策略进行近邻挖掘获得近邻集合,这样对每个样本可获得两个近邻集合,之后对两个近邻集合取交集获得置信度更高的近邻集合。两个集合互相做差集运算后,剩下的集合中也会存在正样本近邻,对集合中的每个样本利用对等模型的特征能力再使用k- reciprocal nearest neighbor策略进行近邻挖掘获得近邻集合,通过判断此近邻集合和上述交集集合的重合度来判断是否将此样本也加入最终的近邻集合。The technical solution of the present invention is as follows: in view of the problem of low quality of pseudo labels generated in previous methods, target domain information mining is performed based on mutual learning, and pseudo labels with higher quality are obtained by combining the feature expression capabilities of two models. First, each model performs neighbor mining according to the k-reciprocal nearest neighbor strategy to obtain a neighbor set, so that two neighbor sets can be obtained for each sample, and then the intersection of the two neighbor sets is taken to obtain a neighbor set with higher confidence. After the two sets are mutually subtracted, there will be positive sample neighbors in the remaining set. For each sample in the set, the feature capabilities of the peer model are used and the k-reciprocal nearest neighbor strategy is used to perform neighbor mining to obtain a neighbor set. By judging the overlap between this neighbor set and the above intersection set, it is judged whether to add this sample to the final neighbor set.

针对伪标签作为监督信息必然存在噪声的问题,使用两个模型同时训练,每个模型为对等模型选择样本进行训练,有效缓解了误差累积的问题,防止了噪声造成模型退化的问题。对于分类损失,样本选择的标准为交叉熵损失的大小,对于三元组损失,样本选择的标准为孤立性。其中孤立性指的是是否能大概率在这个mini-batch中找到和其同身份的样本,如果概率高则其孤立性低,反之,如果概率低则其孤立性高。在此使用KL散度作为衡量样本之间身份概率分布相似度的指标,孤立性高的样本基本上与其他样本的相似度都比较低。In view of the problem that pseudo-labels are inevitably noisy when used as supervisory information, two models are used for simultaneous training. Each model selects samples for the peer model for training, which effectively alleviates the problem of error accumulation and prevents the problem of model degradation caused by noise. For classification loss, the criterion for sample selection is the size of the cross entropy loss. For triple loss, the criterion for sample selection is isolation. Isolation refers to whether it is possible to find samples with the same identity in this mini-batch with a high probability. If the probability is high, its isolation is low. On the contrary, if the probability is low, its isolation is high. Here, KL divergence is used as an indicator to measure the similarity of identity probability distribution between samples. Samples with high isolation basically have a low similarity with other samples.

本发明的一种基于互学习的跨域行人重识别方法,包括:基于互学习的目标域信息挖掘方法和基于互学习的训练策略两部分内容;A cross-domain pedestrian re-identification method based on mutual learning of the present invention includes: a target domain information mining method based on mutual learning and a training strategy based on mutual learning;

所述基于互学习的目标域信息挖掘方法步骤为:The target domain information mining method based on mutual learning includes the following steps:

(d1)利用有标注的源域数据集训练获得源域预训练模型,并通过源域预训练模型提取目标域数据特征,利用DBSCAN聚类算法生成伪标签进行训练,获得目标域预训练模型,互相称为对等模型;(d1) Use the labeled source domain dataset to train to obtain the source domain pre-training model, extract the target domain data features through the source domain pre-training model, use the DBSCAN clustering algorithm to generate pseudo labels for training, and obtain the target domain pre-training model, which are called equivalent models;

(d2)利用步骤(d1)中获得的两个预训练模型,计算得到目标域中每个行人的近邻集合;(d2) Using the two pre-trained models obtained in step (d1), calculate the neighbor set of each pedestrian in the target domain;

(d3)将步骤(d2)的近邻集合转换为Jaccard距离,进而通过DBSCAN 聚类算法生成伪标签;(d3) converting the neighbor set of step (d2) into Jaccard distance, and then generating pseudo labels through DBSCAN clustering algorithm;

所述基于互学习的训练策略步骤为:The steps of the training strategy based on mutual learning are:

(t1)步骤(d1)中的每个预训练模型分别为对等模型选择小交叉熵损失前 80%的样本,并使用这部分样本通过分类损失更新对等模型的参数;(t1) Each pre-trained model in step (d1) selects the first 80% of the samples with small cross entropy loss for the peer model, and uses this part of the samples to update the parameters of the peer model through the classification loss;

(t2)利用步骤(t1)中更新后的每个模型生成的身份概率计算各样本间的 KL散度,代表样本间的身份相似度;(t2) using the identity probabilities generated by each model updated in step (t1), the KL divergence between each sample is calculated, which represents the identity similarity between samples;

(t3)基于KL散度距离定义各样本的孤立性,步骤(t1)中每个模型为对等模型选孤立性排前80%的样本;(t3) Based on the KL divergence distance, the isolation of each sample is defined. In step (t1), each model selects the top 80% of samples in isolation for the peer model;

(t4)利用KL散度计算步骤(t3)中选择样本的rank矩阵,确定三元组的正负样本对;(t4) Using the rank matrix of the sample selected in the KL divergence calculation step (t3), determine the positive and negative sample pairs of the triplet;

(t5)步骤(t1)中每个模型通过对等模型构造的三元组,利用三元组损失进行参数更新;(t5) Each model in step (t1) updates parameters using triplet loss based on the triplet constructed by the peer model;

(t6)根据步骤(t5)中更新后的参数,重新进行目标域信息挖掘,生成更新后的伪标签,利用更新后的伪标签重新进行神经网络的训练;(t6) re-mining the target domain information according to the parameters updated in step (t5), generating updated pseudo labels, and re-training the neural network using the updated pseudo labels;

(t7)训练完成后得到特征模型,进行行人检索。(t7) After training, the feature model is obtained and pedestrian retrieval is performed.

所述步骤(d1)包括如下步骤:The step (d1) comprises the following steps:

(d1.1)利用有标注的源域数据进行训练,得到源域预训练模型;(d1.1) Use labeled source domain data for training to obtain a source domain pre-training model;

(d1.2)利用上述源域预训练模型作为特征模型,提取目标域每个样本的特征,并使用DBSCAN聚类算法生成伪标签;(d1.2) Use the above source domain pre-trained model as the feature model to extract the features of each sample in the target domain, and use the DBSCAN clustering algorithm to generate pseudo labels;

(d1.3)利用步骤(d1.2)中生成的伪标签在目标域数据集上进行初步训练,得到目标域预训练模型。(d1.3) Use the pseudo labels generated in step (d1.2) to perform preliminary training on the target domain dataset to obtain the target domain pre-training model.

所述步骤(d2)包括如下步骤:The step (d2) comprises the following steps:

(d2.1)利用步骤(d1)中获得的两个预训练模型提取目标域数据集中所有行人的特征并生成两个特征矩阵,根据两个特征矩阵和k-reciprocal nearest neighbor策略为每个行人寻找近邻样本;(d2.1) Use the two pre-trained models obtained in step (d1) to extract the features of all pedestrians in the target domain dataset and generate two feature matrices. Use the two feature matrices and the k-reciprocal nearest neighbor strategy to find the nearest neighbor samples for each pedestrian.

(d2.2)基于两个预训练模型的一致性进行近邻挖掘,得到更为置信的近邻样本集合,抛弃掉筛选后剩下的样本;(d2.2) Perform neighbor mining based on the consistency of the two pre-trained models to obtain a more confident set of neighbor samples and discard the remaining samples after screening;

(d2.3)每个预训练模型,利用对等模型的特征表达能力,对步骤(d2.2) 中放弃掉的样本集合进行进一步挖掘,获取有效的近邻样本;(d2.3) Each pre-trained model uses the feature expression ability of the peer model to further mine the sample set abandoned in step (d2.2) to obtain valid neighbor samples;

(d2.4)将步骤(d2.2)和(d2.3)挖掘得到的近邻样本合并得到最终的近邻集合。(d2.4) Merge the neighbor samples mined in steps (d2.2) and (d2.3) to obtain the final neighbor set.

所述步骤(d3)包括如下步骤:The step (d3) comprises the following steps:

(d3.1)根据步骤(d2.3)中得到的每个样本的近邻样本集合,将其转换为 Jaccard距离,得到距离矩阵;(d3.1) According to the neighbor sample set of each sample obtained in step (d2.3), convert it into Jaccard distance to obtain a distance matrix;

(d3.2)根据步骤(d3.1)中获得的距离矩阵,利用DBSCAN聚类算法为每个行人样本赋予伪标签,在此过程中,部分样本被划分为噪声样本;(d3.2) According to the distance matrix obtained in step (d3.1), a pseudo label is assigned to each pedestrian sample using the DBSCAN clustering algorithm. In this process, some samples are classified as noise samples;

(d3.3)利用KNN策略为步骤(d3.2)中的噪声样本赋予伪标签;(d3.3) Use the KNN strategy to assign pseudo labels to the noise samples in step (d3.2);

(d3.4)综合步骤(d3.2)和(d3.3)中的伪标签,得到目标域数据集最终的伪标签。(d3.4) Combine the pseudo-labels in steps (d3.2) and (d3.3) to obtain the final pseudo-labels of the target domain dataset.

本发明与现有的技术相比的优点在于:The advantages of the present invention compared with the prior art are:

(1)本发明重点是研究跨域行人重识别问题,提出了一种基于互学习的目标域信息挖掘方法,从而得到质量更高的监督信息。同时利用每个模型为对等模型进行样本选择来进行网络训练,缓解伪标签中的噪声对模型训练的影响。本发明的目的是当目标域中没有人工标注的行人身份的情况下,使用目标域信息挖掘来获得伪标签作为监督信息也可以得到具有良好特征判别能力的模型。有效缓解在人工标注的源域数据集上训练后的模型无法将特征判别能力很好迁移到无人工标注的目标域数据集上的问题。(1) The present invention focuses on studying the problem of cross-domain pedestrian re-identification and proposes a target domain information mining method based on mutual learning to obtain higher quality supervisory information. At the same time, each model is used to select samples for the peer model to perform network training, thereby alleviating the impact of noise in pseudo-labels on model training. The purpose of the present invention is to use target domain information mining to obtain pseudo-labels as supervisory information when there are no manually labeled pedestrian identities in the target domain, and a model with good feature discrimination ability can also be obtained. It effectively alleviates the problem that the model trained on the manually labeled source domain dataset cannot transfer the feature discrimination ability well to the target domain dataset without manual labeling.

(2)通过互学习进行目标域信息挖掘,想比于单模型进行目标域信息挖掘得到伪标签,本发明可以充分利用两个模型的特征表达能力,在得到质量更高的近邻集合的基础上,得到质量更高的伪标签。(2) By mutual learning, target domain information is mined. Compared with a single model that mines target domain information to obtain pseudo labels, the present invention can fully utilize the feature expression capabilities of the two models to obtain pseudo labels of higher quality on the basis of obtaining a higher quality neighbor set.

(3)通过互学习进行网络训练,每个模型为对等模型选择样本进行训练,本发明可以有效缓解伪标签中噪声的影响,缓解监督信息中的噪声在训练过程中造成模型退化的问题。(3) Through mutual learning, network training is performed. Each model selects samples for training for the peer model. The present invention can effectively alleviate the impact of noise in pseudo-labels and alleviate the problem of model degradation caused by noise in supervisory information during training.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1为本发明的方法实现流程图:FIG1 is a flow chart of the method of the present invention:

图2为本发明中的预训练过程示意图;FIG2 is a schematic diagram of the pre-training process in the present invention;

图3为本发明中的近邻样本挖掘示意图;FIG3 is a schematic diagram of neighbor sample mining in the present invention;

图4为本发明中的基于样本选择的互学习示意图。FIG4 is a schematic diagram of mutual learning based on sample selection in the present invention.

具体实施方式Detailed ways

下面将结合附图详细描述本发明的具体实施。The specific implementation of the present invention will be described in detail below with reference to the accompanying drawings.

如图1所示,本发明一种基于互学习的跨域行人重识别方法,包括如下步骤:As shown in FIG1 , the present invention provides a cross-domain person re-identification method based on mutual learning, comprising the following steps:

步骤一:基于互学习的目标域信息挖掘方法Step 1: Target domain information mining method based on mutual learning

1.1首先需要两个预训练模型来为每个行人图片生成特征向量,两个模型预训练的过程如图2所示,两个模型都是用ResNet50作为主干网络,分别在源域数据和目标域数据集上进行预训练。首先在源域数据集上训练一个初始模型,利用初始模型得到目标域数据的伪标签,利用此伪标签在目标域上初步训练一个初始模型。此步骤后可以得到两个预训练模型Ms和Mt,其中Ms代表使用源域数据进行训练的模型,相应的Mt代表使用目标域数据进行训练的模型。1.1 First, two pre-trained models are needed to generate feature vectors for each pedestrian image. The pre-training process of the two models is shown in Figure 2. Both models use ResNet50 as the backbone network and are pre-trained on the source domain data and the target domain data set respectively. First, an initial model is trained on the source domain data set, and the pseudo-labels of the target domain data are obtained using the initial model. The pseudo-labels are used to preliminarily train an initial model on the target domain. After this step, two pre-trained models Ms and Mt can be obtained, where Ms represents the model trained using the source domain data, and the corresponding Mt represents the model trained using the target domain data.

1.2在两个预训练模型的基础上,进行近邻样本挖掘,其示意图如图3所示。首先基于两个模型的特征表达能力和k-reciprocal nearest neighbor策略为每个样本生成两个近邻样本集合。公式如下所示:1.2 Based on the two pre-trained models, neighbor sample mining is performed, as shown in Figure 3. First, based on the feature expression capabilities of the two models and the k-reciprocal nearest neighbor strategy, two neighbor sample sets are generated for each sample. The formula is as follows:

其中,公式(1)中p代表probe,为需要寻找近邻的行人样本,g代表gallery,为候选的样本,N(p,k)代表样本p的最近的k个邻居集合,其中d(p,gi)< d(p,gi+1),d(p,g)代表任意两个行人p,g之间的欧式距离,|·|代表集合的模长。In formula (1), p represents probe, which is the pedestrian sample for which the nearest neighbor needs to be found; g represents gallery, which is the candidate sample; N(p, k) represents the set of the nearest k neighbors of sample p, where d(p, gi ) < d(p, gi+1 ), d(p, g) represents the Euclidean distance between any two pedestrians p and g, and |·| represents the modulus of the set.

之后,基于模型的一致性进行近邻挖掘,公式如下所示:Afterwards, neighbor mining is performed based on the consistency of the model. The formula is as follows:

Rco(p,k)=Rs(p,k)∩Rt(p,k) (2)R co (p, k) = R s (p, k) ∩ R t (p, k) (2)

其中,公式(2)中Rs(p,k)代表使用源域数据训练的特征模型为基础并以 k-reciprocal nearest neighbor策略进行近邻挖掘获得的近邻集合,类似的Rt(p,k) 则是使用目标域数据训练的特征模型进行近邻挖掘获得的近邻集合,而 Rco(p,k)就是两者取交集获得的近邻集合。In formula (2), Rs (p, k) represents the neighbor set obtained by using the feature model trained with source domain data and performing neighbor mining using the k-reciprocal nearest neighbor strategy. Similarly, Rt (p, k) is the neighbor set obtained by using the feature model trained with target domain data and performing neighbor mining. Rco (p, k) is the neighbor set obtained by taking the intersection of the two.

对Rs(p,k)和Rt(p,k)互相做差集运算后得到样本集合,对这部分样本集合中的每个样本q利用对等模型的特征能力再做一次近邻挖掘,通过判断这次的近邻样本和Rco(p,k)近邻集合的重合度,如果重合度较高,认为样本q也是p的近邻样本。公式如下所示:After performing a difference operation on R s (p, k) and R t (p, k), we get a sample set. We use the feature capability of the peer model to perform neighbor mining again on each sample q in this sample set. By judging the overlap between this neighbor sample and the neighbor set of R co (p, k), if the overlap is high, we consider that sample q is also a neighbor sample of p. The formula is as follows:

其中,公式(3)中,RT是对基于特征模型Mt挖掘出的剩余近邻集合进一步挖掘的结果,相对应RS是对基于特征模型Ms挖掘出的剩余近邻集合进一步挖掘的结果。In formula (3), RT is the result of further mining the remaining neighbor set mined based on the feature model Mt , and correspondingly, RS is the result of further mining the remaining neighbor set mined based on the feature model Ms.

1.3得到每个样本的近邻集合后,将近邻集合转化为Jaccard距离,公式如下所示:1.3 After obtaining the nearest neighbor set of each sample, convert the nearest neighbor set into Jaccard distance. The formula is as follows:

公式中d(p,gi)代表两个样本之间的欧式距离,||·||1代表L1范数,向量 dJ(p,gi)代表最终的Jaccard距离。In the formula, d(p, gi ) represents the Euclidean distance between two samples, ||·|| 1 represents the L1 norm, and the vector d J (p, g i ) represents the final Jaccard distance.

1.4得到Jaccard距离矩阵后,使用DBSCAN聚类算法进行聚类,DBSCAN 聚类算法会把一部分样本划分为噪声样本,对这部分样本使用KNN算法重新赋予伪标签。至此得到每个行人样本的伪标签。1.4 After obtaining the Jaccard distance matrix, use the DBSCAN clustering algorithm for clustering. The DBSCAN clustering algorithm will classify some samples as noise samples, and use the KNN algorithm to re-assign pseudo labels to these samples. So far, the pseudo labels of each pedestrian sample are obtained.

步骤二:基于互学习的网络训练策略Step 2: Network training strategy based on mutual learning

2.1得到伪标签后,使用伪标签作为监督信息进行网络训练,首先制定互学习下样本选择的标准。在此使用两种样本选择标准,一种为根据交叉熵损失的大小来进行样本选择,另一种根据孤立性的大小进行样本选择。其中孤立性指的是是否能大概率在这个mini-batch中找到和其同身份的样本,如果某个行人在mini-batch中能找到多个相同身份的行人样本,则称其孤立性低,反之,如果能找到的相同身份的行人样本数量少甚至找不到相同身份的行人样本则其孤立性高。在此使用KL散度作为衡量样本之间身份概率分布相似度的指标,公式如下所示:2.1 After obtaining the pseudo-labels, use the pseudo-labels as supervision information for network training. First, formulate the standard for sample selection under mutual learning. Two sample selection criteria are used here, one is to select samples based on the size of the cross entropy loss, and the other is to select samples based on the size of the isolation. Isolation refers to whether it is possible to find samples with the same identity in this mini-batch with a high probability. If a pedestrian can find multiple pedestrian samples with the same identity in the mini-batch, its isolation is low. On the contrary, if the number of pedestrian samples with the same identity that can be found is small or even no pedestrian samples with the same identity can be found, its isolation is high. KL divergence is used here as an indicator to measure the similarity of identity probability distribution between samples. The formula is as follows:

其中公式(5)中,qi代表mini-batch外除了p之外的其他样本。In formula (5), qi represents the other samples outside the mini-batch except p.

2.2根据样本选择标准进行样本选择,这部分样本用于分类损失,根据交叉熵损失大小获得的样本集合如下所示:2.2 Sample selection is performed according to the sample selection criteria. This part of the samples is used for classification loss. The sample set obtained according to the size of the cross entropy loss is as follows:

Dce=arg minD′:|D′|=α|D|Lce(N,D′) (6) Dce = argmin D′: |D′| = α|D| Lce (N, D′) (6)

公式(6)中N代表网络,D′代表样本集合,Lce(N,D′)代表当前样本集合的损失总和。根据孤立性大小获得的样本集合如下所示:In formula (6), N represents the network, D′ represents the sample set, and L ce (N, D′) represents the total loss of the current sample set. The sample set obtained according to the isolation size is as follows:

Diso=arg minD′:|D′|=α|D′|Liso(N,D′) (7)D iso = arg min D′: |D′| = α|D′| L iso (N, D′) (7)

公式(7)中Liso(N,D′)代表当前样本集合的孤立性综合。In formula (7), Liso (N, D′) represents the isolated synthesis of the current sample set.

结合两种样本选择的标准,得到最后的样本集合,公式如下所示:Combining the two sample selection criteria, we get the final sample set. The formula is as follows:

Dinter={p|p∈Dce}∩{p|p∈Diso} (8) Dinter ={p| p∈Dce }∩{p| p∈Diso } (8)

公式(8)中,Dinter就是对原先初始通过两种样本选择标准选择的样本再取交集,获得噪声更低的可用于训练的样本集合。In formula (8), Dinter is the intersection of the samples originally selected by the two sample selection criteria to obtain a sample set with lower noise that can be used for training.

定义分类器为C:f→{c1,c2,...,cn},最后分类损失如下所示:Define the classifier as C:f→{ c1 , c2 , ..., cn }, and the final classification loss is as follows:

公式(9)中代表i样本在j类别的真实概率值,/>代表i样本在j类别的网络预测的概率值,/>是依据Mt选择的样本,而/>是用于更新模型Ms的损失函数,相应的/>是用来更新模型Mt的损失函数。每个模型都是用对等模型挑选的样本最后进行训练和模型的更新的。In formula (9), Represents the true probability value of sample i in category j,/> Represents the probability value of the network prediction of sample i in category j,/> is the sample selected based on M t , and /> is the loss function used to update the model Ms , corresponding to/> It is the loss function used to update the model M t . Each model is finally trained and updated using samples selected by the peer model.

2.3根据孤立性的样本选择标准进行样本选择,构建三元组对用于三元组损失函数。最终的三元组损失函数公式如下所示:2.3 Select samples based on the isolated sample selection criteria and construct triple pairs for the triple loss function. The final triple loss function formula is as follows:

公式(10)中,a代表当前样本,也是构建三元组中的锚点样本,da,p代表锚点样本和正样本的特征距离,正样本指和某点样本行人身份一致的样本,da,n代表锚点样本和负样本的特征距离,负样本指和锚点样本行人身份不一致的样本,而在所有正样本对中选择相似性距离最大的最难正样本对,即特征距离最远的样本对,在所有负样本对中选择最难的负样本对,即特征距离最近的样本对,最后组成三元组。是Mt依据孤立性原则选择的样本集合,/>是用于更新Ms所用的损失函数,相应的/>是用来更新Mt所用的损失函数。In formula (10), a represents the current sample, which is also the anchor sample in constructing the triplet. d a,p represents the feature distance between the anchor sample and the positive sample. The positive sample refers to the sample with the same pedestrian identity as the point sample. d a,n represents the feature distance between the anchor sample and the negative sample. The negative sample refers to the sample with inconsistent pedestrian identity with the anchor sample. The most difficult positive sample pair with the largest similarity distance is selected from all positive sample pairs, that is, the sample pair with the farthest feature distance. The most difficult negative sample pair is selected from all negative sample pairs, that is, the sample pair with the closest feature distance, and finally a triplet is formed. is the sample set selected by Mt according to the isolation principle,/> is the loss function used to update Ms , corresponding to/> It is the loss function used to update Mt.

2.4得到相应的损失函数后,使用损失函数进行模型训练和模型参数的更新,基于样本选择的互学习的思想如图4所示,同时训练一段时间后,根据最新的模型进行一次伪标签的更新。图4主要展示了分类损失和三元组损失进行互学习训练的过程。模型A和模型B由于不同的预训练策略,其天然具备不同的特征判别能力,每个模型都为对等模型挑选样本进行训练,从而使每个模型缓解使用具有错误伪标签的样本进行训练的负面影响。2.4 After obtaining the corresponding loss function, the loss function is used to train the model and update the model parameters. The idea of mutual learning based on sample selection is shown in Figure 4. After training for a period of time, the pseudo label is updated according to the latest model. Figure 4 mainly shows the process of mutual learning training with classification loss and triple loss. Model A and Model B have different feature discrimination capabilities due to different pre-training strategies. Each model selects samples for training for the peer model, so that each model can alleviate the negative impact of training with samples with incorrect pseudo labels.

2.5训练完成后,得到两个特征模型,此两个特征模型的特征判别能力基本一致,可任选一个模型用于行人重识别。2.5 After the training is completed, two feature models are obtained. The feature discrimination capabilities of these two feature models are basically the same, and either model can be selected for pedestrian re-identification.

Claims (3)

1.一种基于互学习的跨域行人重识别方法,其特征在于,包括:基于互学习的目标域信息挖掘方法和基于互学习的训练策略两部分内容;1. A cross-domain person re-identification method based on mutual learning, characterized by comprising: a target domain information mining method based on mutual learning and a training strategy based on mutual learning; 所述基于互学习的目标域信息挖掘方法步骤为:The target domain information mining method based on mutual learning includes the following steps: (d1)利用有标注的源域数据集训练获得源域预训练模型,并通过源域预训练模型提取目标域数据特征,利用DBSCAN聚类算法生成伪标签进行训练,获得目标域预训练模型,互相称为对等模型;(d1) Use the labeled source domain dataset to train to obtain the source domain pre-trained model, extract the target domain data features through the source domain pre-trained model, use the DBSCAN clustering algorithm to generate pseudo labels for training, and obtain the target domain pre-trained model, which are called equivalent models to each other; (d2)利用步骤(d1)中获得的两个预训练模型,计算得到目标域中每个行人的近邻集合;(d2) Using the two pre-trained models obtained in step (d1), calculate the neighbor set of each pedestrian in the target domain; (d3)将步骤(d2)的近邻集合转换为Jaccard距离,进而通过DBSCAN聚类算法生成伪标签;(d3) Convert the neighbor set of step (d2) into Jaccard distance, and then generate pseudo labels through DBSCAN clustering algorithm; 所述基于互学习的训练策略步骤为:The steps of the training strategy based on mutual learning are: (t1)步骤(d1)中的每个预训练模型分别为对等模型选择小交叉熵损失前80%的样本,并使用这部分样本通过分类损失更新对等模型的参数;(t1) Each pre-trained model in step (d1) selects the top 80% of samples with small cross entropy loss for the peer model, and uses this part of samples to update the parameters of the peer model through classification loss; (t2)利用步骤(t1)中更新后的每个模型生成的身份概率计算各样本间的KL散度,代表样本间的身份相似度;(t2) Using the identity probabilities generated by each model updated in step (t1), the KL divergence between samples is calculated, which represents the identity similarity between samples; (t3)基于KL散度距离定义各样本的孤立性,步骤(t1)中每个模型为对等模型选孤立性排前80%的样本;(t3) Define the isolation of each sample based on the KL divergence distance. In step (t1), each model selects the top 80% of samples in isolation for the peer model. (t4)利用KL散度计算步骤(t3)中选择样本的rank矩阵,确定三元组的正负样本对;(t4) Using the rank matrix of the sample selected in the KL divergence calculation step (t3), determine the positive and negative sample pairs of the triplet; (t5)步骤(t1)中每个模型通过对等模型构造的三元组,利用三元组损失进行参数更新;(t5) Each model in step (t1) updates parameters using triplet loss based on the triplet constructed by the peer model; (t6)根据步骤(t5)中更新后的参数,重新进行目标域信息挖掘,生成更新后的伪标签,利用更新后的伪标签重新进行神经网络的训练;(t6) re-mining the target domain information according to the parameters updated in step (t5), generating updated pseudo labels, and re-training the neural network using the updated pseudo labels; (t7)训练完成后得到特征模型,进行行人检索;(t7) After training, the feature model is obtained and pedestrian retrieval is performed; 所述步骤(d2)包括如下步骤:The step (d2) comprises the following steps: (d2.1)利用步骤(d1)中获得的两个预训练模型提取目标域数据集中所有行人的特征并生成两个特征矩阵,根据两个特征矩阵和k-reciprocal nearest neighbor策略为每个行人寻找近邻样本;(d2.1) Use the two pre-trained models obtained in step (d1) to extract the features of all pedestrians in the target domain dataset and generate two feature matrices. Use the two feature matrices and the k-reciprocal nearest neighbor strategy to find the nearest neighbor samples for each pedestrian. (d2.2)基于两个预训练模型的一致性进行近邻挖掘,综合两个预训练模型共同筛选出更为置信的近邻样本集合,抛弃掉筛选后剩下的样本;(d2.2) Perform neighbor mining based on the consistency of the two pre-trained models, combine the two pre-trained models to jointly select a more reliable set of neighbor samples, and discard the remaining samples after the screening; (d2.3)每个预训练模型,利用对等模型的特征表达能力,对步骤(d2.2)中放弃掉的样本集合进行进一步挖掘,获取有效的近邻样本;(d2.3) Each pre-trained model uses the feature expression ability of the peer model to further mine the sample set abandoned in step (d2.2) to obtain valid neighbor samples; (d2.4)将步骤(d2.2)和(d2.3)挖掘得到的近邻样本合并得到最终的近邻集合。(d2.4) Merge the neighbor samples mined in steps (d2.2) and (d2.3) to obtain the final neighbor set. 2.根据权利要求1所述的基于互学习的跨域行人重识别方法,其特征在于:所述步骤(d1)包括如下步骤:2. The cross-domain person re-identification method based on mutual learning according to claim 1, characterized in that: the step (d1) comprises the following steps: (d1.1)利用有标注的源域数据进行训练,得到源域预训练模型;(d1.1) Use the labeled source domain data for training to obtain the source domain pre-training model; (d1.2)利用上述源域预训练模型作为特征模型,提取目标域每个样本的特征,并使用DBSCAN聚类算法生成伪标签;(d1.2) Use the above source domain pre-trained model as the feature model to extract the features of each sample in the target domain, and use the DBSCAN clustering algorithm to generate pseudo labels; (d1.3)利用步骤(d1.2)中生成的伪标签在目标域数据集上进行初步训练,得到目标域预训练模型。(d1.3) Use the pseudo labels generated in step (d1.2) to perform preliminary training on the target domain dataset to obtain the target domain pre-trained model. 3.根据权利要求1所述的基于互学习的跨域行人重识别方法,其特征在于:所述步骤(d3)包括如下步骤:3. The cross-domain person re-identification method based on mutual learning according to claim 1, characterized in that: the step (d3) comprises the following steps: (d3.1)根据步骤(d2.3)中得到的每个样本的近邻样本集合,将其转换为Jaccard距离,得到距离矩阵;(d3.1) According to the set of neighboring samples of each sample obtained in step (d2.3), convert it into Jaccard distance to obtain a distance matrix; (d3.2)根据步骤(d3.1)中获得的距离矩阵,利用DBSCAN聚类算法为每个行人样本赋予伪标签,在此过程中,部分样本被划分为噪声样本;(d3.2) Based on the distance matrix obtained in step (d3.1), a pseudo-label is assigned to each pedestrian sample using the DBSCAN clustering algorithm. In this process, some samples are classified as noise samples. (d3.3)利用KNN策略为步骤(d3.2)中的噪声样本赋予伪标签;(d3.3) Use the KNN strategy to assign pseudo labels to the noise samples in step (d3.2); (d3.4)综合步骤(d3.2)和(d3.3)中的伪标签,得到目标域数据集最终的伪标签。(d3.4) Combine the pseudo-labels in steps (d3.2) and (d3.3) to obtain the final pseudo-labels of the target domain dataset.
CN202111468957.5A 2021-12-03 2021-12-03 Cross-domain pedestrian re-recognition method based on mutual learning Active CN114580492B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111468957.5A CN114580492B (en) 2021-12-03 2021-12-03 Cross-domain pedestrian re-recognition method based on mutual learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111468957.5A CN114580492B (en) 2021-12-03 2021-12-03 Cross-domain pedestrian re-recognition method based on mutual learning

Publications (2)

Publication Number Publication Date
CN114580492A CN114580492A (en) 2022-06-03
CN114580492B true CN114580492B (en) 2024-06-21

Family

ID=81772136

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111468957.5A Active CN114580492B (en) 2021-12-03 2021-12-03 Cross-domain pedestrian re-recognition method based on mutual learning

Country Status (1)

Country Link
CN (1) CN114580492B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116246349B (en) * 2023-05-06 2023-08-15 山东科技大学 Single-source domain generalization gait recognition method based on progressive subdomain mining

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110008842A (en) * 2019-03-09 2019-07-12 同济大学 A kind of pedestrian's recognition methods again for more losing Fusion Model based on depth
CN110866536B (en) * 2019-09-25 2022-06-07 西安交通大学 A cross-regional enterprise tax evasion identification method based on PU learning
CN111967294B (en) * 2020-06-23 2022-05-20 南昌大学 An Unsupervised Domain Adaptive Person Re-identification Approach
CN111860678B (en) * 2020-07-29 2024-02-27 中国矿业大学 Unsupervised cross-domain pedestrian re-identification method based on clustering

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于图卷积神经网络的跨域行人再识别;潘少明;王玉杰;种衍文;;华中科技大学学报(自然科学版);20201231(第09期);全文 *
弱监督场景下的行人重识别研究综述;祁磊;于沛泽;高阳;;软件学报;20200915(第09期);全文 *

Also Published As

Publication number Publication date
CN114580492A (en) 2022-06-03

Similar Documents

Publication Publication Date Title
CN111860678B (en) Unsupervised cross-domain pedestrian re-identification method based on clustering
Wu et al. Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning
Mancini et al. Adagraph: Unifying predictive and continuous domain adaptation through graphs
CN112819065B (en) Unsupervised pedestrian sample mining method and unsupervised pedestrian sample mining system based on multi-clustering information
WO2022160772A1 (en) Person re-identification method based on view angle guidance multi-adversarial attention
Bai et al. Hierarchical connectivity-centered clustering for unsupervised domain adaptation on person re-identification
Guo et al. Density-aware feature embedding for face clustering
CN109948561A (en) Method and system for unsupervised image and video person re-identification based on transfer network
CN109902612B (en) Monitoring video abnormity detection method based on unsupervised learning
CN111797814A (en) Unsupervised cross-domain action recognition method based on channel fusion and classifier confrontation
CN111639540A (en) Semi-supervised character re-recognition method based on camera style and human body posture adaptation
CN112906606A (en) Domain-adaptive pedestrian re-identification method based on mutual divergence learning
CN112766237A (en) Unsupervised pedestrian re-identification method based on cluster feature point clustering
Han et al. DMRNet++: Learning discriminative features with decoupled networks and enriched pairs for one-step person search
CN112115780A (en) Semi-supervised pedestrian re-identification method based on deep multi-model cooperation
CN107977610A (en) A kind of face tracking methods based on massive video processing
CN115293217A (en) Unsupervised pseudo tag optimization pedestrian re-identification method based on radio frequency signals
CN111680644A (en) A video behavior clustering method based on deep spatiotemporal feature learning
CN114580492B (en) Cross-domain pedestrian re-recognition method based on mutual learning
CN115984901A (en) Multi-mode-based graph convolution neural network pedestrian re-identification method
Zhao et al. Heterogeneous pseudo-supervised learning for few-shot person re-identification
Yang et al. Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?
Gong et al. Contrastive Mean teacher for intra-camera supervised person re-identification
Liu et al. Improving Consistency of Proxy-Level Contrastive Learning for Unsupervised Person Re-Identification
Casagrande et al. Abnormal motion analysis for tracking-based approaches using region-based method with mobile grid

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant