CN111291611A - Pedestrian re-identification method and device based on Bayesian query expansion - Google Patents

Pedestrian re-identification method and device based on Bayesian query expansion Download PDF

Info

Publication number
CN111291611A
CN111291611A CN201911325604.2A CN201911325604A CN111291611A CN 111291611 A CN111291611 A CN 111291611A CN 201911325604 A CN201911325604 A CN 201911325604A CN 111291611 A CN111291611 A CN 111291611A
Authority
CN
China
Prior art keywords
query
pedestrian
query expansion
bayesian
expansion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911325604.2A
Other languages
Chinese (zh)
Inventor
王景辉
张斯尧
罗茜
王思远
蒋杰
张�诚
李乾
谢喜林
黄晋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changsha Qianshitong Intelligent Technology Co ltd
Original Assignee
Changsha Qianshitong Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changsha Qianshitong Intelligent Technology Co ltd filed Critical Changsha Qianshitong Intelligent Technology Co ltd
Priority to CN201911325604.2A priority Critical patent/CN111291611A/en
Publication of CN111291611A publication Critical patent/CN111291611A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/29Graphical models, e.g. Bayesian networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Library & Information Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pedestrian re-identification method and a device based on Bayesian query expansion, wherein the method comprises the following steps: inputting the query object into a trained pedestrian re-recognition system model to obtain a ranking list of a plurality of candidate objects; re-identifying and re-ordering the query object and a plurality of candidate objects in the ranking list based on Bayesian query expansion; performing PTGAN processing on the re-ordered query objects and candidate objects to realize the migration of a background difference area on the premise of keeping the foreground of the pedestrian unchanged; inputting the query object and the candidate object subjected to the PTGAN processing into a trained Bayesian model; adjusting parameter values of target parameters of the pedestrian re-identification system model according to the real matching probability of each candidate object; and searching out the pedestrian image with the highest similarity through the pedestrian re-identification system model. The invention solves the problems of high cross-camera retrieval difficulty and low re-identification accuracy of the pedestrian re-identification method in the prior art.

Description

Pedestrian re-identification method and device based on Bayesian query expansion
Technical Field
The invention relates to the technical field of computer vision and smart cities, in particular to a pedestrian re-identification method and device based on Bayesian query expansion, a terminal device and a computer readable medium.
Background
With the continuous development of artificial intelligence, computer vision and hardware technology, video image processing technology has been widely applied to intelligent city systems.
Pedestrian Re-identification (Person Re-identification) is also called pedestrian Re-identification, abbreviated Re-ID. The method is a technology for judging whether a specific pedestrian exists in an image or a video sequence by utilizing a computer vision technology. Is widely considered as a sub-problem for image retrieval. Given a monitored pedestrian image, the pedestrian image is retrieved across the device. Due to the difference between different camera devices, pedestrians have the characteristics of rigidity and flexibility, and the appearance is easily influenced by wearing, size, shielding, posture, visual angle and the like, so that the pedestrian re-identification becomes a hot topic which has research value and is very challenging in the field of computer vision.
At present, although the detection capability of pedestrian re-identification has been significantly improved, many of the problems with warfare in practical situations have not been completely solved: such as in complex scenes, differences in light, changes in perspective and pose, a large number of pedestrians in a surveillance camera network, etc. Under the conditions, the cross-camera retrieval is difficult generally, meanwhile, the marking work in the early stage of video image sample training is expensive, a large amount of manpower is consumed, the existing algorithm cannot achieve the expected effect generally, and the re-recognition accuracy is low.
Disclosure of Invention
In view of the above, the present invention provides a pedestrian re-identification method, apparatus, terminal device and computer readable medium based on bayesian query expansion, which can improve the accuracy of pedestrian re-identification under different cameras, and solve the problems of difficult cross-camera search and low re-identification accuracy of the pedestrian re-identification method in the prior art.
The first aspect of the embodiment of the invention provides a pedestrian re-identification method based on Bayesian query expansion, which comprises the following steps:
inputting the query object into a trained pedestrian re-recognition system model to obtain a ranking list of a plurality of candidate objects;
re-identifying and re-ordering the query object and a plurality of candidate objects in the ranking list based on Bayesian query expansion;
performing PTGAN processing on the re-ordered query objects and candidate objects to realize the migration of a background difference area on the premise of keeping the foreground of the pedestrian unchanged;
inputting the query object and the candidate object subjected to the PTGAN processing into a trained Bayesian model, and calculating the real matching probability of each candidate object according to the image distance in the training data;
adjusting parameter values of target parameters of the pedestrian re-identification system model according to the real matching probability of each candidate object;
and searching out the pedestrian image with the highest similarity by inputting the image to be recognized into the adjusted pedestrian re-recognition system model.
Further, performing bayesian query expansion-based re-identification reordering on the query object and the plurality of candidate objects in the ranking list, comprising:
training a Bayes model by using a training set containing true matching and false matching pedestrian pairs to obtain a trained Bayes model;
according to the distance between the query object and a plurality of candidate object images, predicting the real matching probability of each candidate object through the trained Bayesian model;
and performing query expansion according to the real matching probability of each candidate object, and generating a new ranking list through the query expansion.
Further, the query expansion is performed according to the true matching probability of each candidate object, and a new ranking list is generated through the query expansion, including:
fusing features of the query image and the top candidate object using an average pool;
and performing query expansion according to the features of the query image and the top candidate object after the average pool is fused, and generating a new ranking list through the query expansion.
Further, the query expansion is performed according to the true matching probability of each candidate object, and a new ranking list is generated through the query expansion, including:
fusing features of the query image and the top candidate object using the maximum pool;
and performing query expansion according to the features of the query image and the top candidate object after the maximum pool fusion, and generating a new ranking list through the query expansion.
A second aspect of the embodiments of the present invention provides a pedestrian re-identification apparatus based on bayesian query expansion, including:
the ranking list acquisition module is used for inputting the query object into the trained pedestrian re-recognition system model to obtain a ranking list of a plurality of candidate objects;
the re-identification module is used for re-identifying and re-ordering the query object and a plurality of candidate objects in the ranking list based on Bayesian query expansion;
the PTGAN processing module is used for carrying out PTGAN processing on the reordered query objects and candidate objects to realize the migration of a background difference area on the premise of keeping the foreground of the pedestrian unchanged;
the training module is used for inputting the query object and the candidate object subjected to the PTGAN processing into a trained Bayesian model, and calculating the real matching probability of each candidate object according to the image distance in the training data;
the adjusting module is used for adjusting the parameter value of the target parameter of the pedestrian re-identification system model according to the real matching probability of each candidate object;
and the identification module is used for searching out the pedestrian image with the highest similarity by inputting the image to be identified into the adjusted pedestrian re-identification system model.
Further, the re-identification module comprises:
the Bayes training module is used for training a Bayes model by using a training set containing true matching and false matching pedestrian pairs to obtain a trained Bayes model;
the prediction module is used for predicting the real matching probability of each candidate object through the trained Bayesian model according to the distance between the query object and the plurality of candidate object images;
and the query expansion module is used for performing query expansion according to the real matching probability of each candidate object and generating a new ranking list through the query expansion.
Further, the query expansion module comprises:
an average pool fusion module for fusing features of the query image and the top candidate object using the average pool;
and the first expansion updating module is used for performing query expansion according to the features of the query image and the top candidate object after the average pool is fused, and generating a new ranking list through the query expansion.
Further, the query expansion module comprises:
a maximum pool fusion module for fusing features of the query image and the top candidate object using the maximum pool;
and the second expansion updating module is used for performing query expansion according to the features of the query image and the top candidate object after the maximum pool fusion, and generating a new ranking list through the query expansion.
A third aspect of the embodiments of the present invention provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the pedestrian re-identification method based on bayesian query expansion when executing the computer program.
A fourth aspect of the embodiments of the present invention provides a computer-readable medium, which stores a computer program, and when the computer program is processed and executed, the computer program implements the steps of the pedestrian re-identification method based on the bayesian query expansion.
In the embodiment of the invention, the pedestrian re-identification system model is adjusted through re-identification and re-sequencing based on Bayesian query expansion, so that the accuracy of pedestrian re-identification under complex conditions is improved, and the robustness of the system is improved. The pedestrian re-identification method solves the problems that the cross-camera retrieval difficulty is high and the re-identification accuracy rate is low in the pedestrian re-identification method in the prior art.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the embodiments or drawings used in the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
Fig. 1 is a flowchart of a pedestrian re-identification method based on bayesian query expansion according to an embodiment of the present invention;
FIG. 2 is a comparison graph of real-time conversion effects of different pedestrian re-identification methods provided by the embodiment of the invention;
fig. 3 is a schematic structural diagram of a pedestrian re-identification apparatus based on bayesian query expansion according to an embodiment of the present invention;
FIG. 4 is a detailed structure diagram of a re-recognition module provided in an embodiment of the present invention;
fig. 5 is a schematic diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Referring to fig. 1, fig. 1 is a flowchart of a pedestrian re-identification method based on bayesian query expansion according to an embodiment of the present invention. As shown in fig. 1, the pedestrian re-identification method based on bayesian query expansion of the present embodiment includes the following steps:
step S102, inputting the query object into a trained pedestrian re-recognition system model to obtain a ranking list of a plurality of candidate objects;
the pedestrian re-recognition system model can be any existing MA-CNN or RNN and other pedestrian re-recognition network models, and is trained in advance through a training set containing real matching and false matching pedestrian pairs.
Step S104, performing re-identification and re-ordering on the query object and a plurality of candidate objects in the ranking list based on Bayesian query expansion;
further, the re-identification reordering based on Bayesian query expansion is carried out on the query object and a plurality of candidate objects in the ranking list, and comprises the following steps:
step 1, training a Bayes model by using a training set containing true matching and false matching pedestrian pairs to obtain a trained Bayes model;
step 2, predicting the real matching probability of each candidate object through a trained Bayes model according to the distance between the query object and the plurality of candidate object images;
and 3, performing query expansion according to the real matching probability of each candidate object, and generating a new ranking list through the query expansion.
Re-identification re-ranking based on Bayesian Query Expansion (BQE) is performed on a database of trusted images for which model training is required, the BQE generating a new query using information from the initial gallery rank list for re-retrieval of gallery images.
Specifically, the data set is divided into three parts, query, gallery and training data. In the off-line process, Bayesian posterior estimation training is firstly carried out on training data. Given the distance metric, the bayesian model can predict the true match probability of the candidate object. During online retrieval, an initial rank list can be obtained by calculating the similarity between the query and the gallery image. And calculating the real matching probability of each candidate object by using a Bayesian model according to the sorting table. Then, the image features in the initial rank list with high probability are merged with the original query, thereby initiating a new query to perform another round of retrieval. After a new round of retrieval, the query expansion process can be reused, resulting in an iterative algorithm. Formally, each image is represented by a d-dimensional feature vector, expressed as x ∈ Rd. Order to
Figure BDA0002328309750000051
In order to be a training set, the training set,
Figure BDA0002328309750000052
is a gallery set. Then, training images are recorded
Figure BDA0002328309750000053
Query image q and gallery images
Figure BDA0002328309750000054
Are respectively represented as
Figure BDA0002328309750000055
lqAnd
Figure BDA0002328309750000056
is provided with
Figure BDA0002328309750000057
For query image q and gallery images
Figure BDA0002328309750000058
The distance between them. The initial rank list is then expressed as
Figure BDA0002328309750000059
Wherein
Figure BDA00023283097500000510
Thus, the initial rank list may be reordered based on an offline trained bayesian model.
In essence, the Bayesian model represents a match score distribution of true matches and false matches. The model is created on a training set and applied during testing to estimate the probability that the top image really matches the query.
For each image x in the ranked list returned from the pedestrian re-identification system, there is a distance calculated from the learning metric. Since images at smaller distance from the query will be listed at the top, it is necessary to know whether the image list can be reordered using the top image to improve performance. It is important to select candidate images because a false match will adversely affect performance. Given the distance P (x | d (x, q)) between the query and the image, the present invention distinguishes candidates by distance, since images of the same or dissimilar features typically have significantly different distance ranges. The invention adopts a Bayesian model to estimate the probability of the image correlation in the sorting table.
In particular, for query q and gallery image x, based on the distance between the two images, the probability that the two images belong to the same feature is typically computed, i.e.
Figure BDA00023283097500000511
According to Bayes' theorem, the probability can be rewritten as follows:
Figure BDA00023283097500000512
wherein
Figure BDA00023283097500000513
Can be calculated by the following formula,
Figure BDA00023283097500000514
the present invention utilizes training data to estimate probabilities. Can be used directly
Figure BDA00023283097500000515
And
Figure BDA00023283097500000516
to calculate
Figure BDA00023283097500000517
And
Figure BDA00023283097500000518
an approximation of (d). To calculate
Figure BDA00023283097500000519
And
Figure BDA00023283097500000520
it is necessary to calculate the distance between each image in the training data and use the distance range instead of the exact value of the distance. The invention relates to the distance range (
Figure BDA00023283097500000521
Value) into M intervals and then calculating the number of candidates in each interval. Suppose that
Figure BDA0002328309750000061
In the [0.2,0.3 ]]Within this interval, then
Figure BDA0002328309750000062
May be calculated by dividing the candidate number by the frequency of the red bars in the interval.
Figure BDA0002328309750000063
The calculation method of (2) is similar. The number of intervals M is selected according to the size of the data set. In practice, if the distance of the test phase is greater than the upper limit (or less than the lower limit) of the training phase, the result of the upper limit (or lower limit) is used.
For query expansion, a new query will be raised to reorder the candidates. Only K high probability candidates are merged into a new query, where K is less than or equal to the number of true matches and K < n. The value of K is adjusted according to actual conditions. The strategy of feature pooling is diverse.
There are two simple strategies for query expansion: average Query Expansion (AQE) and Maximum Query Expansion (MQE). For both methods, the features of the query image and the top candidate are fused using the average pool and the maximum pool, respectively. For AQE, the extended query is calculated as:
Figure BDA0002328309750000064
a disadvantage of these strategies is that their effectiveness depends to a large extent on the quality of the initial sorting table and the value of the parameter k. When the initial ranking table is not satisfied or the value of k is large, a new query will be constructed using false matches, which will affect the accuracy of the query.
To overcome this deficiency, the present invention assigns different weights to each candidate in feature pooling. Then, an extended probe q for the initial query q is computed by combining the first K images and the query q into one with the probabilitynew. In this context, the invention simply uses an average pool with weights, where the weights are probabilities. The formula is as follows:
Figure BDA0002328309750000065
finally, the distance is calculated using this new query and the initial ranking list is rearranged. The result is then subjected to more iterations. Typically, the expanded query will produce a better ranked list, and thus a better query may be produced. The present invention may repeatedly perform the process of generating a ranked list, a feature pool, and a query expansion. By repeating BQE, the effect will be enhanced. T is expressed as the number of iterations.
Assume that the sizes of the training set and the gallery set are M and N, respectively. Complexity for Bayesian model
Figure BDA0002328309750000066
And (4) performing off-line calculation. For the query expansion process, probabilities need to be computed and new queries constructed. The time complexity of generating a new query is
Figure BDA0002328309750000067
Where K is the number of pooled images. Since the parameter K is less than the true match number and K ≦ N, the complexity may be limited to
Figure BDA0002328309750000068
Then using the complexity
Figure BDA0002328309750000069
The pair-wise distance of the trust image is calculated. Results are obtained, for a query, with a computational complexity of
Figure BDA00023283097500000610
And step S106, carrying out PTGAN processing on the reordered query objects and candidate objects, and realizing the migration of the background difference area on the premise of keeping the foreground of the pedestrian unchanged.
Ptgan (person Transfer gan) is a generative countermeasure network aimed at Re-identifying Re-ID problems. In the invention, the largest characteristic of the PTGAN is to realize the migration of the background area difference on the premise of ensuring the unchanged foreground of the pedestrian as much as possible. First, the loss function of the PTGAN network consists of two parts:
Figure BDA0002328309750000071
wherein L isStyleRepresenting the loss of the generated style, or domain difference loss, is whether the generated image resembles a new dataset style. L isIDThe loss of identity representing the generated image is to verify that the generated image is the same person as the original image. λ there1Is a weight that balances the two losses. These two losses are defined as follows:
firstly, the Loss function (Loss) of the PTGAN is divided into two parts; the first part is LStyleThe concrete formula is as follows:
Figure BDA0002328309750000072
wherein the content of the first and second substances,
Figure BDA0002328309750000073
represents a loss of standard antagonism, LCycRepresenting a loss of periodic consistency, A, B is a two frame GAN processed image, let G be the image a to B style mapping function,
Figure BDA0002328309750000074
for the style mapping function of B to a, λ 2 is the weight of segmentation loss and identity loss.
The above parts are all normal losses of PTGAN in order to ensure that the difference area (domain) of the generated picture and the desired data set is the same.
Secondly, in order to ensure that the foreground is not changed in the process of picture migration, a foreground segmentation is firstly carried out on the video image by using the PSPNet to obtain a mask (mask layer) area. Generally speaking, the traditional generation of countermeasure networks such as CycleGAN is not used for Re-ID tasks, and therefore there is no need to ensure that the identity information of the foreground object is not changed, as a result of which the foreground may be blurredSuch as poor quality, and even worse, the appearance of the pedestrian may change. To solve this problem, the present invention proposes LIDLoss, foreground extracted by PSPNet, this foreground is a mask, and the final loss of identity information is:
Figure RE-GDA0002478469840000075
wherein, M (a) and M (b) are two divided foreground mask layers, and the identity information Loss function (Loss) can restrain the foreground of the pedestrian to keep unchanged as much as possible in the migration process.
Wherein G (a) is a pedestrian image transferred in the image a,
Figure RE-GDA0002478469840000076
is the pedestrian image, IE, shifted in the image ba~pdata(a)For data distribution of image a, IEb~pdata(b)For the data distribution of b, m (a) and m (b) are two divided mask regions.
Fig. 2 shows a comparison graph of real-time conversion effects of different pedestrian re-recognition methods, wherein the first row of pictures is pictures to be converted, and the fourth row shows the result of PTGAN conversion, and it can be seen that the image quality generated by PTGAN is higher than that of the third row of pictures using Cycle-GAN conversion results. For example, the appearance of the person remains unchanged and the style is effectively transferred to another camera. Shadows, road markings and backgrounds are automatically generated, similar to the effect of another camera. Meanwhile, PTGAN can handle the noise segmentation result generated by PSPNet well. It can be seen that the algorithm of the invention intuitively can better ensure the identity information of the pedestrian compared with the traditional annular generation countermeasure network (CycleGAN).
Step S108, inputting the query object and the candidate object subjected to the PTGAN processing into a trained Bayesian model, and calculating the real matching probability of each candidate object according to the image distance in the training data;
step S110, adjusting parameter values of target parameters of the pedestrian re-identification system model according to the real matching probability of each candidate object;
and step S112, inputting the image to be recognized into the adjusted pedestrian re-recognition system model, and searching out the pedestrian image with the highest similarity.
In the embodiment of the invention, the pedestrian re-identification system model is adjusted through re-identification and re-sequencing based on Bayesian query expansion, so that the accuracy of pedestrian re-identification under complex conditions is improved, and the robustness of the system is improved. The pedestrian re-identification method solves the problems that the cross-camera retrieval difficulty is high and the re-identification accuracy rate is low in the pedestrian re-identification method in the prior art.
Referring to fig. 3, fig. 3 is a block diagram illustrating a pedestrian re-identification apparatus based on bayesian query expansion according to an embodiment of the present invention. As shown in fig. 3, the pedestrian re-recognition 20 based on the bayesian query expansion of the present embodiment includes a ranking list obtaining module 202, a re-recognition module 204, a PTGAN processing module 206, a training module 208, an adjustment module 210, and a recognition module 212. The ranking list obtaining module 202, the re-recognition module 204, the PTGAN processing module 206, the training module 208, the adjusting module 210, and the recognition module 212 are respectively configured to perform the specific methods in S102, S104, S106, S108, S110, and S112 in fig. 1, and details can be referred to the related introduction of fig. 1 and are only briefly described here:
the ranking list acquisition module 202 is configured to input the query object into the trained pedestrian re-recognition system model to obtain a ranking list of multiple candidate objects;
a re-identification module 204, configured to perform re-identification and re-ordering on the query object and multiple candidate objects in the ranking list based on bayesian query expansion;
a PTGAN processing module 206, configured to perform PTGAN processing on the re-ordered query object and candidate object, so as to implement migration of a background difference area on the premise that a human foreground is not changed;
the training module 208 is configured to input the query object and the candidate object subjected to the PTGAN processing into a trained bayesian model, and calculate a true matching probability of each candidate object according to an image distance in training data;
an adjusting module 210, configured to adjust a parameter value of a target parameter of the pedestrian re-identification system model according to the true matching probability of each candidate object;
and the identification module 212 is used for searching out the pedestrian image with the highest similarity by inputting the image to be identified into the adjusted pedestrian re-identification system model.
Further, referring to fig. 4, the re-identification module 204 includes:
a bayesian training module 2041, configured to train a bayesian model using a training set including true matching and false matching pedestrian pairs to obtain a trained bayesian model;
a predicting module 2042, configured to predict, according to distances between the query object and multiple candidate object images, a true matching probability of each candidate object through the trained bayesian model;
the query expansion module 2042 is configured to perform query expansion according to the true matching probability of each candidate object, and generate a new ranking list through the query expansion.
Further, the query expansion module comprises:
an average pool fusion module for fusing features of the query image and the top candidate object using the average pool;
and the first expansion updating module is used for performing query expansion according to the features of the query image and the top candidate object after the average pool is fused, and generating a new ranking list through the query expansion.
Further, the query expansion module comprises:
a maximum pool fusion module for fusing features of the query image and the top candidate object using the maximum pool;
and the second expansion updating module is used for performing query expansion according to the features of the query image and the top candidate object after the maximum pool fusion, and generating a new ranking list through the query expansion.
In the embodiment of the invention, the pedestrian re-identification system model is adjusted through re-identification and re-sequencing based on Bayesian query expansion, so that the accuracy of pedestrian re-identification under complex conditions is improved, and the robustness of the system is improved. The pedestrian re-identification method solves the problems that the cross-camera retrieval difficulty is high and the re-identification accuracy rate is low in the pedestrian re-identification method in the prior art.
Fig. 5 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 5, the terminal device 10 of this embodiment includes: a processor 100, a memory 101 and a computer program 102 stored in said memory 101 and executable on said processor 100, such as a program for pedestrian re-identification based on bayesian query expansion. The processor 100, when executing the computer program 102, implements the steps in the above-described method embodiments, for example, the steps of S102, S104, S106, S108, S110, S112 shown in fig. 1. Alternatively, the processor 100, when executing the computer program 102, implements the functions of the modules/units in the above-described device embodiments, such as the functions of the ranking list obtaining module 202, the re-recognition module 204, the PTGAN processing module 206, the training module 208, the adjusting module 210, and the recognition module 212 shown in fig. 5.
Illustratively, the computer program 102 may be partitioned into one or more modules/units that are stored in the memory 101 and executed by the processor 100 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 102 in the terminal device 10. For example, the ranking list acquisition module 202, the re-recognition module 204, the PTGAN processing module 206, the training module 208, the adjustment module 210, and the recognition module 212. (modules in the virtual device), the specific functions of each module are as follows:
the ranking list acquisition module 202 is configured to input the query object into the trained pedestrian re-recognition system model to obtain a ranking list of multiple candidate objects;
a re-identification module 204, configured to perform re-identification and re-ordering on the query object and multiple candidate objects in the ranking list based on bayesian query expansion;
a PTGAN processing module 206, configured to perform PTGAN processing on the re-ordered query object and candidate object, so as to implement migration of a background difference area on the premise that a human foreground is not changed;
the training module 208 is configured to input the query object and the candidate object subjected to the PTGAN processing into a trained bayesian model, and calculate a true matching probability of each candidate object according to an image distance in training data;
an adjusting module 210, configured to adjust a parameter value of a target parameter of the pedestrian re-identification system model according to the true matching probability of each candidate object;
and the identification module 212 is used for searching out the pedestrian image with the highest similarity by inputting the image to be identified into the adjusted pedestrian re-identification system model.
The terminal device 10 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. Terminal device 10 may include, but is not limited to, a processor 100, a memory 101. Those skilled in the art will appreciate that fig. 5 is merely an example of the terminal device 10 and does not constitute a limitation of the terminal device 10 and may include more or less components than those shown, or some components may be combined, or different components, for example, the terminal device may also include an input output device, a network access device, a bus, etc.
The Processor 100 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 101 may be an internal storage unit of the terminal device 10, such as a hard disk or a memory of the terminal device 10. The memory 101 may also be an external storage device of the terminal device 10, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the terminal device 10. Further, the memory 101 may also include both an internal storage unit of the terminal device 10 and an external storage device. The memory 101 is used for storing the computer program and other programs and data required by the terminal device 10. The memory 101 may also be used to temporarily store data that has been output or is to be output.
It will be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing functional units and modules are merely illustrated in terms of their division, and in practical applications, the foregoing functional allocation may be performed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to perform all or part of the above described functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, and is not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (10)

1. A pedestrian re-identification method based on Bayesian query expansion is characterized by comprising the following steps:
inputting the query object into a trained pedestrian re-recognition system model to obtain a ranking list of a plurality of candidate objects;
re-identifying and re-ordering the query object and a plurality of candidate objects in the ranking list based on Bayesian query expansion;
performing PTGAN processing on the re-ordered query objects and candidate objects to realize the migration of a background difference area on the premise of keeping the foreground of the pedestrian unchanged;
inputting the query object and the candidate object subjected to the PTGAN processing into a trained Bayesian model, and calculating the real matching probability of each candidate object according to the image distance in the training data;
adjusting parameter values of target parameters of the pedestrian re-identification system model according to the real matching probability of each candidate object;
and searching out the pedestrian image with the highest similarity by inputting the image to be recognized into the adjusted pedestrian re-recognition system model.
2. The pedestrian re-identification method based on Bayesian query expansion as claimed in claim 1, wherein the re-identification re-ranking based on Bayesian query expansion is performed on the query object and the plurality of candidate objects in the ranking list, and comprises:
training a Bayes model by using a training set containing true matching and false matching pedestrian pairs to obtain a trained Bayes model;
predicting the real matching probability of each candidate object through the trained Bayesian model according to the distance between the query object and the plurality of candidate object images;
and performing query expansion according to the real matching probability of each candidate object, and generating a new ranking list through the query expansion.
3. The pedestrian re-identification method based on Bayesian query expansion as claimed in claim 2, wherein said query expansion is performed according to the true match probability of each candidate object, and a new ranking list is generated by the query expansion, including:
fusing features of the query image and the top candidate object using an average pool;
and performing query expansion according to the features of the query image and the top candidate object after the average pool is fused, and generating a new ranking list through the query expansion.
4. The pedestrian re-identification method based on Bayesian query expansion as claimed in claim 2, wherein said query expansion is performed according to the true match probability of each candidate object, and a new ranking list is generated by the query expansion, including:
fusing features of the query image and the top candidate object using the maximum pool;
and performing query expansion according to the features of the query image and the top candidate object after the maximum pool fusion, and generating a new ranking list through the query expansion.
5. A pedestrian re-identification device based on Bayesian query expansion is characterized by comprising the following components:
the ranking list acquisition module is used for inputting the query object into the trained pedestrian re-recognition system model to obtain a ranking list of a plurality of candidate objects;
the re-identification module is used for re-identifying and re-ordering the query object and a plurality of candidate objects in the ranking list based on Bayesian query expansion;
the PTGAN processing module is used for carrying out PTGAN processing on the reordered query objects and candidate objects to realize the migration of a background difference area on the premise of keeping the foreground of the pedestrian unchanged;
the training module is used for inputting the query object and the candidate object subjected to the PTGAN processing into a trained Bayesian model, and calculating the real matching probability of each candidate object according to the image distance in the training data;
the adjusting module is used for adjusting the parameter value of the target parameter of the pedestrian re-identification system model according to the real matching probability of each candidate object;
and the identification module is used for searching out the pedestrian image with the highest similarity by inputting the image to be identified into the adjusted pedestrian re-identification system model.
6. The pedestrian re-identification device based on Bayesian query expansion as claimed in claim 5, wherein the re-identification module comprises:
the Bayes training module is used for training a Bayes model by using a training set containing true matching and false matching pedestrian pairs to obtain a trained Bayes model;
the prediction module is used for predicting the real matching probability of each candidate object through the trained Bayesian model according to the distance between the query object and the plurality of candidate object images;
and the query expansion module is used for performing query expansion according to the real matching probability of each candidate object and generating a new ranking list through the query expansion.
7. The pedestrian re-identification device based on Bayesian query expansion as claimed in claim 5, wherein the query expansion module comprises:
an average pool fusion module for fusing features of the query image and the top candidate object using the average pool;
and the first expansion updating module is used for performing query expansion according to the features of the query image and the top candidate object after the average pool is fused, and generating a new ranking list through the query expansion.
8. The pedestrian re-identification device based on Bayesian query expansion as claimed in claim 5, wherein the query expansion module comprises:
a maximum pool fusion module for fusing features of the query image and the top candidate object using the maximum pool;
and the second expansion updating module is used for performing query expansion according to the features of the query image and the top candidate object after the maximum pool fusion, and generating a new ranking list through the query expansion.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1-4 when executing the computer program.
10. A computer-readable medium, in which a computer program is stored which, when being processed and executed, carries out the steps of the method according to any one of claims 1 to 4.
CN201911325604.2A 2019-12-20 2019-12-20 Pedestrian re-identification method and device based on Bayesian query expansion Pending CN111291611A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911325604.2A CN111291611A (en) 2019-12-20 2019-12-20 Pedestrian re-identification method and device based on Bayesian query expansion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911325604.2A CN111291611A (en) 2019-12-20 2019-12-20 Pedestrian re-identification method and device based on Bayesian query expansion

Publications (1)

Publication Number Publication Date
CN111291611A true CN111291611A (en) 2020-06-16

Family

ID=71025360

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911325604.2A Pending CN111291611A (en) 2019-12-20 2019-12-20 Pedestrian re-identification method and device based on Bayesian query expansion

Country Status (1)

Country Link
CN (1) CN111291611A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113128466A (en) * 2021-05-11 2021-07-16 深圳大学 Pedestrian re-identification method, system, electronic device and storage medium
TWI769635B (en) * 2020-08-25 2022-07-01 新加坡商商湯國際私人有限公司 Network training pedestrian re-identification method and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104933183A (en) * 2015-07-03 2015-09-23 重庆邮电大学 Inquiring term rewriting method merging term vector model and naive Bayes
CN105389562A (en) * 2015-11-13 2016-03-09 武汉大学 Secondary optimization method for monitoring video pedestrian re-identification result based on space-time constraint
CN107145826A (en) * 2017-04-01 2017-09-08 浙江大学 The pedestrian's recognition methods again reordered based on double constraint metric learnings and sample
CN108256439A (en) * 2017-12-26 2018-07-06 北京大学 A kind of pedestrian image generation method and system based on cycle production confrontation network
CN110110755A (en) * 2019-04-04 2019-08-09 长沙千视通智能科技有限公司 Based on the pedestrian of PTGAN Regional disparity and multiple branches weight recognition detection algorithm and device
CN110457992A (en) * 2019-06-25 2019-11-15 上海高重信息科技有限公司 Pedestrian based on Bayes's optimisation technique recognition methods, device and system again

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104933183A (en) * 2015-07-03 2015-09-23 重庆邮电大学 Inquiring term rewriting method merging term vector model and naive Bayes
CN105389562A (en) * 2015-11-13 2016-03-09 武汉大学 Secondary optimization method for monitoring video pedestrian re-identification result based on space-time constraint
CN107145826A (en) * 2017-04-01 2017-09-08 浙江大学 The pedestrian's recognition methods again reordered based on double constraint metric learnings and sample
CN108256439A (en) * 2017-12-26 2018-07-06 北京大学 A kind of pedestrian image generation method and system based on cycle production confrontation network
CN110110755A (en) * 2019-04-04 2019-08-09 长沙千视通智能科技有限公司 Based on the pedestrian of PTGAN Regional disparity and multiple branches weight recognition detection algorithm and device
CN110457992A (en) * 2019-06-25 2019-11-15 上海高重信息科技有限公司 Pedestrian based on Bayes's optimisation technique recognition methods, device and system again

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
YUTIAN LIN 等: "Bayesian query expansion for multi-camera person re-identification" *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI769635B (en) * 2020-08-25 2022-07-01 新加坡商商湯國際私人有限公司 Network training pedestrian re-identification method and storage medium
CN113128466A (en) * 2021-05-11 2021-07-16 深圳大学 Pedestrian re-identification method, system, electronic device and storage medium
CN113128466B (en) * 2021-05-11 2023-12-05 深圳大学 Pedestrian re-recognition method, system, electronic device and storage medium

Similar Documents

Publication Publication Date Title
Xie et al. Multilevel cloud detection in remote sensing images based on deep learning
CN112052787B (en) Target detection method and device based on artificial intelligence and electronic equipment
CN108898086B (en) Video image processing method and device, computer readable medium and electronic equipment
CN107529650B (en) Closed loop detection method and device and computer equipment
US10366304B2 (en) Localization and mapping method
WO2019100724A1 (en) Method and device for training multi-label classification model
JP2022534337A (en) Video target tracking method and apparatus, computer apparatus, program
KR101531618B1 (en) Method and system for comparing images
CN111161311A (en) Visual multi-target tracking method and device based on deep learning
CN109871821B (en) Pedestrian re-identification method, device, equipment and storage medium of self-adaptive network
CN110765860A (en) Tumble determination method, tumble determination device, computer apparatus, and storage medium
CN110555428B (en) Pedestrian re-identification method, device, server and storage medium
CN111126249A (en) Pedestrian re-identification method and device combining big data and Bayes
CN111291768B (en) Image feature matching method and device, equipment and storage medium
WO2007038642A2 (en) Apparatus and method for trajectory-based identification of digital data content
CA3148760C (en) Automated image retrieval with graph neural network
CN114677565B (en) Training method and image processing method and device for feature extraction network
CN111008631B (en) Image association method and device, storage medium and electronic device
CN116580257A (en) Feature fusion model training and sample retrieval method and device and computer equipment
CN108205657A (en) Method, storage medium and the mobile terminal of video lens segmentation
CN112336342A (en) Hand key point detection method and device and terminal equipment
TW202109357A (en) Method and electronic equipment for visual positioning and computer readable storage medium thereof
CN111291611A (en) Pedestrian re-identification method and device based on Bayesian query expansion
Wang et al. Salient object detection using biogeography-based optimization to combine features
CN113704276A (en) Map updating method and device, electronic equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination