CN114428875A - Pedestrian re-identification database building method and device, computer equipment and storage medium - Google Patents
Pedestrian re-identification database building method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN114428875A CN114428875A CN202011179551.0A CN202011179551A CN114428875A CN 114428875 A CN114428875 A CN 114428875A CN 202011179551 A CN202011179551 A CN 202011179551A CN 114428875 A CN114428875 A CN 114428875A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- matching
- track
- characteristic
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000003860 storage Methods 0.000 title claims abstract description 14
- 230000002159 abnormal effect Effects 0.000 claims abstract description 26
- 230000008859 change Effects 0.000 claims abstract description 19
- 238000012216 screening Methods 0.000 claims abstract description 8
- 238000004590 computer program Methods 0.000 claims description 12
- 238000001914 filtration Methods 0.000 claims description 10
- 238000010276 construction Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 11
- 238000012544 monitoring process Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000011840 criminal investigation Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a pedestrian re-identification database building method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a first pedestrian characteristic, assigning a pedestrian ID corresponding to the first pedestrian characteristic, and storing the pedestrian ID in a ReID library; acquiring a second pedestrian characteristic, performing characteristic matching on the second pedestrian characteristic and the first pedestrian characteristic in the ReID library, screening and de-duplicating matching information, canceling redundant matching pairs on the same pedestrian ID, and canceling splicing of pedestrian tracks corresponding to the redundant matching pairs; obtaining a matching pair lower than the minimum confidence threshold value, and canceling the splicing of the pedestrian tracks in the matching pair; and detecting the characteristic change of the bound matching pairs in real time, and updating the characteristic change of the area of the pedestrian in the store. The method and the device can ensure that the user ID which is wrongly identified is corrected under the condition that shielding exists or other modules are abnormal, so that the user experience and the robustness of the system are improved.
Description
Technical Field
The present application relates to the field of image retrieval technologies, and in particular, to a method, an apparatus, a computer device, and a storage medium for pedestrian re-identification and library building.
Background
With the rapid development of information technology, image processing technology is widely applied to aspects of life, and pedestrian re-identification and library building technology is a hot topic researched in academia and industry in recent years. The pedestrian re-identification database building technology is a technology for judging whether a specific person exists in an image or a video sequence by utilizing a computer vision technology, can be matched with technologies such as pedestrian detection, pedestrian tracking and the like, is widely applied to the fields of intelligent video monitoring, intelligent security, unmanned driving, intelligent criminal investigation, intelligent retail sale and the like, and has wide technical application prospects.
The scenes in actual life are complex and changeable, and the considered factors include the change of illumination, the change of wearing of the postures of pedestrians, the limitation of monitoring visual angles and monitoring coverage ranges, the personalized requirements and the like. At present, a few mature pedestrian tracking schemes are based on ideal input data, and once the pedestrian tracking schemes are used in the situations of multiple shelters, multiple pedestrians and multiple similar samples, scientific ReiD library building and query mechanisms are not assisted, the false detection rate of the system can be greatly improved, so that the intelligent retail settlement system such as an unmanned shop and the like cannot normally operate, and shopping settlement is influenced.
Disclosure of Invention
The invention aims to solve the problem of pedestrian tracking in a monitoring scene. Therefore, in order to solve the above technical problems, it is necessary to provide a pedestrian re-identification library building method, device, computer equipment and storage medium, which combine with a pedestrian re-identification library building technology, and enable the pedestrian re-identification library building technology to realize full-scene tracking across cameras in a scene where tracking is disabled due to occlusion.
A pedestrian re-identification database building method, the method comprising:
tracking a pedestrian track of a pedestrian, detecting that the pedestrian enters a registration area, acquiring a multi-angle picture of the pedestrian, acquiring a first pedestrian characteristic, assigning a pedestrian ID corresponding to the first pedestrian characteristic, and storing the pedestrian ID and the first pedestrian characteristic to a ReID library;
detecting that a pedestrian enters an in-store area, acquiring a picture corresponding to any pedestrian track of the pedestrian, extracting a second pedestrian feature of the pedestrian track, performing feature matching on the second pedestrian feature and a first pedestrian feature in a ReID (ReID) library, if matching is successful, forming a matching pair of the pedestrian track corresponding to the second pedestrian feature and a pedestrian ID corresponding to the first pedestrian feature, and outputting pairing information for splicing all pedestrian tracks of the pedestrian IDs;
screening and de-duplicating pairing information, canceling redundant matching pairs on the same pedestrian ID, and canceling splicing of pedestrian tracks corresponding to the redundant matching pairs;
obtaining a matching pair lower than the minimum confidence threshold value, and canceling the splicing of the pedestrian tracks in the matching pair;
and detecting the characteristic change of the bound matching pairs in real time, and updating the characteristic change of the area of the pedestrian in the store.
In one embodiment, when the pedestrian track of the pedestrian enters the registration area, the camera group is adopted to collect multi-angle pictures of the pedestrian, and the texture and the depth characteristics of the pedestrian are obtained at least from the front, the side and the overhead overlooking angles.
In one embodiment, if only one pedestrian track is detected and only one pedestrian ID to be bound exists in the ReID library, the pedestrian track is directly matched with the pedestrian ID, and the pedestrian track is spliced with other pedestrian tracks of the pedestrian ID;
if a plurality of pedestrian tracks are detected, acquiring the pedestrian features of each pedestrian track, performing feature matching in the ReID library, determining the pedestrian ID matched with the pedestrian track, outputting pairing information, and discarding the pedestrian track which is not successfully matched.
In one embodiment, if the pedestrian ID is successfully matched with at least one pedestrian track, one pedestrian track is reserved to form a matching pair with the pedestrian ID, the other pedestrian tracks are subjected to secondary matching, and the pedestrian track with the minimum distance and the pedestrian ID are forcibly matched respectively to form the matching pair.
In one embodiment, all the matching pairs are traversed, a minimum confidence threshold value is preset, the similarity distance between the matched pedestrian track and any pedestrian ID in the ReID library is respectively obtained, and the matching of the matching pairs with the similarity distance lower than the minimum confidence threshold value is cancelled.
In one embodiment, extracting the abnormal pedestrian human body features comprises extracting feature information of human body instance segmentation by using a color histogram.
In one embodiment, the distance determination includes,
acquiring the human body characteristics of the abnormal pedestrians and the human body characteristics of each pedestrian image in the image data information base;
calculating the similarity between the human body characteristics of the abnormal pedestrians and the human body characteristics of each pedestrian image in the image data information base;
determining a characteristic distance between the abnormal pedestrian image and the pedestrian image;
when an abnormal pedestrian image exists, calculating the similarity between the human body characteristics of the abnormal pedestrian and the human body characteristics of each pedestrian image in the image data information base, determining the characteristic distance between the abnormal pedestrian image and the pedestrian image, and matching the pedestrian image in the image data information base corresponding to the minimum distance into the abnormal pedestrian image by using minimum distance matching;
when at least two abnormal pedestrian images exist, respectively calculating the similarity of the human body characteristics of the at least two abnormal pedestrian images and each pedestrian image in the image data information base, respectively determining the characteristic distance between the at least two abnormal pedestrian images and the pedestrian image, calculating the obtained distance matrix, and respectively matching the pedestrian images in the image data information base corresponding to the result into at least two abnormal pedestrian images;
and completing image matching.
A pedestrian re-identification garage building device comprises:
the acquisition module is used for tracking the pedestrian track of the pedestrian, detecting that the pedestrian enters the registration area, acquiring the multi-angle picture of the pedestrian, acquiring a first pedestrian characteristic, assigning a pedestrian ID corresponding to the first pedestrian characteristic, and storing the pedestrian ID and the first pedestrian characteristic to a ReID library;
the matching module is used for detecting that a pedestrian enters an in-store area, acquiring a picture corresponding to any pedestrian track of the pedestrian, extracting a second pedestrian feature of the pedestrian track, performing feature matching on the second pedestrian feature and a first pedestrian feature in a ReID (ReID) library, forming a matching pair of the pedestrian track corresponding to the second pedestrian feature and a pedestrian ID corresponding to the first pedestrian feature if matching is successful, and outputting pairing information for splicing all pedestrian tracks of the pedestrian IDs;
the duplication removing module is used for screening and removing duplication of the pairing information, canceling redundant matching pairs on the same pedestrian ID and canceling splicing of pedestrian tracks corresponding to the redundant matching pairs;
the filtering module is used for acquiring a matching pair lower than a minimum confidence coefficient threshold value and canceling the splicing of the pedestrian tracks in the matching pair;
and the updating module is used for detecting the characteristic change of the bound matching pair in real time and updating the characteristic change of the area of the pedestrian in the store.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the above method when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.
The pedestrian re-identification database building method, the pedestrian re-identification database building device, the computer equipment and the storage medium have the following effects:
1. texture and depth information of each orientation of the pedestrian are acquired through the multi-angle depth camera for feature registration, and the characteristics of the pedestrian are fully acquired to the maximum extent so that the problem of shielding in a complex scene can be solved.
2. And performing efficient cross matching on all unmatched pedestrian positions and unmatched ReIDs in the library by using efficient multi-target matching logic.
3. Using accurate deduplication logic, multiple pedestrians with similar characteristics are prevented from being mistakenly merged into the same ReID.
4. The mismatch in the matching stage is corrected using accurate filtering logic.
5. And dynamically updating the characteristic change of the pedestrians in the store, such as wearing and taking off coats, hats, sunglasses and the like by using a characteristic updating mechanism.
Drawings
FIG. 1 is a flow chart illustrating a method for pedestrian re-identification database construction according to an embodiment;
FIG. 2 is a schematic overall flowchart of a pedestrian re-identification database construction method according to an embodiment;
FIG. 3 is a block diagram of a pedestrian re-identification library creating apparatus according to an embodiment;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The application provides a pedestrian tracking re-identification library building method for a complex scene, which can accurately and efficiently track a plurality of objects simultaneously, and in one embodiment, as shown in fig. 1 to 2, the method comprises the following steps:
s100, tracking the pedestrian track of the pedestrian, detecting that the pedestrian enters a registration area, collecting multi-angle pictures of the pedestrian, acquiring first pedestrian features, assigning a pedestrian ID corresponding to the first pedestrian features, and storing the pedestrian ID and the first pedestrian features to a ReID library.
In this embodiment, a pedestrian enters an in-store registration area through a store gate, preferably, a camera group is used to collect multi-angle RGB-D pictures of the pedestrian, so as to fully acquire texture and depth features of three angles of front, side and overhead overlooking of the pedestrian to the maximum extent, acquire a first pedestrian feature of the pedestrian for registration of a feature library, acquire a feature of an input picture by using a feature extraction model, assign a temporary pedestrian ID, and store the pedestrian ID and the first pedestrian feature in a ReID library.
Specifically, when the pedestrian track enters the registration area from the gate area, the processing of the track by the instant activation ReID module is entered, and the registration mode is entered. At this time, a camera group including 3 cameras of the front, back and side faces is shot at the same time, preferably, 35 frames (1 second) are shot continuously, the image characteristics of 35 frames shot by each camera are respectively averaged, then a ReID feature library of the pedestrian track is taken into account and a new ReID is assigned to the track, and the above strategy can be expressed by the following formula:
wherein, for equation (1), ReidFeatureiDenotes the average characteristic of the ith plane (front, back, side), NiIndicating the number of image frames counted by tracking registration within 35 frames (1 second) of continuous shooting corresponding to the ith surface, and then outputting the characteristics describing the ReID by using an average strategy.
S200, detecting that the pedestrian enters an in-store area, collecting a picture corresponding to any pedestrian track of the pedestrian, extracting a second pedestrian feature of the pedestrian track, carrying out feature matching on the second pedestrian feature and a first pedestrian feature in a ReID library, if matching is successful, forming a matching pair of the pedestrian track corresponding to the second pedestrian feature and a pedestrian ID corresponding to the first pedestrian feature, and outputting pairing information for splicing all pedestrian tracks of the pedestrian IDs.
In this embodiment, a pedestrian track of a pedestrian is detected to enter an in-store area, after a continuous multi-frame picture appears in the pedestrian track in the store, feature extraction is performed on a picture acquired by any one camera of a camera group through a model, a second pedestrian feature is acquired, and a ReID search module is called to perform feature matching.
Specifically, when any new pedestrian trajectory appears, 3 consecutive pedestrian positions are deemed to form a new pedestrian trajectory, and only when the new pedestrian trajectory appears in the in-store area, this time, the ReID module intervenes and enters a search mode to find a matching ReID.
If the pedestrian track position and the unbound pedestrian ID are only one, the pedestrian track is directly bound with the pedestrian ID, otherwise, the ReiD feature matching model is called to carry out pairing matching under other conditions, a matching pair is formed, and the pedestrian track which is not successfully matched is discarded. Preferably, for a many-to-many matching mode, in order to achieve high operation efficiency, a Hungarian matching mode is adopted, and all unmatched pedestrian tracks and unmatched pedestrian ID features in the ReID library are matched in a cross mode to form matching pairs.
The matching search by using the ReID module can be actually regarded as a feature-based image retrieval process, different objects often adopt different measurement modes due to different feature extractions, common measurement modes include cosine distance, euclidean distance, babbitt distance or correlation and the like, color histogram features often use a correlation measurement mode, the euclidean distance is often used for spatial location features, and the cosine distance is often used for distance measurement among most texture features or depth learning features. The cosine distance measure can be expressed as follows:
dist＝1-cos(θ) (3)；
wherein, for formula (2), a and b are feature representations of a certain query image and a certain bottom library image in the ReID registry, cos (θ) is the similarity of the rest chords, and for formula (3), dist is the cosine distance between the two feature representations, which is used for the assignment matrix calculation in hungarian matching.
In the embodiment, a cosine distance calculation mode is adopted, high operation efficiency is achieved under the condition of one-to-many, single unmatched pedestrian tracks are compared with unmatched pedestrian ID features in the ReID library one by one, and the pedestrian ID with the minimum distance is bound. And because the value range of the cosine distance mode is [0,1], the method has probability significance in accordance with the theorem and can further provide basis for the subsequent filtering step.
S300, screening and de-duplicating the pairing information, canceling redundant matching pairs on the same pedestrian ID, and canceling splicing the pedestrian tracks corresponding to the redundant matching pairs.
In this embodiment, the matching information after ReID matching is checked to perform screening and deduplication, matching of matching pairs in which different face images of the same pedestrian ID are respectively matched with the pedestrian tracks is cancelled, and splicing of pedestrian tracks corresponding to redundant matching pairs is cancelled.
In particular, due to problems including, but not limited to, model defects or the fact that the feature distances of two samples are close, it may occur that two pedestrian trajectories each match a different face-oriented feature on the same pedestrian ID. So after the hungarian matching of step S200, the pedestrian trajectories assigned the same pedestrian ID are taken out to be subjected to the secondary review, thereby reducing the unmatched pedestrian trajectories. Specifically, the pedestrian trajectory and the pedestrian ID having the smallest distance are forcibly matched, and even if the distance is greater than a preset distance threshold, the pedestrian trajectory and the pedestrian ID are matched to form a matching pair. While the remaining pedestrian trajectories are not matched.
S400, obtaining a matching pair lower than the minimum confidence threshold value, and canceling the stitching of the pedestrian track in the matching pair.
In this embodiment, the filtering mechanism intervenes to examine track-ID matching pairs that are below a minimum confidence threshold and cancel the match, and to cancel the stitching of the pedestrian tracks in the matching pair.
Specifically, examining all matching pairs of pedestrian trajectories and pedestrian IDs that have been matched, the accuracy of the matching can be further improved using a similarity threshold constraint, which can be expressed by the following formula:
wherein, for formula (4), DmatchedQ_GRepresenting the similarity distance between a certain pedestrian position matched with the pedestrian position and a certain ReiD in the ReiD registry base, G representing the ReiD, setting a filtering threshold value Filter _ Thresh, considering the condition that the similarity distance is greater than the filtering threshold value as that the reliability is too low, and removing the binding of the matching pair with the similarity lower than the filtering threshold value
And for the pedestrian track which is not matched at last, if the pedestrian track of the pedestrian appears in the in-store area, the pedestrian track is not processed until the next frame has better characteristics and is matched successfully.
And S500, detecting the characteristic change of the bound matching pairs in real time, and updating the characteristic change of the area of the pedestrian in the store.
In the embodiment, the characteristic change condition of the bound track-pedestrian ID matching pair is checked at any time, and the characteristic change caused by changing the clothes and hat ornament by the pedestrian in the store is dynamically updated.
In particular, in principle it is only recommended to add new features, against replacing features, avoiding polluting the original ReID feature library. However, due to the fact that the scene is complex, a pedestrian crowding scene may occur, and the target detection technology cannot accurately segment different pedestrians, the positions of all matched tracks under the camera at the same depth are checked, and whether the ReID library needs to be updated is judged by adopting the following formula:
wherein, for formula (5), two matched tracks are represented, and the detection frames are Bdboxi and Bdboxj, IOU respectivelyBdboxi_BdboxjFor its degree of overlap, for equation (6), we mean setting the no-update overlap threshold to NotUpdate _ Thresh, and if the degree of overlap of two detection boxes is greater than this threshold, it is reasonable to believe that the detection boxes are not correctly positioned, and the features will introduce other background noise that will contaminate the original ReID feature library, and therefore no feature update is performed on reids that match on both detection boxes, while for detection boxes below NotUpdate _ Thresh, we can enter the features of the candidate updated ReID library, and update the features as needed.
In one embodiment, step S600 is further included, deleting the bound ReID.
Specifically, when it is detected that the pedestrian trajectory enters the exit area from the in-store area, that is, the pedestrian has exited the store, the pedestrian ID to which the pedestrian trajectory is bound is deleted from the ReID library.
It should be understood that, although the steps in the flowchart are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 3, there is provided a pedestrian re-identification library reconstruction apparatus applying the above method, including: acquisition module 100, matching module 200, deduplication module 300, filtering module 400, and update module 500. Wherein:
the collecting module 100 is used for tracking the pedestrian track of the pedestrian, detecting that the pedestrian enters the registration area, collecting the multi-angle picture of the pedestrian, acquiring the first pedestrian feature, assigning the pedestrian ID corresponding to the first pedestrian feature, and storing the pedestrian ID and the first pedestrian feature to the ReID library.
The matching module 200 is configured to detect that a pedestrian enters an in-store area, collect a picture corresponding to any pedestrian trajectory of the pedestrian, extract a second pedestrian feature of the pedestrian trajectory, perform feature matching on the second pedestrian feature and a first pedestrian feature in the ReID library, form a matching pair of the pedestrian trajectory corresponding to the second pedestrian feature and a pedestrian ID corresponding to the first pedestrian feature if matching is successful, and output pairing information for splicing all pedestrian trajectories of the pedestrian IDs.
And the duplication eliminating module 300 is used for screening and eliminating duplication of the pairing information, canceling redundant matching pairs on the same pedestrian ID and canceling splicing of pedestrian tracks corresponding to the redundant matching pairs.
And the filtering module 400 is configured to obtain a matching pair lower than the minimum confidence threshold, and cancel stitching of the pedestrian trajectory in the matching pair.
And the updating module 500 is used for detecting the characteristic change of the bound matching pair in real time and updating the characteristic change of the area of the pedestrian in the store.
For specific definition of the pedestrian re-identification library building device, reference may be made to the above definition of the pedestrian re-identification library building method, and details are not repeated here. The various modules in the above-described apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
The pedestrian re-identification library building device can cover all the areas of the corresponding monitoring scenes by utilizing multiple groups of depth cameras in the corresponding monitoring scenes, and can process the situation that multiple people enter and exit the corresponding monitoring scenes simultaneously. The tracking module requires ReID module intervention at lower error rates. The accuracy rate of the ReiD identification is high, and the normal operation of the visual tracking system under the corresponding scene can be ensured to a great extent. Meanwhile, the system can be switched into a data acquisition mode and a preprocessing mode and is used for model optimization iteration of modules including pedestrian detection, pedestrian tracking, ReID identification and the like.
In one embodiment, a computer device is provided, which may be a data management server, and its internal structure diagram may be as shown in fig. 4. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer equipment is used for communicating with an external data source terminal through network connection so as to receive data uploaded by the data source terminal. The computer program is executed by a processor to implement a pedestrian re-identification banking method.
Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the above-mentioned pedestrian re-identification library construction method when executing the computer program.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The invention has rich application scenes and can be widely applied to a plurality of scenes such as unmanned supermarkets, security monitoring, criminal investigation and the like. By taking application to intelligent retail as an example, unmanned store projects can be developed, brand-new shopping experience of non-perception shopping is created, and the technical capabilities of multi-user shopping entering a store, real-time shopping cart inventory checking and automatic settlement after the shopping is out of the store are realized. Specifically, because the unmanned shop scene is different from the security scene of ordinary public place, the camera of security scene is mostly downward to take a photograph to one side and allow certain degree of sheltering from to it appears many people to appear such complex conditions many times to appear to be few. But missing detection and false detection in an unmanned store are strictly not allowed to occur because an automatic settlement function is caused to be abnormal. The simple use of the target tracking or the traditional pedestrian re-identification module is difficult to adapt to such a scene, consumes huge computing resources, and is difficult to realize mass production deployment of the scheme. According to the pedestrian re-identification database building method, the device, the computer equipment and the storage medium, the intervention of a characteristic database building query mechanism capable of efficiently dispersing the inter-class distance and the convergence intra-class distance can be utilized according to the existing resources, so that the matching accuracy is improved, on the basis of extracting the characteristics of a targeted model, a multi-characteristic-dimension ReID database building algorithm framework is matched, the classification operation burden of the model is shared, and the matching accuracy and the operation efficiency are improved to a great extent.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A pedestrian re-identification database building method, the method comprising:
tracking a pedestrian track of a pedestrian, detecting that the pedestrian enters a registration area, collecting a multi-angle picture of the pedestrian, acquiring a first pedestrian characteristic, assigning a pedestrian ID corresponding to the first pedestrian characteristic, and storing the pedestrian ID and the first pedestrian characteristic to a ReID library;
detecting that the pedestrian enters an in-store area, acquiring a picture corresponding to any pedestrian track of the pedestrian, extracting a second pedestrian feature of the pedestrian track, performing feature matching on the second pedestrian feature and a first pedestrian feature in the ReID library, if matching is successful, forming a matching pair of the pedestrian track corresponding to the second pedestrian feature and a pedestrian ID corresponding to the first pedestrian feature, and outputting pairing information for splicing all pedestrian tracks of the pedestrian IDs;
screening and de-duplicating the pairing information, canceling redundant matching pairs on the same pedestrian ID, and canceling splicing of pedestrian tracks corresponding to the redundant matching pairs;
obtaining a matching pair lower than the minimum confidence threshold value, and canceling the splicing of the pedestrian tracks in the matching pair;
and detecting the characteristic change of the bound matching pairs in real time, and updating the characteristic change of the area of the pedestrian in the store.
2. The method according to claim 1, wherein when the pedestrian track of the pedestrian enters the registration area, a camera group is adopted to collect multi-angle pictures of the pedestrian, and the texture and depth characteristics of the pedestrian are obtained at least from the front, side and overhead overlooking angles.
3. The method according to claim 1, characterized in that if only one pedestrian track is detected and only one pedestrian ID to be bound exists in the ReID library, the pedestrian track is directly matched with the pedestrian ID, and the pedestrian track is spliced with other pedestrian tracks of the pedestrian ID;
if a plurality of pedestrian tracks are detected, acquiring the pedestrian features of each pedestrian track, performing feature matching in the ReiD library, determining the pedestrian ID matched with the pedestrian track, outputting pairing information, and discarding the pedestrian track which is not successfully matched.
4. The method of claim 3,
if the pedestrian ID is successfully matched with at least one pedestrian track, one pedestrian track is reserved to form a matching pair with the pedestrian ID, the other pedestrian tracks are subjected to secondary matching, and the pedestrian track with the minimum distance and the pedestrian ID are forcibly matched respectively to form a matching pair.
5. The method of claim 1, wherein all matching pairs are traversed, a minimum confidence threshold is preset, the similarity distance between the matched pedestrian track and any pedestrian ID in the Re ID library is respectively obtained, and the matching of the matching pair with the similarity distance lower than the minimum confidence threshold is cancelled.
6. The method according to claim 3, wherein the extracting the abnormal pedestrian human body features comprises extracting feature information of the human body instance segmentation by using a color histogram.
7. The method according to claim 5 or 6, wherein the distance determination comprises,
acquiring the human body characteristics of the abnormal pedestrian and the human body characteristics of each pedestrian image in the image data information base;
calculating the similarity between the human body characteristics of the abnormal pedestrians and the human body characteristics of each pedestrian image in the image data information base;
determining a characteristic distance between the abnormal pedestrian image and the pedestrian image;
when an abnormal pedestrian image exists, calculating the similarity between the human body characteristics of the abnormal pedestrian and the human body characteristics of each pedestrian image in the image data information base, determining the characteristic distance between the abnormal pedestrian image and the pedestrian image, and matching the pedestrian image in the image data information base corresponding to the minimum distance into the abnormal pedestrian image by using minimum distance matching;
when at least two abnormal pedestrian images exist, respectively calculating the similarity of the human body characteristics of the at least two abnormal pedestrian images and each pedestrian image in the image data information base, respectively determining the characteristic distance between the at least two abnormal pedestrian images and the pedestrian image, calculating a distance matrix, and respectively matching the pedestrian images in the image data information base corresponding to the result into the at least two abnormal pedestrian images;
and completing image matching.
8. A pedestrian re-identification garage construction apparatus, the apparatus comprising:
the system comprises an acquisition module, a registration module and a ReID (remote identity) library, wherein the acquisition module is used for tracking the pedestrian track of a pedestrian, detecting that the pedestrian enters a registration area, acquiring a multi-angle picture of the pedestrian, acquiring a first pedestrian characteristic, assigning a pedestrian ID corresponding to the first pedestrian characteristic, and storing the pedestrian ID and the first pedestrian characteristic to the ReID library;
the matching module is used for detecting that the pedestrian enters the in-store area, acquiring a picture corresponding to any pedestrian track of the pedestrian, extracting a second pedestrian feature of the pedestrian track, performing feature matching on the second pedestrian feature and a first pedestrian feature in the ReiD library, if matching is successful, forming a matching pair of the pedestrian track corresponding to the second pedestrian feature and a pedestrian ID corresponding to the first pedestrian feature, and outputting pairing information for splicing all pedestrian tracks of the pedestrian IDs;
the duplication removing module is used for screening and removing duplication from the pairing information, canceling redundant matching pairs on the same pedestrian ID and canceling splicing of pedestrian tracks corresponding to the redundant matching pairs;
the filtering module is used for acquiring a matching pair lower than a minimum confidence coefficient threshold value and canceling the splicing of the pedestrian tracks in the matching pair;
and the updating module is used for detecting the characteristic change of the bound matching pair in real time and updating the characteristic change of the area of the pedestrian in the store.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented when the computer program is executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011179551.0A CN114428875A (en) | 2020-10-29 | 2020-10-29 | Pedestrian re-identification database building method and device, computer equipment and storage medium |
| CA3136997A CA3136997A1 (en) | 2020-10-29 | 2021-10-29 | A pedestrian re-identification for database construction method, apparatus, computer device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011179551.0A CN114428875A (en) | 2020-10-29 | 2020-10-29 | Pedestrian re-identification database building method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114428875A true CN114428875A (en) | 2022-05-03 |
Family
ID=81309218
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011179551.0A Pending CN114428875A (en) | 2020-10-29 | 2020-10-29 | Pedestrian re-identification database building method and device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114428875A (en) |
| CA (1) | CA3136997A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115328962A (en) * | 2022-10-13 | 2022-11-11 | 亿海蓝（北京）数据技术股份公司 | Ship static information processing method and device, electronic equipment and storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114694184B (en) * | 2022-05-27 | 2022-10-14 | 电子科技大学 | Pedestrian re-identification method and system based on multi-template feature updating |
| CN115546493B (en) * | 2022-12-06 | 2023-05-02 | 成都智元汇信息技术股份有限公司 | Method, device and medium for filtering and weighing package identification information based on column data |
| CN117876973B (en) * | 2024-03-13 | 2024-06-07 | 中科盛阳信息技术有限公司 | Deep learning-based park security monitoring method and system |
-
2020
- 2020-10-29 CN CN202011179551.0A patent/CN114428875A/en active Pending
-
2021
- 2021-10-29 CA CA3136997A patent/CA3136997A1/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115328962A (en) * | 2022-10-13 | 2022-11-11 | 亿海蓝（北京）数据技术股份公司 | Ship static information processing method and device, electronic equipment and storage medium |
| CN115328962B (en) * | 2022-10-13 | 2022-12-27 | 亿海蓝（北京）数据技术股份公司 | Ship static information processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CA3136997A1 (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114428875A (en) | Pedestrian re-identification database building method and device, computer equipment and storage medium | |
| TWI677825B (en) | Method of video object tracking and apparatus thereof and non-volatile computer readable storage medium | |
| Hodan et al. | Bop: Benchmark for 6d object pose estimation | |
| CN107093171B (en) | Image processing method, device and system | |
| US7693310B2 (en) | Moving object recognition apparatus for tracking a moving object based on photographed image | |
| JP3584334B2 (en) | Human detection tracking system and human detection tracking method | |
| Merad et al. | Fast people counting using head detection from skeleton graph | |
| KR101781358B1 (en) | Personal Identification System And Method By Face Recognition In Digital Image | |
| TWI798815B (en) | Target re-identification method, device, and computer readable storage medium | |
| Li et al. | An anti-fraud system for car insurance claim based on visual evidence | |
| CN111723773B (en) | Method and device for detecting carryover, electronic equipment and readable storage medium | |
| CN114677633B (en) | Multi-component feature fusion-based pedestrian detection multi-target tracking system and method | |
| CN112633255B (en) | Target detection method, device and equipment | |
| Tiwari et al. | A survey on shadow detection and removal in images and video sequences | |
| CN112541403A (en) | Indoor personnel falling detection method utilizing infrared camera | |
| CN114998928A (en) | Cross-modal pedestrian re-identification method based on multi-granularity feature utilization | |
| de Carvalho et al. | Anomaly detection with a moving camera using multiscale video analysis | |
| Santos et al. | Car recognition based on back lights and rear view features | |
| GB2516142A (en) | Method of tracking objects using hyperspectral imagery | |
| CN113869364A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| CN116229336B (en) | Video moving target identification method, system, storage medium and computer | |
| US20180053321A1 (en) | Image Target Relative Position Determining Method, Device, and System Thereof | |
| CN113642450B (en) | Video face recognition method, system and storage medium | |
| CN115761879A (en) | Gait recognition method and model training method, device, terminal and storage medium thereof | |
| CN111860100B (en) | Pedestrian number determining method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20220503 |
|
| WD01 | Invention patent application deemed withdrawn after publication |