WO2023236514A1 - 一种跨摄像头的多目标追踪方法、装置、设备及介质 - Google Patents
一种跨摄像头的多目标追踪方法、装置、设备及介质 Download PDFInfo
- Publication number
- WO2023236514A1 WO2023236514A1 PCT/CN2022/142129 CN2022142129W WO2023236514A1 WO 2023236514 A1 WO2023236514 A1 WO 2023236514A1 CN 2022142129 W CN2022142129 W CN 2022142129W WO 2023236514 A1 WO2023236514 A1 WO 2023236514A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target
- targets
- classified
- camera
- unclassified
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 95
- 230000000007 visual effect Effects 0.000 claims abstract description 86
- 239000011159 matrix material Substances 0.000 claims description 94
- 238000001514 detection method Methods 0.000 claims description 50
- 238000012545 processing Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 16
- 230000027455 binding Effects 0.000 claims 2
- 238000009739 binding Methods 0.000 claims 2
- 238000004364 calculation method Methods 0.000 claims 1
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 16
- 230000015556 catabolic process Effects 0.000 description 9
- 238000006731 degradation reaction Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 238000000605 extraction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000025174 PANDAS Diseases 0.000 description 1
- 208000021155 Paediatric autoimmune neuropsychiatric disorders associated with streptococcal infection Diseases 0.000 description 1
- 240000000220 Panda oleosa Species 0.000 description 1
- 235000016496 Panda oleosa Nutrition 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012958 reprocessing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Definitions
- This application relates to the field of artificial intelligence, and in particular to a cross-camera multi-target tracking method, device, equipment and medium.
- target tracking is one of the most valuable research directions in the field of artificial intelligence machine vision.
- target tracking topics are divided into two subcategories: single target tracking (SOT, Single Object Tracking) and multiple target tracking (MOT, Multi Object Tracking): Single target tracking focuses on the tracking of a specific target or the tracking problem of a simpler scene.
- SOT Single target tracking
- MOT Multi Object Tracking
- multi-target tracking issues are more involved, such as the self-driving data set KITTI, which includes tracking annotations for both vehicles and pedestrians; the MOT-Challenge data set, which is a target tracking data set focusing on pedestrian tracking; and the PANDA data set, which focuses on
- KITTI self-driving data set
- MOT-Challenge data set which is a target tracking data set focusing on pedestrian tracking
- PANDA data set which focuses on
- the problem of pedestrian tracking in ultra-large-scale scenes is more complex, pedestrians are more widely distributed, and the problem is more difficult.
- these datasets usually frame the tracking problem under the same camera.
- the target's action trajectory usually spans cameras.
- the cross-camera target tracking algorithm for target tracking of pedestrians, existing methods usually use a two-step superposition method to achieve: the first step is to track the target in a single camera to form a local trajectory; the second step The first step is to use the classic tracklet-to-tracklet matching algorithm to match and splice several output results of single-camera tracking; using this cross-camera target tracking method to perform single segment tracking and then track matching will cause errors in track matching. performance degradation.
- the purpose of this application is to provide a cross-camera multi-target tracking method that can more accurately achieve cross-camera multi-target tracking.
- the specific plan is as follows:
- this application discloses a cross-camera multi-target tracking method, including:
- the remaining targets after deduplication and the second type of targets in non-overlapping visual space areas captured at different shooting times are classified respectively to obtain the remaining targets after deduplication and the non-overlapping visual space.
- Each second type of target in the area corresponds to its corresponding path trajectory.
- determine the first type of targets in the video frame that are located in the overlapping visual space area between different cameras and have the same shooting time including:
- determine whether the first cosine distance meets the target preset conditions including:
- the first cosine distance corresponding to each group of different moving targets captured by different cameras at the same shooting time is saved to the first preset distance matrix; where the storage position of the first cosine distance in the preset distance matrix is based on The position determined by the identification number of the moving target corresponding to the first cosine distance;
- the first preset condition is whether the first cosine distance is less than the first A preset distance threshold
- the second preset condition is whether the first cosine distance is the minimum value among the corresponding row and column values.
- classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas including:
- the second cosine distance is used to determine whether the target among the unclassified targets and the target among the classified targets are the same target, and the unclassified targets are classified based on the judgment result.
- use the second cosine distance to determine whether the targets in the unclassified targets and the targets in the classified targets are the same target including:
- the third preset condition is whether the second cosine distance is less than the second preset distance threshold.
- the fourth preset condition is whether the second cosine distance is the minimum value among the corresponding row and column values;
- the targets among the unclassified targets and the targets among the classified targets are the same target. If the third preset condition and the fourth preset condition are not met, Then the targets in the unclassified targets and the targets in the classified targets are not the same target.
- use the characteristic information of classified targets at historical shooting time and the characteristic information of unclassified targets at the current shooting time to determine the second cosine distance between the classified target and the unclassified target include:
- the cosine distance with the smallest value is selected from the cosine distances as the second cosine distance between the classified target and the unclassified target.
- calculate the cosine distances between various feature information of classified targets at historical shooting time and various feature information of unclassified targets at the current shooting time to obtain corresponding multiple cosine distances including :
- a third preset distance matrix of multiple cosine distances Use the first feature matrix and the second feature matrix to perform cosine distance operations to obtain the relationship between various feature information of classified targets stored at different camera historical shooting times and various feature information of unclassified targets at the current shooting time.
- a third preset distance matrix of multiple cosine distances Use the first feature matrix and the second feature matrix to perform cosine distance operations to obtain the relationship between various feature information of classified targets stored at different camera historical shooting times and various feature information of unclassified targets at the current shooting time.
- the cosine distance with the smallest value is selected from the cosine distance as the second cosine distance between the classified target and the unclassified target, including:
- the cosine distance with the smallest value is selected from the cosine distances in the third preset distance matrix as the second cosine distance between the classified target and the unclassified target.
- bind various feature information of the same classified target at historical shooting times corresponding to different cameras to obtain multiple bound information, and store the bound information in the first feature matrix in sequence include:
- a plurality of third feature matrices are integrated to obtain a first feature matrix storing various feature information of the classified target.
- use the characteristic information of classified targets at historical shooting time and the characteristic information of unclassified targets at the current shooting time to determine the second cosine distance between the classified target and the unclassified target include:
- the cosine distance with the smallest value is selected from multiple cosine distances as the second cosine distance between the classified target and the unclassified target.
- the cosine distance with the smallest value is selected from multiple cosine distances as the second cosine distance between the classified target and the unclassified target, including:
- the cosine distance with the smallest value is selected from a plurality of cosine distances in several fourth preset distance matrices as the second cosine distance between the classified target and the unclassified target.
- classify the remaining targets after deduplication and the second type of targets in non-overlapping visual space areas captured at different shooting times including:
- this application discloses a cross-camera multi-target tracking device, including:
- the video frame acquisition module is configured to acquire video frames captured by several cameras
- the deduplication module is set to determine the first type of targets in the video frame that are located in the overlapping visual space area between different cameras and have the same shooting time, and perform deduplication processing on the first type of targets to obtain the remaining targets after deduplication;
- the classification module is set to classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas based on time order, so as to obtain the remaining targets after each deduplication.
- the present application discloses an electronic device, including a processor and a memory; wherein the processor implements the aforementioned cross-camera multi-target tracking method when executing a computer program stored in the memory.
- the present application discloses a non-volatile readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned multi-target tracking method across cameras is implemented.
- this application obtains video frames captured by several cameras; determines the first type of target in the video frame that is located in the overlapping visual space area between different cameras and has the same shooting time, and performs deduplication processing on the first type of target to obtain Remaining targets after deduplication; based on chronological order, classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas to obtain the remaining targets after each deduplication
- the corresponding path trajectories of the target and each second type of target in non-overlapping visual space areas are examples of the target and each second type of target in non-overlapping visual space areas.
- this application deduplicates the first type of targets that overlap the visual space area and have the same shooting time, so as to connect the same target captured by different cameras at the same shooting time and complete the airspace matching of the target; in After the target is deduplicated, the remaining targets after deduplication taken at different shooting times and the second type of targets in non-overlapping visual space areas are classified to obtain the corresponding path trajectories to complete the time domain matching of the target; this The process does not require matching of target trajectories in different cameras, but deduplication and classification of targets to obtain cross-camera target trajectories. Performance degradation will not be caused by trajectory matching errors, and cross-camera tracking can be achieved more accurately. Multiple target tracking.
- Figure 1 is a schematic diagram of the existing cross-camera multi-target tracking method
- Figure 2 is a flow chart of a cross-camera multi-target tracking method provided by this application.
- Figure 3 is a schematic diagram of the input information of an airspace matcher for pedestrians provided by this application.
- Figure 4 is a schematic diagram of the output information of an airspace matcher for pedestrians provided by this application.
- Figure 5 is a schematic diagram of information stored in a target trajectory buffer for pedestrians provided by this application.
- Figure 6 is a flow chart of a specific cross-camera multi-target tracking method provided by this application.
- FIG. 7 is a schematic diagram of regional division provided by this application.
- Figure 8 is a flow chart of a specific cross-camera multi-target tracking method provided by this application.
- Figure 9 is a schematic diagram of the multi-target tracking process across cameras
- Figure 10 is a schematic diagram of the multi-target tracking process of a camera provided by this application.
- Figure 11 is a structural diagram of a camera multi-target tracking system provided by this application.
- Figure 12 is a schematic diagram of the workflow of the airspace matcher
- Figure 13 is a schematic diagram of the workflow of the time domain matcher
- Figure 14 is a structural diagram of a cross-camera multi-target tracking device provided by this application.
- Figure 15 is a structural diagram of an electronic device.
- cross-camera target tracking algorithms are usually implemented using a two-step superposition method: the first step is to track the target on a single camera to form a local trajectory; the second step is to use the classic tracklet-to-tracklet matching algorithm to track the single camera.
- the first step is to track the target on a single camera to form a local trajectory; the second step is to use the classic tracklet-to-tracklet matching algorithm to track the single camera.
- Several output results are matched and spliced; using this cross-camera target tracking method, single segment tracking and then trajectory matching will cause performance degradation due to trajectory matching errors.
- this application provides a cross-camera multi-target tracking solution, which can more accurately achieve cross-camera multi-target tracking.
- an embodiment of the present application discloses a cross-camera multi-target tracking method, which method includes:
- Step S11 Obtain video frames captured by several cameras.
- camera identifiers will be set for several cameras to distinguish different cameras.
- the camera identifiers can be represented by camera IDs (Identity documents).
- the representation of camera IDs includes but is not limited to numbers. ,letter.
- the detector coordinates the moving targets in different video frames based on the detection network to obtain the coordinates of the detection frame corresponding to the target in the corresponding video frame, and extract different features through the embedded feature extractor.
- Embedded features of moving targets in video frames it should be noted that embedded features are feature information used to distinguish moving targets.
- embedded features include but are not limited to pedestrian facial features and pedestrian clothing features.
- the detector can use classic target detection models such as Yolo (You Only Look Once) and FasterRCNN; the embedded feature extractor can be trained through metric learning using classic network structures such as ResNeSt and EfficientNet.
- the coordinates, feature information and camera identification corresponding to the moving target are integrated to obtain the original detection information corresponding to the moving target.
- Step S12 Determine the first type of targets in the video frame that are located in the overlapping visual space area between different cameras and have the same shooting time, and perform deduplication processing on the first type of targets to obtain the remaining targets after deduplication.
- the function of the airspace matcher is to match the same target captured by different cameras at the same time node, because for a certain target, it is in the process of crossing from the field of view of one camera to the field of view of another camera. , will be imaged under two cameras. Therefore, the purpose of this matcher is to classify these target samples and merge the same targets under different cameras, that is, to perform deduplication processing.
- the airspace matcher is used to perform deduplication processing on the first type of targets to connect the moving targets corresponding to different cameras to complete airspace matching.
- deduplication processing first input each object to the airspace matcher.
- the spatial matcher uses the coordinates in the original detection information to determine the moving targets located in the overlapping visual space areas between different cameras and with the same shooting time as the first type of targets, and then uses the embedded features, also That is, the feature information determines the first type of target that represents the same target, and then classifies the original detection information of the first type of target that represents the same target to complete the deduplication process of the first type of target to obtain the remaining targets after deduplication and
- the corresponding target detection information is the target detection information of each pedestrian after deduplication, that is, the output information of the airspace matcher.
- the internal and external parameters corresponding to several cameras are used to calculate the visual space area in the space, and the connection between the visual space area of the camera and the position coordinates in the corresponding video frame is established. Therefore, the original The coordinates in the detection information determine the location of the target in the area of visual space.
- the target detection information obtained by using the spatial matcher includes camera ID-coordinate pairs and embedded features.
- the embedded features in the target detection information are in matrix form.
- the pedestrians with pedestrian IDs 1 and 2 in Figure 3 are the same pedestrians captured by the cameras with camera IDs 1 and 2 respectively, and the pedestrians with pedestrian IDs 3 and 4 are respectively. It is the same pedestrian captured by the cameras with camera IDs 2 and 3. Therefore, during airspace matching, the pedestrians with pedestrian IDs 1 and 2 are deduplicated, and the pedestrians with pedestrian IDs 3 and 4 are deduplicated.
- Step S13 Based on chronological order, classify the remaining targets after deduplication and the second type of targets in non-overlapping visual space areas captured at different shooting times to obtain the remaining targets after deduplication and non-overlapping targets.
- the time domain matcher is used to compare the matching results output by the spatial domain matching with the characteristic information of the classified targets stored in the target trajectory buffer, thereby continuously updating the pedestrian trajectory of each frame. module.
- this application uses the time domain matcher to classify the airspace matching results obtained from the airspace matcher into the recorded targets and the historical detection information corresponding to the recorded targets in the pedestrian trajectory buffer based on time order, and obtain the corresponding targets.
- the spatial domain matching results include the remaining targets after deduplication and corresponding target detection information, and the second type of target in the non-overlapping visual space area and the corresponding target detection information.
- the target detection information of the pedestrian photographed at different shooting times for the pedestrian with pedestrian ID 1 is classified in chronological order and marked with timing IDs 1 and 2.
- the pedestrian with pedestrian ID 2 was photographed when the timing ID was 2.
- the pedestrian's embedded features and coordinates and other information are stored in the form of a dictionary.
- the first-level directory of the dictionary is the identification of each pedestrian, that is, the pedestrian ID.
- the second-level directory is the ID of the pedestrian's appearance in the sequence.
- the third-level directory is the camera ID of the pedestrian at that time.
- the query content is the coordinates of the pedestrian in this state. and embedded features.
- classifying the remaining targets after deduplication and the second type of targets in non-overlapping visual space areas captured at different shooting times means classifying the remaining targets after deduplication and the non-overlapping visual space areas.
- the target detection information corresponding to the second type of target is saved in the target trajectory buffer. It should be pointed out that because too much feature information is not helpful for target tracking, it is necessary to monitor each classified target in the target trajectory buffer.
- the classified duration corresponding to the target determine whether the classified duration is greater than the preset duration threshold, and if so, delete the characteristic information corresponding to the corresponding classified target.
- the process of deleting the feature information corresponding to the classified target can avoid the memory overflow problem of the target trajectory buffer. Specifically, when the preset duration threshold is between 15 seconds and 20 seconds, the performance of the target trajectory buffer can be guaranteed.
- this application obtains video frames captured by several cameras; determines the first type of target in the video frame that is located in the overlapping visual space area between different cameras and has the same shooting time, and performs deduplication processing on the first type of target to obtain Remaining targets after deduplication; based on chronological order, classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas to obtain the remaining targets after each deduplication
- the corresponding path trajectories of the target and each second type of target in non-overlapping visual space areas are examples of the target and each second type of target in non-overlapping visual space areas.
- this application deduplicates the first type of targets that overlap the visual space area and have the same shooting time, so as to connect the same target captured by different cameras at the same shooting time and complete the airspace matching of the target; in After the target is deduplicated, the remaining targets after deduplication taken at different shooting times and the second type of targets in non-overlapping visual space areas are classified to obtain the corresponding path trajectories to complete the time domain matching of the target; this The process does not require matching of target trajectories in different cameras, but deduplication and classification of targets to obtain cross-camera target trajectories. Performance degradation will not be caused by trajectory matching errors, and cross-camera tracking can be achieved more accurately. Multiple target tracking.
- an embodiment of the present application discloses a specific cross-camera multi-target tracking method, which method includes:
- Step S21 Obtain video frames captured by several cameras.
- step S21 For more specific processing of step S21, reference may be made to the corresponding content disclosed in the foregoing embodiments, and details will not be described again here.
- Step S22 Determine the characteristic information of different moving targets captured by different cameras at the same shooting time in the overlapping visual space area, and determine the first cosine distance between the characteristic information of the different moving targets.
- the feature information is the embedded feature in the corresponding original detection information.
- the division of overlapping visual space areas and non-overlapping visual space areas is based on the number of cameras that capture the corresponding area and the camera ID (Identity document). As shown in Figure 7, cameras 1, 2, 3, 4 Divide the visual space area into 11 areas. Among the 11 areas, 2, 4, 5, 6, 7, 8, and 10 are overlapping visual space areas, and 1, 3, 9, and 11 are non-overlapping visual space areas. It should be pointed out that when the target spans multiple areas in Figure 7 at the same time, the target is classified into the area visible to the most cameras.
- the characteristic information of different moving targets captured by different cameras at the same shooting time in the overlapping visual space area is determined, and the different movements are determined.
- the first cosine distance between the target’ s feature information. For example, when the target is located in overlapping space area 2, corresponding to camera 1 and camera 2, then the characteristic information of all moving targets of camera 1 and camera 2 in overlapping space area 2 is calculated, and when camera 1 and camera 2 are in overlapping space area 2
- the first cosine distance between the feature information of all targets in The first cosine distance between the characteristic information of all targets and the characteristic information of all targets in the overlapping space area 5 of camera 1, camera 2 and camera 3.
- the characteristic information of the target of the camera 1 in the overlapping space area 2 and the characteristics of the target of the camera 2 in the overlapping space area 2 can also be calculated.
- Step S23 Determine whether the first cosine distance satisfies the target preset condition. If so, determine that different moving targets are the same target to obtain the corresponding first-type target, and then perform deduplication processing on the first-type target to obtain the deduplicated target. remaining targets.
- determining whether the first cosine distance meets the target preset condition specifically includes: saving the first cosine distance corresponding to each group of different moving targets captured by different cameras at the same shooting time to the first preset distance matrix; wherein, the storage position of the first cosine distance in the first preset distance matrix is a position determined based on the identification number of the moving target corresponding to the cosine distance; respectively determine the distance between any two cameras in the preset distance matrix.
- the first preset condition is whether the first cosine distance is less than the first preset distance threshold
- the second preset condition is the first cosine distance Is it the minimum value among the corresponding row and column values?
- the row and column are the rows and columns between any two cameras mentioned above.
- the specific steps for judging that different moving targets are the same target are: performing cosine operation on all n moving targets captured by different cameras in area k at the same shooting time to obtain a [n*n] distance matrix D, and then perform distance masking on the moving targets under the same camera.
- the method is to set the distance value of the corresponding position in the distance matrix to infinity.
- all pairs of moving targets whose first cosine distances under any two different cameras in the distance matrix are smaller than the first preset distance threshold ⁇ are proposed.
- the two moving targets corresponding to the first cosine distance are the same target; for example, if the target is located in overlapping space area 2, corresponding to camera 1 and camera 2, then calculate the position of camera 1 and camera 2 in the overlapping space area.
- the characteristic information of all moving targets in 2 and the first cosine distance between the characteristic information of all targets in the overlapping space area 2 between camera 1 and camera 2; save the first cosine distance to the preset distance matrix, Set the first cosine distance between the same cameras to infinity, and find the first cosine distance that satisfies the first preset condition and the second preset condition from the first cosine distance between camera 1 and camera 2 , the moving targets corresponding to the cosine distance are judged to be the same target.
- the first cosine distance is set to infinity, find the first cosine distance that satisfies the first preset condition and the second preset condition from the first cosine distance between camera 1 and camera 2, and divide the cosine distance into
- the corresponding moving targets are judged to be the same target, and the first cosine distance that satisfies the first preset condition and the second preset condition is found from the first cosine distance between camera 1 and camera 3, and the cosine distance is
- the corresponding moving targets are judged to be the same target, and then the first cosine distance that satisfies the first preset condition and the second preset condition is found from the first cosine distance between camera 2 and camera 3, and the cosine distance is The corresponding moving targets are judged to be the same target.
- min(D[i,*]) and min(D[j,*]) are the minimum values of the corresponding row and column values under two different cameras.
- Step S24 Based on chronological order, classify the remaining targets after deduplication and the second type of targets in non-overlapping visual space areas captured at different shooting times to obtain each remaining target after deduplication and non-overlapping targets. The corresponding path trajectories of each second type of target in the overlapping visual space area.
- different moving targets are determined to be second type targets; it should be pointed out that the preset distance can be created when the moving target is located in an overlapping visual space area Matrix, the moving targets in the overlapping visual space area are the first type targets; when different moving targets are located in the non-overlapping visual space area, no preset distance matrix is created, and the moving targets in the non-overlapping visual space area are directly used as the second type targets. .
- this application obtains video frames captured by several cameras; determines the first type of target in the video frame that is located in the overlapping visual space area between different cameras and has the same shooting time, and performs deduplication processing on the first type of target to obtain Remaining targets after deduplication; based on chronological order, classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas to obtain the remaining targets after each deduplication
- the corresponding path trajectories of the target and each second type of target in non-overlapping visual space areas are examples of the target and each second type of target in non-overlapping visual space areas.
- this application uses feature information to deduplicate the first type of targets that overlap the visual space area and have the same shooting time, so as to connect the same target captured by different cameras at the same shooting time, and complete the airspace of the target.
- Matching the use of feature information can eliminate the problem of inaccurate matching caused by differences in target morphology, making the matching more accurate; after completing the deduplication process on the target, the remaining targets after deduplication and the non-overlapping visual space captured at different shooting times
- the second type of targets in the area are classified to obtain the corresponding path trajectories, and the time domain matching of the targets is completed; this process does not require matching of target trajectories in different cameras, but deduplication and classification of targets are performed to obtain the cross-
- the camera's target trajectory will not cause performance degradation due to trajectory matching errors, and can more accurately achieve multi-target tracking across cameras.
- this embodiment of the present application discloses a specific cross-camera multi-target tracking method, which method includes:
- Step S31 Obtain video frames captured by several cameras.
- step S31 For more specific processing of step S31, reference may be made to the corresponding content disclosed in the foregoing embodiments, and will not be described again here.
- Step S32 Determine the first type of target in the video frame that is located in the overlapping visual space area between different cameras and has the same shooting time, and perform deduplication processing on the first type of target to obtain the remaining targets after deduplication.
- step S32 For more specific processing of step S32, reference may be made to the corresponding content disclosed in the foregoing embodiments, and will not be described again here.
- Step S33 Determine the second cosine distance between the classified target and the unclassified target using the characteristic information of the classified target at the historical shooting time and the characteristic information of the unclassified target at the current shooting time; and Unclassified targets include remaining targets after deduplication and second-category targets that have not yet been classified.
- historical detection information including characteristic information of classified targets under historical shooting time is stored in the target trajectory buffer; thus, the module that continuously updates the pedestrian trajectory of each frame utilizes classified targets under historical shooting time.
- the characteristic information of the object and the characteristic information of the unclassified target at the current shooting time, determining the second cosine distance between the classified target and the unclassified target includes two methods.
- a feature matrix various feature information of unclassified targets corresponding to different cameras at the current shooting time is stored in a second feature matrix; the first feature matrix and the second feature matrix are used to perform cosine distance operations and are saved
- a third preset distance matrix of multiple cosine distances between various characteristic information of classified targets under different camera historical shooting times and various characteristic information of unclassified targets under the current shooting time; from the third preset Let the cosine distance with the smallest value be selected from the cosine distances in the distance matrix as the second cosine distance between the classified target and the unclassified target.
- the process of storing various characteristic information of classified targets at historical shooting times corresponding to different cameras into the first feature matrix can be a process of storing each classified target at historical shooting times corresponding to different cameras.
- Various feature information are bound to obtain multiple bound information, and the bound information is sequentially stored in the first feature matrix.
- the purpose of obtaining the bound information is to continuously store various characteristic information of the same classified target.
- it can be as follows: storing various feature information of the same classified target at the historical shooting time corresponding to different cameras into a third feature matrix to obtain multiple third feature matrices; integrating multiple third feature matrices to obtain the stored The first feature matrix of various feature information of classified targets.
- using the third feature matrix to store various feature information of the same classified target can ensure that various feature information of the same classified target is continuously stored.
- the specific details in this specific embodiment are: the historical detection information captured by all cameras at the historical shooting time for each classified target in the target trajectory buffer, and the corresponding The feature information in the historical detection information is integrated to construct a feature matrix to obtain the feature matrix FT i corresponding to each classified target.
- the size of each feature matrix is [m, d], where d is the feature dimension and m is the history.
- the number of targets i captured by all cameras during the shooting time; then integrate the feature matrices FT i corresponding to each classified target to obtain the feature matrix FT, where M ⁇ m i , which is captured by all cameras during the historical shooting time the number of all targets; then perform a cosine distance operation on the feature information corresponding to the unclassified targets output by the spatial matcher and the feature information in the feature matrix FT to obtain a distance matrix of size [M, N], where M is the number of all targets captured by all cameras at the historical shooting time in the target trajectory cache, and N is the number of all targets captured by all cameras that have not yet been classified as targets output by the airspace matcher; then, cache the target trajectory Extract the relevant positions index r and index h of the classified target r and the unclassified target h in the FT, and extract the minimum value of the relevant area in the distance matrix of size [M, N] and add Go to DT rh to construct a distance matrix DT with a size of [P, Q], where P
- the above-mentioned specific process of screening the second cosine distance can be: storing various feature information of classified targets at the historical shooting time corresponding to the same camera into the fourth feature matrix to obtain the number of cameras. Corresponding several fourth feature matrices; store various feature information of unclassified targets corresponding to the same camera at the current shooting time into the fifth feature matrix to obtain several fifth feature matrices corresponding to the number of cameras ; Use the fourth feature matrix and the fifth feature matrix corresponding to the same camera to perform cosine distance operations to obtain various feature information of classified targets at the historical shooting time corresponding to the same camera and unclassified targets at the current shooting time.
- a fourth preset distance matrix of cosine distances between various feature information to obtain several fourth preset distance matrices corresponding to the number of cameras; from a plurality of cosine distances in several fourth preset distance matrices The cosine distance with the smallest value is selected as the second cosine distance between the classified target and the unclassified target.
- the specific details in this specific embodiment are: extract the feature information from the historical detection information of each classified target in the same camera under the historical shooting time to obtain the feature matrix FT kl on each camera, where k is the camera ID, l is the ID of the classified target, and U feature matrices FH k are obtained by merging all classified targets under the same camera; based on all unclassified targets captured by the same camera, U feature matrices FH k are obtained , where U is the number of cameras; the feature matrix FT k and the feature matrix FH corresponding to the same camera are regarded as a set of matrix pairs, and each set of matrix pairs is subjected to cosine distance operation to obtain U feature matrices DG k .
- the target formula is:
- Step S34 Use the second cosine distance to determine whether the targets among the unclassified targets and the targets among the classified targets are the same target, and classify the unclassified targets based on the judgment result.
- the second cosine distance is used to determine whether the targets in the unclassified targets and the targets in the classified targets are the same target.
- the distance between the classified targets and the unclassified targets is The second cosine distance is stored in the second preset distance matrix; where the storage location of the second cosine distance in the second preset distance matrix is the classified target and the unclassified target corresponding to the second cosine distance.
- the position determined by the identification number respectively determine whether the second cosine distance in the second preset distance matrix satisfies the third preset condition and the fourth preset condition; the third preset condition is whether the second cosine distance is less than the Two preset distance thresholds, and the fourth preset condition is whether the second cosine distance is the minimum value among the corresponding row and column values; if the third preset condition and the fourth preset condition are met, then the objects in the target that have not yet been classified The target and the target among the classified targets are the same target. If the third preset condition and the fourth preset condition are not satisfied, the target among the unclassified targets and the target among the classified targets are not the same target.
- the second cosine distance that satisfies the third preset condition and the fourth preset condition is selected from the distance matrices DB and DT, and the second cosine distance that satisfies the third preset condition and the fourth preset condition is determined.
- One of the classified targets corresponding to the chord distance and one of the unclassified targets are the same target, and then the targets in the unclassified targets are classified into the targets in the classified targets in chronological order.
- the third preset condition is whether the second cosine distance is less than the second preset distance threshold
- the fourth preset condition is whether the second cosine distance is the minimum value among the corresponding row and column values.
- this application obtains video frames captured by several cameras; determines the first type of target in the video frame that is located in the overlapping visual space area between different cameras and has the same shooting time, and performs deduplication processing on the first type of target to obtain Remaining targets after deduplication; based on chronological order, classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas to obtain the remaining targets after each deduplication
- the corresponding path trajectories of the target and each second type of target in non-overlapping visual space areas are examples of the target and each second type of target in non-overlapping visual space areas.
- this application deduplicates the first type of targets that overlap the visual space area and have the same shooting time, so as to connect the same target captured by different cameras at the same shooting time and complete the airspace matching of the target; in After the target is deduplicated, the remaining targets after deduplication taken at different shooting times and the second type of targets in non-overlapping visual space areas are classified according to the characteristic information to obtain the corresponding path trajectory to complete the time domain of the target. Matching; classification based on feature information can eliminate the problem of inaccurate classification caused by differences in target morphology, making classification more accurate. This process does not require matching of target trajectories in different cameras, but deduplication and classification of targets to obtain cross-camera target trajectories. Performance degradation will not be caused by trajectory matching errors, and cross-camera implementation can be achieved more accurately. multi-target tracking.
- the DeepSort algorithm uses Kalman filter and Hungarian matching, combined with tools such as target detection and metric learning, to achieve target matching between adjacent frames under a single camera to achieve tracking;
- JDE Java Development Environment
- the target tracking system extracts target detection features and metric learning features at the same time, thus simplifying the training process of the algorithm;
- FairMOT realizes the feature mismatch between the detection problem and the target re-identification task, and abandons the traditional target detection training mode.
- using key point detection instead solves the problem of mismatch between the target detection center and the target movement center; CenterTrack also improves the accuracy of the tracking system by solving this mismatch problem.
- this application proposes a multi-target tracking method across cameras.
- it is a multi-target tracking process across cameras, and the movement trajectories of different pedestrians between different cameras are tracked, such as the process of pedestrian No. 2 moving from camera 1 to camera 3.
- Figure 10 it is a schematic diagram of the multi-target tracking process of the camera provided by this application.
- FIG. 11 is a system structure diagram of the multi-target tracking of the camera provided by this application.
- the system mainly includes a target detector 01 and an embedded feature extractor 02 , spatial domain matcher 03, time domain matcher 04 and target trajectory buffer 05.
- Figure 12 shows the workflow of the airspace matcher.
- the original detection information corresponding to the moving target is sent to the airspace matcher, a camera ID is randomly selected, that is, a video frame is randomly selected, and then a detection frame is selected from the video frame.
- the target area of the moving target in the visual space area is determined based on the coordinates corresponding to the detection frame of the moving target (that is, the moving target area allocation).
- the moving target is classified into the common area that can be photographed by the most cameras; if the target area is an overlapping visual space area, the corresponding distance matrix is calculated for the target area where the moving target is located, and the distance matrix is added to the same camera
- the first cosine distance between the moving targets under the distance is masked, that is, the self-distance is masked; then the target corresponding to the first cosine distance that meets the preset conditions is used as the first type of target, and the moving target is deduplicated.
- the target is regarded as the second type of target; all moving targets in the target area are deduplicated and then the moving targets in other areas in the video frame are deduplicated; all moving targets in the video frame are deduplicated and then deduplicated.
- Figure 13 shows the workflow of the time domain matcher. It first receives the airspace matching results sent from the airspace matcher, and then uses the characteristic information of unclassified targets in the airspace matching results and the classified targets in the target trajectory buffer library. Feature information, calculate the second cosine distance (that is, distance operation) between unclassified targets and classified targets, and compare the unclassified targets corresponding to the second cosine distance that meets the preset conditions. A target and a target among the classified targets are regarded as the same target, and the target detection information of the targets among the unclassified targets is classified into the targets among the classified targets according to the time sequence.
- the second cosine distance that is, distance operation
- an embodiment of the present application discloses a cross-camera multi-target tracking device, including:
- the video frame acquisition module 11 is configured to acquire video frames captured by several cameras
- the deduplication module 12 is configured to determine the first type of targets located in the overlapping visual space area between different cameras in the video frame and with the same shooting time, and perform deduplication processing on the first type of targets to obtain the remaining targets after deduplication;
- the classification module 13 is configured to classify the remaining targets after deduplication photographed at different shooting times and the second type of targets in non-overlapping visual space areas based on time order, so as to obtain each deduplication result.
- the corresponding path trajectories of the remaining targets and each second type of target in the non-overlapping visual space area are configured to classify the remaining targets after deduplication photographed at different shooting times and the second type of targets in non-overlapping visual space areas based on time order, so as to obtain each deduplication result.
- this application obtains video frames captured by several cameras; determines the first type of target in the video frame that is located in the overlapping visual space area between different cameras and has the same shooting time, and performs deduplication processing on the first type of target to obtain Remaining targets after deduplication; based on chronological order, classify the remaining targets after deduplication captured at different shooting times and the second type of targets in non-overlapping visual space areas to obtain the remaining targets after each deduplication
- the corresponding path trajectories of the target and each second type of target in non-overlapping visual space areas are examples of the target and each second type of target in non-overlapping visual space areas.
- this application deduplicates the first type of targets that overlap the visual space area and have the same shooting time, so as to connect the same target captured by different cameras at the same shooting time and complete the airspace matching of the target; in After the target is deduplicated, the remaining targets after deduplication taken at different shooting times and the second type of targets in non-overlapping visual space areas are classified to obtain the corresponding path trajectories to complete the time domain matching of the target; this The process does not require matching of target trajectories in different cameras, but deduplication and classification of targets to obtain cross-camera target trajectories. Performance degradation will not be caused by trajectory matching errors, and cross-camera tracking can be achieved more accurately. Multiple target tracking.
- Figure 15 is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the figure cannot be considered as any limitation on the scope of use of the present application.
- FIG. 15 is a schematic structural diagram of an electronic device 20 provided by an embodiment of the present application.
- the electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, an input and output interface 24, a communication interface 25 and a communication bus 26.
- the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps of the cross-camera multi-target tracking method disclosed in any of the foregoing embodiments.
- the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20;
- the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be applicable Any communication protocol of the technical solution of this application is not specifically limited here;
- the input and output interface 24 is used to obtain external input data or output data to the external world, and its specific interface type can be selected according to specific application needs. Here Not specifically limited.
- the memory 22, as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.
- the memory 22 can include a random access memory as a running memory and a non-volatile memory for external memory storage.
- the storage resources on the memory include operating system 221, computer program 222, etc., and the storage method can be short-term storage or permanent storage.
- the operating system 221 is used to manage and control each hardware device and computer program 222 on the electronic device 20 on the source host.
- the operating system 221 can be Windows, Unix, Linux, etc.
- the computer program 222 may further include computer programs that can be used to complete other specific tasks.
- the input and output interface 24 may specifically include, but is not limited to, a USB interface, a hard disk reading interface, a serial interface, a voice input interface, a fingerprint input interface, etc.
- embodiments of the present application also disclose a non-volatile readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned cross-camera multi-target tracking method is implemented.
- the non-volatile readable storage media mentioned here include random access memory (Random Access Memory, RAM), memory, read-only memory (Read-Only Memory, ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, magnetic disk or optical disk or any other form of storage medium known in the technical field.
- RAM Random Access Memory
- ROM read-only memory
- electrically programmable ROM electrically erasable programmable ROM
- register hard disk
- RAM random access memory
- ROM read-only memory
- electrically programmable ROM electrically erasable programmable ROM
- registers hard disks, removable disks, CD-ROMs, or anywhere in the field of technology. any other known form of storage media.
- This application performs deduplication processing on first-category targets that overlap visual space areas and have the same shooting time, so as to connect the same target captured by different cameras at the same shooting time, and complete the airspace matching of the target; after completing the deduplication of the target
- the remaining targets after deduplication and the second type of targets in non-overlapping visual space areas captured at different shooting times are classified to obtain corresponding path trajectories to complete the time domain matching of targets; this process does not need to be performed.
- the targets are deduplicated and classified to obtain target trajectories across cameras. Performance degradation will not be caused by trajectory matching errors, and multi-target tracking across cameras can be achieved more accurately.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
本申请公开了一种跨摄像头的多目标追踪方法、装置、设备及介质,涉及人工智能领域,该方法包括:获取若干摄像头拍摄到的视频帧;确定视频帧中位于不同所述摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。由此可见,本申请不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,能够更精准地实现跨摄像头的多目标追踪。
Description
相关申请的交叉引用
本申请要求于2022年06月06日提交中国专利局、申请号为202210627280.3、申请名称“一种跨摄像头的多目标追踪方法、装置、设备及介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本申请涉及人工智能领域,特别涉及一种跨摄像头的多目标追踪方法、装置、设备及介质。
当前,目标追踪是人工智能机器视觉领域最具应用价值的研究方向之一,通常情况下,目标追踪课题分为两个子类:单目标追踪(SOT,Single Object Tracking)和多目标追踪(MOT,Multi Object Tracking):单目标追踪侧重于对某个特定目标的追踪或者较简单场景的追踪问题,此时场景中目标区域内可见的目标数量极少;多目标追踪使用范围较广泛,常用于普通场景下多个目标的同时追踪。当前,多目标追踪问题更多被涉及,如自动驾驶数据集KITTI,同时包括车辆和行人的追踪标注;MOT-Challenge数据集,是一个专注于行人追踪的目标追踪数据集;PANDA数据集,聚焦超大规模场景下的行人追踪问题,场景更复杂,行人分布更广泛,问题更难。但是,这些数据集通常将追踪问题设定在同一个摄像头下。然而在真实的使用场景下,例如违法分子追踪、失踪人口搜索、违规车辆追查等公安和交通场景下,目标的行动轨迹通常是跨摄像头的。
对于跨摄像头目标追踪算法,如图1所示,为针对行人的目标追踪,现有方法通常采用两步叠加的方法来实现:第一步在单摄像头进行目标追踪,从而形成局部轨迹;第二步采用经典的tracklet-to-tracklet匹配算法对单摄像头追踪的若干输出结果进行匹配和拼接;利用此种跨摄像头目标追踪方法,单一的进行片段追踪再进行轨迹匹配,会造成由于轨迹匹配失误造成的性能衰减。
综上如何更加精准地实现跨摄像头的多目标追踪是当前亟待解决的问题。
发明内容
有鉴于此,本申请的目的在于提供一种跨摄像头的多目标追踪方法,能够更加精准地实现跨摄像头的多目标追踪。其具体方案如下:
第一方面,本申请公开了一种跨摄像头的多目标追踪方法,包括:
获取若干摄像头拍摄到的视频帧;
确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;
基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。
可选的,确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,包括:
确定重叠视觉空间区域上由不同摄像头在同一拍摄时间下拍摄到的不同运动目标的特征信息;
确定不同运动目标的特征信息之间的第一余弦距离;
判断第一余弦距离是否满足目标预设条件,若是则判定不同运动目标均为同一目标,以得到相应的第一类目标。
可选的,判断第一余弦距离是否满足目标预设条件,包括:
将不同摄像头在同一拍摄时间下拍摄到的各组不同运动目标对应的第一余弦距离保存至第一预设距离矩阵;其中,第一余弦距离在预设距离矩阵中的存储位置为基于第一余弦距离对应的运动目标的标识号确定的位置;
分别判断第一预设距离矩阵中任意两个不同摄像头之间的第一余弦距离是否满足第一预设条件和第二预设条件;第一预设条件为第一余弦距离是否小于第一预设距离阈值,第二预设条件为第一余弦距离是否为对应的行列数值中的最小值。
可选的,基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,包括:
利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定已归类目标和还未归类目标之间的第二余弦距离;还未归类目标包括还未归类的去重后剩余目标和第二类目标;
利用第二余弦距离判断还未归类目标中的目标与已归类目标中的目标是否为同一目标,并基于判断结果对还未归类目标进行归类。
可选的,利用第二余弦距离判断还未归类目标中的目标与已归类目标中的目标是否为同一目标,包括:
将已归类目标和还未归类目标之间的第二余弦距离存放至第二预设距离矩阵;其中,第二余弦距离在第二预设距离矩阵中的存储位置为基于第二余弦距离对应的已归类目标和未归类目标的标识号确定的位置;
分别判断第二预设距离矩阵中的第二余弦距离是否满足第三预设条件和第四预设条件;第三预设条件为第二余弦距离是否小于第二预设距离阈值,第四预设条件为第二余弦距离是否为对应的行列数值中的最小值;
若满足第三预设条件和第四预设条件,则还未归类目标中的目标与已归类目标中的目标为同一目标,若不满足第三预设条件和第四预设条件,则还未归类目标中的目标与已归类目标中的目标不为同一目标。
可选的,利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定已归类目标和还未归类目标之间的第二余弦距离,包括:
分别计算历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到相应的多个余弦距离;
从余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
可选的,分别计算历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到相应的多个余弦距离,包括:
将不同摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第一特征矩阵中,并将不同摄像头对应的当前拍摄时间下还未归类目标的各种特征信息存放至第二特征矩阵中;
利用第一特征矩阵和第二特征矩阵进行余弦距离运算,得到保存有不同摄像头历史拍摄时间下已归类目标的各种特征信息和当前拍摄时间下还未归类目标的各种特征信息之间的多个余弦距离的第三预设距离矩阵;
相应的,从余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离,包括:
从第三预设距离矩阵中的余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
可选的,将不同摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第一特征矩阵中,包括:
将不同摄像头对应的历史拍摄时间下同一已归类目标的各种特征信息进行绑定以得到多个绑定后信息,并将绑定后信息依次存放至第一特征矩阵中。
可选的,将不同摄像头对应的历史拍摄时间下同一已归类目标的各种特征信息进行绑定以得到多个绑定后信息,并将绑定后信息依次存放至第一特征矩阵中,包括:
将不同摄像头对应的历史拍摄时间下每个已归类目标的各种特征信息存放至第三特征矩阵,以得到多个第三特征矩阵;
整合多个第三特征矩阵以得到存放有已归类目标的各种特征信息的第一特征矩阵。
可选的,利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定已归类目标和还未归类目标之间的第二余弦距离,包括:
分别计算同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到每个摄像头对应的已归类目标的各种特征信息与还未归类目标的各种特征信息之间的多个余弦距离;
从多个余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
可选的,分别计算同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到每个摄像头对应的已归类目标的各种特征信息与还未归类目标的各种特征信息之间的多个余弦距离,包括:
将同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第四特征矩阵中,以得到与摄像头的数量对应的若干个第四特征矩阵;
将同一摄像头对应的当前拍摄时间下还未归类目标的各种特征信息存放至第五特征矩阵中,以得到与摄像头的数量对应的若干个第五特征矩阵;
利用同一摄像头对应的第四特征矩阵和第五特征矩阵进行余弦距离运算,得到保存有同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离的第四预设距离矩阵,以得到与摄像头的数量对应的若干个第四预设距离矩阵;
相应的,从多个余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离,包括:
从若干个第四预设距离矩阵中的多个余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
可选的,基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,还包括:
监测每个已归类目标对应的已归类时长;
判断已归类时长是否大于预设时长阈值,如果是则将相应的已归类目标对应的特征信息进行删除。
第二方面,本申请公开了一种跨摄像头的多目标追踪装置,包括:
视频帧获取模块,设置为获取若干摄像头拍摄到的视频帧;
去重模块,设置为确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;
归类模块,设置为基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。
第三方面,本申请公开了一种电子设备,包括处理器和存储器;其中,处理器执行存储器中保存的计算机程序时实现前述公开的跨摄像头的多目标追踪方法。
第四方面,本申请公开了一种非易失性可读存储介质,用于存储计算机程序;其中,计算机程序被处理器执行时实现前述公开的跨摄像头的多目标追踪方法。
可见,本申请获取若干摄像头拍摄到的视频帧;确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。由此可见,本申请对重叠视觉空间区域的并且拍摄时间相同的第一类目标进行去重处理,以对不同摄像头在相同拍摄时间拍摄到的同一目标进行目标衔接,完成目标的空域匹配;在对目标完成去重处理后,将不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类得到相应的路径轨迹,完成目标的时域匹配;此过程不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,不会由于轨迹匹配失误而造成性能衰减,能够更精准地实现跨摄像头的多目标追踪。
此处的附图被并入说明书中并构成本说明书的一部分,示出了符合本申请的实施例,并与说明书一起用于解释本申请实施例的原理。
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据提供的附图获得其他的附图。
图1为现有的跨摄像头的多目标追踪方法示意图;
图2为本申请提供的一种跨摄像头的多目标追踪方法流程图;
图3为本申请提供的一种针对行人的空域匹配器输入信息示意图;
图4为本申请提供的一种针对行人的空域匹配器输出信息示意图;
图5为本申请提供的一种针对行人的目标轨迹缓存器存储信息示意图;
图6为本申请提供的一种具体的跨摄像头的多目标追踪方法流程图;
图7为本申请提供的一种区域划分示意图;
图8为本申请提供的一种具体的跨摄像头的多目标追踪方法流程图;
图9为跨摄像头的多目标追踪过程示意图;
图10为本申请提供的一种摄像头的多目标追踪流程示意图;
图11为本申请提供的一种摄像头的多目标追踪系统结构图;
图12为空域匹配器的工作流程示意图;
图13为时域匹配器的工作流程示意图;
图14为本申请提供的一种跨摄像头的多目标追踪装置结构图;
图15为一种电子设备结构图。
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。
当前对于跨摄像头目标追踪算法,通常采用两步叠加的方法来实现:第一步在单摄像头进行目标追踪,从而形成局部轨迹;第二步采用经典的tracklet-to-tracklet匹配算法对单摄像头追踪的若干输出结果进行匹配和拼接;利用此种跨摄像头目标追踪方法,单一的进行片段追踪再进行轨迹匹配,会造成由于轨迹匹配失误造成的性能衰减。
为了克服上述问题,本申请提供了一种跨摄像头的多目标追踪方案,能够更加精准地实现跨摄像头的多目标追踪。
参见图2所示,本申请实施例公开了一种跨摄像头的多目标追踪方法,该方法包括:
步骤S11:获取若干摄像头拍摄到的视频帧。
本申请实施例中,在获取若干摄像头拍摄到的视频帧之前,会为若干摄像头设置摄像头标识以区分不同摄像头,摄像头标识可用摄像头ID(Identity document)表示,摄像头ID的表示方式包括但不限于数字、字母。获取若干摄像头拍摄到的视频帧之后,通过检测器基于检测网络对不同视频帧中的运动目标进行坐标定位,得到目标对应的检测框在对应的视频帧中的坐标,通过嵌入特征提取器提取不同视频帧中的运动目标的嵌入特征;需要指出的是,嵌入特征为用于区分运动目标的特征信息,当运动目标为行人时,嵌入特征包括但不限于行人面部特征,行人衣物特征。
需要指出的实时,检测器可采用Yolo(You Only Look Once)、FasterRCNN等经典目标检测模型;嵌入特征提取器可采用ResNeSt、EfficientNet等经典网络结构通过度量学习训练得到。
本申请实施例中,进行坐标定位和嵌入特征提取后,综合运动目标对应的坐标、特征信息和摄像头标识,以获取运动目标对应的原始检测信息,对于目标a,原始检测信息可表示为F
a={“摄像头ID”:1,“坐标”:[x
1,y
1,x
1,y
1],“嵌入特征”:f
d},如图3,为不同摄像头拍摄的各行人的原始检测信息,也即空域匹配器的输入信息。
步骤S12:确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标。
本申请实施例中,空域匹配器的作用是将同一时间节点下不同摄像头拍到的同一目标进行匹配,由于对于某一目标来说,他在从一个摄像头视野跨越到另一个摄像头视野的过程中,会在两个摄像头下成像,因此,本匹配器的目的是将这些目标样本进行归类,将不同摄像头下相同的目标进行合并,也即进行去重处理。
本申请实施例中,利用空域匹配器对第一类目标进行去重处理,以进行不同摄像头对应的运动目标的目标 衔接,完成空域匹配,在进行去重处理时,首先向空域匹配器输入各运动目标的原始检测信息,空域匹配器利用原始检测信息中的坐标,确定出位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的运动目标作为第一类目标,然后利用嵌入特征,也即特征信息确定出表示同一目标的第一类目标,然后将表示同一目标的第一类目标的原始检测信息进行归类,以完成对第一类目标的去重处理得到去重后剩余目标以及对应的目标检测信息,如图4所示为去重后各行人的目标检测信息,也即空域匹配器的输出信息。
需要指出的是,利用若干摄像头对应的内参和外参计算出在空间中的视觉空间区域,并建立摄像头的视觉空间区域与相应的视频帧中的位置坐标之间的联系,因此,能够利用原始检测信息中的坐标确定目标在视觉空间区域中的位置。
需要指出的是,利用空域匹配器得到的目标检测信息包括摄像头ID-坐标对,嵌入特征两个信息,其中,目标检测信息中的嵌入特征为矩阵形式,例如,目标a在摄像头1和摄像头2下同时拍摄到,目标检测信息可表示为G
a={“摄像头ID-坐标”:[[1,[x
11,y
11,x
12,y
12]],[2,[x
21,y
21,x
22,y
22]]],“嵌入特征”:[f
d1,f
d2]}。如图4所示,为空域匹配后各行人的目标检测信息。需要指出的是,如图3和图4所示,图3中行人ID为1和2的行人分别为摄像头ID为1和2的摄像头拍摄到的同一行人,行人ID为3和4的行人分别为摄像头ID为2和3的摄像头拍摄到的同一行人,因此在空域匹配时,将行人ID为1和2的行人进行去重,将行人ID为3和4的行人进行去重。
步骤S13:基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。
本申请实施例中,时域匹配器是用来将空域匹配输出的匹配结果同存储在目标轨迹缓存器中存储的已归类目标的特征信息进行比对,从而持续更新每一帧的行人轨迹的模块。具体的,本申请利用时域匹配器基于时间顺序将从空域匹配器得到的空域匹配结果归类到行人轨迹缓存器中的已记录目标和已记录目标对应的历史检测信息中,得到对应的目标轨迹;空域匹配结果包括去重后剩余目标以及对应的目标检测信息和非重叠视觉空间区域上的第二类目标以及对应的目标检测信息。如图5所示,为目标轨迹缓存器中存储的时域匹配后基于时间顺序归类的行人以及对应的历史检测信息,也即目标轨迹缓存器中的存储信息。图5中,将行人ID为1的行人在不同拍摄时间拍摄的行人的目标检测信息按照时间顺序进行归类,并标注了时序ID1和2,行人ID为2的行人是在时序ID为2时拍摄到的行人,将行人ID为2的行人进行归类,并标注时序ID为2。
需要指出的是,当目标为行人时,目标轨迹缓存器中记载的内容如图5所示,行人的嵌入特征和坐标等信息通过字典的形式进行存储。字典的一级目录为每个行人的标识,也即行人ID,二级目录为该行人出现的时序ID,三级目录为该行人在该时序的摄像机ID,查询内容为该状态下行人的坐标和嵌入特征。
本申请实施例中,对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,是指将去重后剩余目标以及非重叠视觉空间区域上的第二类目标对应的目标检测信息保存到目标轨迹缓存器中,需要指出的是,由于,过多的特征信息对目标追踪没有帮助,因此需要监测目标轨迹缓存器中每个已归类目标对应的已归类时长;判断已归类时长是否大于预设时长阈值,如果是则将相应的已归类目标对应的特征信息进行删除。对已归类目标对应的特征信息进行删除的过程能够规避目标轨迹缓存器的内存溢出问题。具体的,预设时长阈值在15秒到20秒之间时,可以保证目标轨迹缓存器的性能。
可见,本申请获取若干摄像头拍摄到的视频帧;确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。由此可见,本申请对重叠视觉空间区域的并且拍摄时间相同的第一类目标进行去重处理,以对不同摄像头在相同拍摄时间拍摄到的同一目标进行目标衔接,完成目标的空域匹配;在对目标完成去重处理后,将不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类得到相应的路径轨迹,完成目标的时域匹配;此过程不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,不会由于轨迹匹配失误而造成性能衰减,能够更精准地实现跨摄像头的多目标追踪。
参见图6所示,本申请实施例公开了一种具体的跨摄像头的多目标追踪方法,该方法包括:
步骤S21:获取若干摄像头拍摄到的视频帧。
其中,关于步骤S21的更加具体的处理过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。
步骤S22:确定重叠视觉空间区域上由不同摄像头在同一拍摄时间下拍摄到的不同运动目标的特征信息,确定不同运动目标的特征信息之间的第一余弦距离。
本申请实施例中,特征信息为相应的原始检测信息中的嵌入特征。
本申请实施例中,对于重叠视觉空间区域与非重叠视觉空间区域的划分是基于拍摄到相应区域的摄像头个数和摄像头ID(Identity document),如图7所示,摄像头1、2、3、4将视觉空间区域分为11个区域,11个区域中2、4、5、6、7、8、10为重叠视觉空间区域,1、3、9、11为非重叠视觉空间区域。需要指出的是当目标同时跨越图7中的多个区域时,将目标归类为最多摄像头可见的区域。
本申请实施例中,当根据运动目标的坐标判断出目标位于重叠视觉空间区域后,确定重叠视觉空间区域上由不同摄像头在同一拍摄时间下拍摄到的不同运动目标的特征信息,并确定不同运动目标的特征信息之间的第一余弦距离。例如,当目标位于重叠空间区域2时,对应摄像头1和摄像头2,则计算摄像头1和摄像头2在重叠空间区域2中全部运动目标的特征信息,与,摄像头1和摄像头2在重叠空间区域2中全部目标的特征信息之间的第一余弦距离;当目标位于重叠空间区域5时,对应摄像头1、摄像头2和摄像头3,则计算摄像头1、摄像头2和摄像头3在重叠空间区域5中的全部目标的特征信息,与,摄像头1、摄像头2和摄像头3在重叠空间区域5中的全部目标的特征信息之间的第一余弦距离。
需要指出的是,当目标位于重叠空间区域2时,对应摄像头1和摄像头2,也可以计算摄像头1在重叠空间区域2中的目标的特征信息与摄像头2在重叠空间区域2中的目标的特征信息之间的第一余弦距离。
步骤S23:判断第一余弦距离是否满足目标预设条件,若是则判定不同运动目标均为同一目标,以得到相应的第一类目标,然后对第一类目标进行去重处理,得到去重后剩余目标。
本申请实施例中,判断第一余弦距离是否满足目标预设条件具体包括:将不同摄像头在同一拍摄时间下拍摄到的各组不同运动目标对应的第一余弦距离保存至第一预设距离矩阵;其中,第一余弦距离在第一预设距离矩阵中的存储位置为基于余弦距离对应的运动目标的标识号确定的位置;分别判断预设距离矩阵中任意两个摄像头之间的第一余弦距离是否满足第一预设条件和第二预设条件;第一预设条件为第一余弦距离是否小于第一预设距离阈值,第二预设条件为第一余弦距离是否为对应的行列数值中的最小值,行列为上述任意两个摄像头 之间的行列。
本申请实施例中,判断不同运动目标均为同一目标的具体步骤为:对区域k的不同摄像头在相同拍摄时间拍摄到的全部的n个运动目标做余弦运算可得一个[n*n]的距离矩阵D,之后,对其中同摄像头下的运动目标进行距离屏蔽,做法是将距离矩阵中对应位置的距离值置为无穷大。最后,将距离矩阵中任意两个不同摄像头下的所有第一余弦距离小于第一预设距离阈值μ的运动目标对提出来,若该第一余弦距离为任意两个不同摄像头下的该行与该列最小,则该第一余弦距离对应的两个运动目标为同一目标;例如,目标位于重叠空间区域2,对应摄像头1和摄像头2,则计算摄像头1和摄像头2在重叠空间区域2中全部运动目标的特征信息,与,摄像头1和摄像头2在重叠空间区域2中全部目标的特征信息之间的第一余弦距离;将第一余弦距离保存至预设距离矩阵中,将相同摄像头之间的第一余弦距离设置为无穷大,从摄像头1和摄像头2之间的第一余弦距离中找出满足第一预设条件和第二预设条件的第一余弦距离,将该余弦距离对应的运动目标判断为同一目标。
需要指出的是,当运动目标位于重叠空间区域5时,对应摄像头1、摄像头2和摄像头3,计算摄像头1、摄像头2和摄像头3在重叠空间区域5中的全部运动目标的特征信息,与,摄像头1、摄像头2和摄像头3在重叠空间区域5中的全部运动目标的特征信息之间的第一余弦距离之后,将第一余弦距离保存至预设距离矩阵中,将相同摄像头之间的第一余弦距离设置为无穷大,从摄像头1和摄像头2之间的第一余弦距离中找出满足第一预设条件和第二预设条件的第一余弦距离,将该余弦距离对应的运动目标判断为同一目标,并从摄像头1和摄像头3之间的第一余弦距离中找出满足第一预设条件和第二预设条件的第一余弦距离,将该余弦距离对应的运动目标判断为同一目标,然后从摄像头2和摄像头3之间的第一余弦距离中找出满足第一预设条件和第二预设条件的第一余弦距离,将该余弦距离对应的运动目标判断为同一目标。
需要指出的是,若距离矩阵D中,两个不同摄像头下的第i行第j列的距离值d
ij小于第一预设距离阈值μ,且d
ij=min(D[i,*])=min(D[j,*]),则第i个运动目标与第j个运动目标为同一目标。其中,min(D[i,*])和min(D[j,*])为两个不同摄像头下相应行列数值的最小值。
步骤S24:基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。
本申请实施例中,若判定第一余弦距离不满足目标预设条件,则判定不同运动目标为第二类目标;需要指出的是,可以在运动目标位于重叠视觉空间区域时创建预设距离矩阵,重叠视觉空间区域中的运动目标为第一类目标;而当不同运动目标位于非重叠视觉空间区域时不创建预设距离矩阵,直接将非重叠视觉空间区域的运动目标作为第二类目标。
可见,本申请获取若干摄像头拍摄到的视频帧;确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。由此可见,本申请利用特征信息对重叠视觉空间区域的并且拍摄时间相同的第一类目标进行去重处理,以对不同摄像头在相同拍摄时间拍摄到的同一目标进行目标衔接,完成目标的空域匹配;利用特征信息可以排除目标形态差异导致的匹配失准的问题,使匹配更加准确;在对目标完成去重处理后,将不同拍摄时间下拍摄到的去重后剩余目标以及非 重叠视觉空间区域上的第二类目标进行归类得到相应的路径轨迹,完成目标的时域匹配;此过程不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,不会由于轨迹匹配失误而造成性能衰减,能够更精准地实现跨摄像头的多目标追踪。
参见图8所示,本申请实施例公开了一种具体的跨摄像头的多目标追踪方法,该方法包括:
步骤S31:获取若干摄像头拍摄到的视频帧。
其中,关于步骤S31的更加具体的处理过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。
步骤S32:确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标。
其中,关于步骤S32的更加具体的处理过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。
步骤S33:利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定已归类目标和还未归类目标之间的第二余弦距离;还未归类目标包括还未归类的去重后剩余目标和第二类目标。
本申请实施例中,历史拍摄时间下已归类目标的包括特征信息的历史检测信息存储在目标轨迹缓存器中;从而持续更新每一帧的行人轨迹的模块利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定已归类目标和还未归类目标之间的第二余弦距离包括两种方法。
分别计算历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到相应的多个余弦距离;从余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离,具体的,将不同摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第一特征矩阵中,并将不同摄像头对应的当前拍摄时间下还未归类目标的各种特征信息存放至第二特征矩阵中;利用第一特征矩阵和第二特征矩阵进行余弦距离运算,得到保存有不同摄像头历史拍摄时间下已归类目标的各种特征信息和当前拍摄时间下还未归类目标的各种特征信息之间的多个余弦距离的第三预设距离矩阵;从第三预设距离矩阵中的余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
需要指出的是,将不同摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第一特征矩阵中的过程,可以为将不同摄像头对应的历史拍摄时间下每个已归类目标的各种特征信息进行绑定以得到多个绑定后信息,并将绑定后信息依次存放至第一特征矩阵中。其中,得到绑定后信息的目的是将同一已归类目标的各种特征信息进行连续存放。具体可以为:将不同摄像头对应的历史拍摄时间下同一已归类目标的各种特征信息存放至第三特征矩阵,以得到多个第三特征矩阵;整合多个第三特征矩阵以得到存放有已归类目标的各种特征信息的第一特征矩阵。其中,利用第三特征矩阵存放同一已归类目标的各种特征信息,可以确保同一已归类目标的各种特征信息连续存放。
需要指出的是,该具体实施例中的具体细节为:将目标轨迹缓存器中的每个已归类目标在历史拍摄时间下所有摄像头拍摄到的历史检测信息,将每个已归类目标对应的历史检测信息中的特征信息进行整合构建特征矩阵以得到每个已归类目标对应的特征矩阵FT
i,每个特征矩阵大小为[m,d],其中,d为特征维度,m为历史拍摄时间下所有摄像头拍摄到的目标i的个数;然后将每个已归类目标对应的特征矩阵FT
i进行整合得到特征矩阵FT,其中M=∑m
i,为历史拍摄时间下所有摄像头拍摄到的所有目标的个数;然后将空域匹 配器输出还未归类目标对应的特征信息与特征矩阵FT中的特征信息进行余弦距离运算,得到大小为[M,N]的距离矩阵,其中M为目标轨迹缓存器中历史拍摄时间下所有摄像头拍摄到的所有目标的个数,N为空域匹配器输出还未归类目标所有摄像头下拍摄到的所有目标的个数;然后,将目标轨迹缓存器中的已归类目标r和还未归类目标h在FT中相关的位置index
r和index
h提取出来,并将大小为[M,N]的距离矩阵中相关区域的最小值提取出来添加到DT
rh中以构建大小为[P,Q]的距离矩阵DT,其中,P为目标轨迹缓存器中的实际目标个数,Q为还未归类目标中的实际行人个数;其中,DT中存储的为第二余弦距离;其中,DT
rh为已归类目标r和还未归类目标h之间的距离矩阵。
分别计算同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到每个摄像头对应的已归类目标的各种特征信息与还未归类目标的各种特征信息之间的多个余弦距离;从多个余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
需要指出的是,上述筛选第二余弦距离的具体过程可以为:将同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第四特征矩阵中,以得到与摄像头的数量对应的若干个第四特征矩阵;将同一摄像头对应的当前拍摄时间下还未归类目标的各种特征信息存放至第五特征矩阵中,以得到与摄像头的数量对应的若干个第五特征矩阵;利用同一摄像头对应的第四特征矩阵和第五特征矩阵进行余弦距离运算,得到保存有同一摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离的第四预设距离矩阵,以得到与摄像头的数量对应的若干个第四预设距离矩阵;从若干个第四预设距离矩阵中的多个余弦距离中筛选出数值最小的余弦距离作为已归类目标和还未归类目标之间的第二余弦距离。
需要指出的是,该具体实施例中的具体细节为:取出历史拍摄时间下同一摄像头中每个已归类目标的历史检测信息中的特征信息,以得到各个摄像头上的特征矩阵FT
kl,其中k为摄像头ID,l为已归类目标的ID,合并同一摄像头下所有已归类目标得到U个特征矩阵FT
k;根据同一摄像头拍摄的所有还未归类目标,得到U个特征矩阵FH
k,其中,U为摄像头个数;将对应同一摄像头的特征矩阵FT
k和特征矩阵FH作为一组矩阵对,每组矩阵对进行余弦距离运算得到U个特征矩阵DG
k,从不同特征矩阵DG
k中将目标轨迹缓存器中的已归类目标r和还未归类目标h之间的余弦距离提取处理,并根据目标公式找到最小的余弦距离作为第二余弦距离添加到距离矩阵DB中;目标公式为:
步骤S34:利用第二余弦距离判断还未归类目标中的目标与已归类目标中的目标是否为同一目标,并基于判断结果对还未归类目标进行归类。
本申请实施例中利用第二余弦距离判断还未归类目标中的目标与已归类目标中的目标是否为同一目标,具体为:将已归类目标和还未归类目标之间的第二余弦距离存放至第二预设距离矩阵;其中,第二余弦距离在第二预设距离矩阵中的存储位置为基于第二余弦距离对应的已归类目标和未归类目标的标识号确定的位置;分别 判断第二预设距离矩阵中的第二余弦距离是否满足第三预设条件和第四预设条件;第三预设条件为第二余弦距离是否小于第二预设距离阈值,第四预设条件为第二余弦距离是否为对应的行列数值中的最小值;若满足第三预设条件和第四预设条件,则还未归类目标中的目标与已归类目标中的目标为同一目标,若不满足第三预设条件和第四预设条件,则还未归类目标中的目标与已归类目标中的目标不为同一目标。具体实施例中,从距离矩阵DB和DT中筛选出满足第三预设条件和第四预设条件的第二余弦距离,判断满足第三预设条件和第四预设条件的第二余弦距离对应的已归类目标中的一个目标和还未归类目标中的一个目标为同一目标,然后将还未归类目标中的目标按照时间顺序归类到已归类目标中的目标中;其中,第三预设条件为第二余弦距离是否小于第二预设距离阈值,第四预设条件为第二余弦距离是否为对应的行列数值中的最小值。
可见,本申请获取若干摄像头拍摄到的视频帧;确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。由此可见,本申请对重叠视觉空间区域的并且拍摄时间相同的第一类目标进行去重处理,以对不同摄像头在相同拍摄时间拍摄到的同一目标进行目标衔接,完成目标的空域匹配;在对目标完成去重处理后,根据特征信息将不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类得到相应的路径轨迹,完成目标的时域匹配;根据特征信息进行归类可以排除目标形态差异导致的归类失准的问题,使归类更加准确。此过程不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,不会由于轨迹匹配失误而造成性能衰减,能够更精准地实现跨摄像头的多目标追踪。
现有目标追踪问题的解决方法通常聚焦于单摄像头场景下。例如,DeepSort算法使用卡尔曼滤波器和匈牙利匹配,结合目标检测和度量学习等工具,实现了单摄像头下相邻帧之间的目标匹配从而实现追踪;JDE(Java Development Environment)着力设计单阶段的目标追踪系统,同时进行目标检测特征和度量学习特征的提取,从而简化了算法的训练流程;FairMOT意识到检测问题和目标重识别任务之间特征的不匹配性,放弃了传统目标检测的训练模式,采用关键点检测来替代,解决了目标检测中心与目标移动中心不匹配的问题;CenterTrack同样通过解决这个不匹配问题提升追踪系统精度。上述方法在单摄像头多目标追踪领域均相继取得了很好的成绩,具有很不错的鲁棒性。然而,这些方法无法解决跨摄像头下的追踪问题,并且,现有的跨摄像头目标追踪方法只能单一的进行片段追踪再进行轨迹匹配,这样会造成由于轨迹匹配失误造成的性能衰减。因此,本申请提出了一种跨摄像头的多目标追踪方法。如图9所示,为跨摄像头的多目标追踪过程,追踪到不同行人在不同摄像头之间的运动轨迹,如2号行人从摄像头1运动到摄像头3的过程。如图10所示,为本申请提供的摄像头的多目标追踪流程示意图,图中循环跨摄像头的每一帧,通过目标检测网络和嵌入特征提取器对每个行人进行坐标定位和特征提取,再经由空域和时域匹配机制来进行行人的跟踪,完成行人轨迹的迭代生成,图11为本申请提供的摄像头的多目标追踪的系统结构图,系统主要包括目标检测器01、嵌入特征提取器02、空域匹配器03、时域匹配器04和目标轨迹缓存器05。
图12为空域匹配器的工作流程,首先将运动目标对应的原始检测信息发送至空域匹配器,随机选取一个摄像头ID,也即随机选取一个视频帧,然后从视频帧中选择一个检测框中的运动目标,根据运动目标的检测框对应的坐标确定运动目标在视觉空间区域中的目标区域(也即运动目标区域分配),此时需要遵循最大公共区域 原则,也即当运动目标跨越图7中的多个区域时,将运动目标归类为最多摄像头可拍摄到的公共区域;若目标区域为重叠视觉空间区域则对运动目标所在的目标区域计算相应的距离矩阵,并将距离矩阵中同摄像头下的运动目标之间的第一余弦距离进行距离屏蔽,也即自距离进行屏蔽;然后将满足预设条件的第一余弦距离对应的目标作为第一类目标,进行运动目标去重处理得到去重后剩余目标,并得到去重后剩余目标对应的目标检测信息,然后依次对目标区域中的其它行人进行去重处理;若目标区域为重叠视觉空间区域则直接将目标区域中的运动目标作为第二类目标;对目标区域的所有运动目标进行去重处理后依次对该视频帧中其它区域的运动目标进行去重处理;对该视频帧中的所有运动目标进行去重处理后依次通过摄像头ID选取其它视频帧,完成对其它视频帧中的运动目标进行去重的步骤,直至对所有视频帧中的运动目标进行去重处理(也即遍历完所有视频帧中的每个区域);需要指出的是每当对某一运动目标进行去重处理后,需要将该运动目标对应的目标检测信息保存到预设数据库中,并将该运动目标对应的所有检测框进行删除,以防止再次选取该运动目标在其它视频帧中对应的检测框重复进行去重处理。
图13为时域匹配器的工作流程,首先接收从空域匹配器发送的空域匹配结果,然后利用空域匹配结果中还未归类目标的特征信息和目标轨迹缓存器库中的已归类目标的特征信息,计算出还未归类目标和已归类目标之间的第二余弦距离(也即距离运算),将满足预设条件的第二余弦距离对应的还未归类目标中的一个目标和已归类目标中的一个目标作为同一目标,并根据时间先后顺序将还未归类目标中的目标的目标检测信息归类到已归类目标中的目标中。
参见图14所示,本申请实施例公开了一种跨摄像头的多目标追踪装置,包括:
视频帧获取模块11,设置为获取若干摄像头拍摄到的视频帧;
去重模块12,设置为确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;
归类模块13,设置为基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。
其中,关于上述各个模块更加具体的工作过程可以参考前述实施例中公开的相应内容,在此不再进行赘述。
可见,本申请获取若干摄像头拍摄到的视频帧;确定视频帧中位于不同摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一去重后剩余目标以及非重叠视觉空间区域上的每一第二类目标各自对应的路径轨迹。由此可见,本申请对重叠视觉空间区域的并且拍摄时间相同的第一类目标进行去重处理,以对不同摄像头在相同拍摄时间拍摄到的同一目标进行目标衔接,完成目标的空域匹配;在对目标完成去重处理后,将不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类得到相应的路径轨迹,完成目标的时域匹配;此过程不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,不会由于轨迹匹配失误而造成性能衰减,能够更精准地实现跨摄像头的多目标追踪。
进一步的,本申请实施例还提供了一种电子设备,图15是根据一示例性实施例示出的电子设备20结构图, 图中的内容不能认为是对本申请的使用范围的任何限制。
图15为本申请实施例提供的一种电子设备20的结构示意图。该电子设备20,具体可以包括:至少一个处理器21、至少一个存储器22、电源23、输入输出接口24、通信接口25和通信总线26。其中,存储器22用于存储计算机程序,计算机程序由处理器21加载并执行,以实现前述任意实施例公开的跨摄像头的多目标追踪方法的相关步骤。
本实施例中,电源23用于为电子设备20上的各硬件设备提供工作电压;通信接口25能够为电子设备20创建与外界设备之间的数据传输通道,其所遵循的通信协议是能够适用于本申请技术方案的任意通信协议,在此不对其进行具体限定;输入输出接口24,用于获取外界输入数据或向外界输出数据,其具体的接口类型可以根据具体应用需要进行选取,在此不进行具体限定。
另外,存储器22作为资源存储的载体,可以是只读存储器、随机存储器、磁盘或者光盘等,存储器22作为可以包括作为运行内存的随机存取存储器和用于外部内存的存储用途的非易失性存储器,其上的存储资源包括操作系统221、计算机程序222等,存储方式可以是短暂存储或者永久存储。
其中,操作系统221用于管理与控制源主机上电子设备20上的各硬件设备以及计算机程序222,操作系统221可以是Windows、Unix、Linux等。计算机程222除了包括能够用于完成前述任一实施例公开的由电子设备20执行的跨摄像头的多目标追踪方法的计算机程序之外,还可以进一步包括能够用于完成其他特定工作的计算机程序。
本实施例中,输入输出接口24具体可以包括但不限于USB接口、硬盘读取接口、串行接口、语音输入接口、指纹输入接口等。
进一步的,本申请实施例还公开了一种非易失性可读存储介质,用于存储计算机程序;其中,计算机程序被处理器执行时实现前述公开的跨摄像头的多目标追踪方法。
关于该方法的具体步骤可以参考前述实施例中公开的相应内容,在此不再进行赘述。
这里所说的非易失性可读存储介质包括随机存取存储器(Random Access Memory,RAM)、内存、只读存储器(Read-Only Memory,ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、磁碟或者光盘或技术领域内所公知的任意其他形式的存储介质。其中,计算机程序被处理器执行时实现前述跨摄像头的多目标追踪方法。关于该方法的具体步骤可以参考前述实施例中公开的相应内容,在此不再进行赘述。
本说明书中各个实施例采用递进的方式描述,每个实施例重点说明的都是与其它实施例的不同之处,各个实施例之间相同或相似部分互相参见即可。对于实施例公开的装置而言,由于其与实施例公开的跨摄像头的多目标追踪方法相对应,所以描述的比较简单,相关之处参见方法部分说明即可。
专业人员还可以进一步意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、计算机软件或者二者的结合来实现,为了清楚地说明硬件和软件的可互换性,在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本申请的范围。
结合本文中所公开的实施例描述算法的步骤可以直接用硬件、处理器执行的软件模块,或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程 ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
最后,还需要说明的是,在本文中,诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来,而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上对本申请所提供的一种跨摄像头的多目标追踪方法、装置、设备及介质进行了详细介绍,本文中应用了具体个例对本申请的原理及实施方式进行了阐述,以上实施例的说明只是用于帮助理解本申请的方法及其核心思想;同时,对于本领域的一般技术人员,依据本申请的思想,在具体实施方式及应用范围上均会有改变之处,综上,本说明书内容不应理解为对本申请的限制。
本申请对重叠视觉空间区域的并且拍摄时间相同的第一类目标进行去重处理,以对不同摄像头在相同拍摄时间拍摄到的同一目标进行目标衔接,完成目标的空域匹配;在对目标完成去重处理后,将不同拍摄时间下拍摄到的去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类得到相应的路径轨迹,完成目标的时域匹配;此过程不需要进行不同摄像头中的目标轨迹的匹配,而是进行目标的去重和归类从而得到跨摄像头的目标轨迹,不会由于轨迹匹配失误而造成性能衰减,能够更精准地实现跨摄像头的多目标追踪。
Claims (20)
- 一种跨摄像头的多目标追踪方法,包括:获取若干摄像头拍摄到的视频帧;确定所述视频帧中位于不同所述摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对所述第一类目标进行去重处理,得到去重后剩余目标;基于时间先后顺序,分别对在不同拍摄时间下拍摄到的所述去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一所述去重后剩余目标以及所述非重叠视觉空间区域上的每一所述第二类目标各自对应的路径轨迹。
- 根据权利要求1所述的跨摄像头的多目标追踪方法,其中,所述确定所述视频帧中位于不同所述摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,包括:确定所述重叠视觉空间区域上由不同摄像头在同一拍摄时间下拍摄到的不同运动目标的特征信息;确定所述不同运动目标的特征信息之间的第一余弦距离;判断所述第一余弦距离是否满足目标预设条件,若是则判定所述不同运动目标均为同一目标,以得到相应的第一类目标。
- 根据权利要求2所述的跨摄像头的多目标追踪方法,其中,所述判断所述第一余弦距离是否满足目标预设条件,包括:将不同所述摄像头在所述同一拍摄时间下拍摄到的各组所述不同运动目标对应的所述第一余弦距离保存至第一预设距离矩阵;其中,所述第一余弦距离在所述预设距离矩阵中的存储位置为基于所述第一余弦距离对应的运动目标的标识号确定的位置;分别判断所述第一预设距离矩阵中任意两个不同所述摄像头之间的所述第一余弦距离是否满足第一预设条件和第二预设条件;所述第一预设条件为所述第一余弦距离是否小于第一预设距离阈值,所述第二预设条件为所述第一余弦距离是否为对应的行列数值中的最小值。
- 根据权利要求3所述的跨摄像头的多目标追踪方法,其中,所述基于时间先后顺序,分别对在不同拍摄时间下拍摄到的所述去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,包括:利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定所述已归类目标和所述还未归类目标之间的第二余弦距离;所述还未归类目标包括还未归类的所述去重后剩余目标和所述第二类目标;利用所述第二余弦距离判断所述还未归类目标中的目标与所述已归类目标中的目标是否为同一目标,并基于判断结果对所述还未归类目标进行归类。
- 根据权利要求4所述的跨摄像头的多目标追踪方法,其中,所述利用所述第二余弦距离判断所述还未归类目标中的目标与所述已归类目标中的目标是否为同一目标,包括:将所述已归类目标和所述还未归类目标之间的所述第二余弦距离存放至第二预设距离矩阵;其中,所述第二余弦距离在所述第二预设距离矩阵中的存储位置为基于所述第二余弦距离对应的所述已归类目标和所述未归类目标的标识号确定的位置;分别判断所述第二预设距离矩阵中的所述第二余弦距离是否满足第三预设条件和第四预设条件;所述第三预设条件为所述第二余弦距离是否小于第二预设距离阈值,所述第四预设条件为所述第二余弦距离是否为对应的行列数值中的最小值;若满足所述第三预设条件和所述第四预设条件,则所述还未归类目标中的目标与所述已归类目标中的目标为同一目标,若不满足所述第三预设条件和所述第四预设条件,则所述还未归类目标中的目标与所述已归类目标中的目标不为同一目标。
- 根据权利要求4所述的跨摄像头的多目标追踪方法,其中,所述利用历史拍摄时间下已归类目标的特征信息与当前拍摄时间下还未归类目标的特征信息,确定所述已归类目标和所述还未归类目标之间的第二余弦距离,包括:分别计算历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到相应的多个余弦距离;从所述余弦距离中筛选出数值最小的所述余弦距离作为所述已归类目标和所述还未归类目标之间的第二余弦距离。
- 根据权利要求6所述的跨摄像头的多目标追踪方法,其中,所述分别计算历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到相应的多个余弦距离,包括:将不同所述摄像头对应的历史拍摄时间下所述已归类目标的各种特征信息存放至第一特征矩阵中,并将不同所述摄像头对应的当前拍摄时间下所述还未归类目标的各种特征信息存放至第二特征矩阵中;利用所述第一特征矩阵和所述第二特征矩阵进行余弦距离运算,得到保存有不同所述摄像头历史拍摄时间下所述已归类目标的各种特征信息和所述当前拍摄时间下还未归类目标的各种特征信息之间的多个余弦距离的第三预设距离矩阵;相应的,所述从所述余弦距离中筛选出数值最小的所述余弦距离作为所述已归类目标和所述还未归类目标之间的第二余弦距离,包括:从所述第三预设距离矩阵中的所述余弦距离中筛选出数值最小的所述余弦距离作为所述已归类目标和所述还未归类目标之间的第二余弦距离。
- 根据权利要求7所述的跨摄像头的多目标追踪方法,其中,所述将不同所述摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第一特征矩阵中,包括:将不同所述摄像头对应的历史拍摄时间下同一所述已归类目标的各种特征信息进行绑定以得到多个绑定后信息,并将所述绑定后信息依次存放至第一特征矩阵中。
- 根据权利要求8所述的跨摄像头的多目标追踪方法,其中,所述将不同所述摄像头对应的历史拍摄时间下同一所述已归类目标的各种特征信息进行绑定以得到多个绑定后信息,并将所述绑定后信息依次存放至第一特征矩阵中,包括:将不同所述摄像头对应的历史拍摄时间下同一所述已归类目标的各种特征信息存放至第三特征矩阵,以得到多个所述第三特征矩阵;整合多个所述第三特征矩阵以得到存放有所述已归类目标的各种特征信息的第一特征矩阵。
- 根据权利要求4所述的跨摄像头的多目标追踪方法,其中,所述利用历史拍摄时间下已归类目标的特 征信息与当前拍摄时间下还未归类目标的特征信息,确定所述已归类目标和所述还未归类目标之间的第二余弦距离,包括:分别计算同一所述摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到每个所述摄像头对应的所述已归类目标的所述各种特征信息与所述还未归类目标的所述各种特征信息之间的多个余弦距离;从所述多个余弦距离中筛选出数值最小的所述余弦距离作为所述已归类目标和所述还未归类目标之间的第二余弦距离。
- 根据权利要求10所述的跨摄像头的多目标追踪方法,其中,所述分别计算同一所述摄像头对应的历史拍摄时间下已归类目标的各种特征信息与当前拍摄时间下还未归类目标的各种特征信息之间的余弦距离,以得到每个所述摄像头对应的所述已归类目标的所述各种特征信息与所述还未归类目标的所述各种特征信息之间的多个余弦距离,包括:将同一所述摄像头对应的历史拍摄时间下已归类目标的各种特征信息存放至第四特征矩阵中,以得到与所述摄像头的数量对应的若干个所述第四特征矩阵;将同一所述摄像头对应的当前拍摄时间下还未归类目标的各种特征信息存放至第五特征矩阵中,以得到与所述摄像头的数量对应的若干个所述第五特征矩阵;利用同一所述摄像头对应的所述第四特征矩阵和所述第五特征矩阵进行余弦距离运算,得到保存有同一所述摄像头对应的历史拍摄时间下所述已归类目标的各种特征信息与当前拍摄时间下所述还未归类目标的各种特征信息之间的余弦距离的第四预设距离矩阵,以得到与所述摄像头的数量对应的若干个所述第四预设距离矩阵;相应的,所述从所述多个余弦距离中筛选出数值最小的所述余弦距离作为所述已归类目标和所述还未归类目标之间的第二余弦距离,包括:从若干个所述第四预设距离矩阵中的多个所述余弦距离中筛选出数值最小的所述余弦距离作为所述已归类目标和所述还未归类目标之间的第二余弦距离。
- 根据权利要求1至11任一项所述的跨摄像头的多目标追踪方法,其中,所述基于时间先后顺序,分别对在不同拍摄时间下拍摄到的所述去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,还包括:监测每个已归类目标对应的已归类时长;判断所述已归类时长是否大于预设时长阈值,如果是则将相应的所述已归类目标对应的特征信息进行删除。
- 根据权利要求1所述的跨摄像头的多目标追踪方法,其中,在获取若干摄像头拍摄到的视频帧之前,所述方法还包括:通过摄像头ID为所述若干摄像头设置摄像头标识,以区分不同摄像头。
- 根据权利要求13所述的跨摄像头的多目标追踪方法,其中,在获取若干摄像头拍摄到的视频帧之后,所述方法还包括:通过检测器基于检测网络对所述视频帧中的运动目标进行坐标定位,得到所述运动目标对应的检测框在对应的视频帧中的坐标;通过嵌入特征提取器提取所述视频帧中的运动目标的嵌入特征。
- 根据权利要求14所述的跨摄像头的多目标追踪方法,其中,在通过嵌入特征提取器提取所述视频帧中的运动目标的嵌入特征之后,所述方法还包括:依据所述运动目标对应的检测框在对应的视频帧中的坐标、所述嵌入特征和摄像头标识,获取所述运动目标对应的原始检测信息。
- 根据权利要求15所述的跨摄像头的多目标追踪方法,其中,确定所述视频帧中位于不同所述摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对所述第一类目标进行去重处理,得到去重后剩余目标包括:依据所述原始检测信息中的坐标,确定位于不同所述摄像头之间的重叠视觉空间区域的并且拍摄时间相同的运动目标作为第一类目标;依据所述原始检测信息中的嵌入特征,确定表示同一目标的第一类目标;对表示同一目标的第一类目标的原始检测信息进行归类,以完成对所述第一类目标的去重处理,得到去重后剩余目标。
- 根据权利要求14所述的跨摄像头的多目标追踪方法,在通过检测器基于检测网络对所述视频帧中的运动目标进行坐标定位之前,所述方法还包括:通过所述若干摄像头对应的内参和外参计算出所述若干摄像头在空间中的视觉空间区域;建立所述若干摄像头的视觉空间区域与所述视频帧中的坐标之间的联系,以实现依据所述原始检测信息中的坐标确定所述运动目标在视觉空间区域中的位置。
- 一种跨摄像头的多目标追踪装置,包括:视频帧获取模块,设置为获取若干摄像头拍摄到的视频帧;去重模块,设置为确定所述视频帧中位于不同所述摄像头之间的重叠视觉空间区域的并且拍摄时间相同的第一类目标,并对所述第一类目标进行去重处理,得到去重后剩余目标;归类模块,设置为基于时间先后顺序,分别对在不同拍摄时间下拍摄到的所述去重后剩余目标以及非重叠视觉空间区域上的第二类目标进行归类,以得到每一所述去重后剩余目标以及所述非重叠视觉空间区域上的每一所述第二类目标各自对应的路径轨迹。
- 一种电子设备,其中,包括处理器和存储器;其中,所述处理器执行所述存储器中保存的计算机程序时实现如权利要求1至17任一项所述的跨摄像头的多目标追踪方法。
- 一种非易失性可读存储介质,其中,用于存储计算机程序;其中,所述计算机程序被处理器执行时实现如权利要求1至17任一项所述的跨摄像头的多目标追踪方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210627280.3A CN114708304B (zh) | 2022-06-06 | 2022-06-06 | 一种跨摄像头的多目标追踪方法、装置、设备及介质 |
CN202210627280.3 | 2022-06-06 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023236514A1 true WO2023236514A1 (zh) | 2023-12-14 |
Family
ID=82177946
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/142129 WO2023236514A1 (zh) | 2022-06-06 | 2022-12-26 | 一种跨摄像头的多目标追踪方法、装置、设备及介质 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN114708304B (zh) |
WO (1) | WO2023236514A1 (zh) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114708304B (zh) * | 2022-06-06 | 2022-10-28 | 苏州浪潮智能科技有限公司 | 一种跨摄像头的多目标追踪方法、装置、设备及介质 |
CN117455957B (zh) * | 2023-12-25 | 2024-04-02 | 山东未来网络研究院(紫金山实验室工业互联网创新应用基地) | 一种基于深度学习的车辆轨迹定位追踪方法及系统 |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014092552A2 (en) * | 2012-12-13 | 2014-06-19 | Mimos Berhad | Method for non-static foreground feature extraction and classification |
CN111709974A (zh) * | 2020-06-22 | 2020-09-25 | 苏宁云计算有限公司 | 基于rgb-d图像的人体跟踪方法及装置 |
CN111914664A (zh) * | 2020-07-06 | 2020-11-10 | 同济大学 | 基于重识别的车辆多目标检测和轨迹跟踪方法 |
CN113516036A (zh) * | 2021-05-08 | 2021-10-19 | 上海依图网络科技有限公司 | 一种监控区域内目标对象的数量检测方法及装置 |
CN114708304A (zh) * | 2022-06-06 | 2022-07-05 | 苏州浪潮智能科技有限公司 | 一种跨摄像头的多目标追踪方法、装置、设备及介质 |
-
2022
- 2022-06-06 CN CN202210627280.3A patent/CN114708304B/zh active Active
- 2022-12-26 WO PCT/CN2022/142129 patent/WO2023236514A1/zh unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2014092552A2 (en) * | 2012-12-13 | 2014-06-19 | Mimos Berhad | Method for non-static foreground feature extraction and classification |
CN111709974A (zh) * | 2020-06-22 | 2020-09-25 | 苏宁云计算有限公司 | 基于rgb-d图像的人体跟踪方法及装置 |
CN111914664A (zh) * | 2020-07-06 | 2020-11-10 | 同济大学 | 基于重识别的车辆多目标检测和轨迹跟踪方法 |
CN113516036A (zh) * | 2021-05-08 | 2021-10-19 | 上海依图网络科技有限公司 | 一种监控区域内目标对象的数量检测方法及装置 |
CN114708304A (zh) * | 2022-06-06 | 2022-07-05 | 苏州浪潮智能科技有限公司 | 一种跨摄像头的多目标追踪方法、装置、设备及介质 |
Also Published As
Publication number | Publication date |
---|---|
CN114708304B (zh) | 2022-10-28 |
CN114708304A (zh) | 2022-07-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2023236514A1 (zh) | 一种跨摄像头的多目标追踪方法、装置、设备及介质 | |
Huh et al. | Fighting fake news: Image splice detection via learned self-consistency | |
CN110046266B (zh) | 一种照片的智能管理方法及装置 | |
WO2021103721A1 (zh) | 基于部件分割的识别模型训练、车辆重识别方法及装置 | |
CN108038176B (zh) | 一种路人库的建立方法、装置、电子设备及介质 | |
CN103984738A (zh) | 一种基于搜索匹配的角色标注方法 | |
CN104303193A (zh) | 基于聚类的目标分类 | |
WO2022142417A1 (zh) | 目标跟踪方法、装置、电子设备及存储介质 | |
CN114155284A (zh) | 基于多目标行人场景的行人跟踪方法、装置、设备及介质 | |
CN112905824A (zh) | 目标车辆追踪方法、装置、计算机设备及存储介质 | |
CN112309126B (zh) | 车牌检测方法、装置、电子设备及计算机可读存储介质 | |
WO2023197232A1 (zh) | 目标跟踪方法及装置、电子设备、计算机可读介质 | |
WO2021114985A1 (zh) | 一种同行对象识别方法、装置、服务器及系统 | |
WO2023015938A1 (zh) | 三维点检测的方法、装置、电子设备及存储介质 | |
CN113837006A (zh) | 一种人脸识别方法、装置、存储介质及电子设备 | |
KR20170095599A (ko) | 동영상 검색 시스템 및 그 방법 | |
CN114332716B (zh) | 视频中场景的聚类方法、装置、电子设备及存储介质 | |
KR102426594B1 (ko) | 크라우드소싱 환경에서 객체의 위치를 추정하는 시스템 및 방법 | |
CN115393755A (zh) | 视觉目标跟踪方法、装置、设备以及存储介质 | |
CN112257666B (zh) | 目标图像内容的聚合方法、装置、设备及可读存储介质 | |
JP2016045538A (ja) | 情報処理装置、画像判定方法、及びプログラム | |
CN110751065B (zh) | 训练数据的采集方法及装置 | |
CN113780424A (zh) | 一种基于背景相似度的照片实时在线聚类方法及系统 | |
Duanmu et al. | A multi-view pedestrian tracking framework based on graph matching | |
CN111639640A (zh) | 基于人工智能的车牌识别方法、装置及设备 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22945642 Country of ref document: EP Kind code of ref document: A1 |