CN114648557A - Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle - Google Patents

Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle Download PDF

Info

Publication number
CN114648557A
CN114648557A CN202210311724.2A CN202210311724A CN114648557A CN 114648557 A CN114648557 A CN 114648557A CN 202210311724 A CN202210311724 A CN 202210311724A CN 114648557 A CN114648557 A CN 114648557A
Authority
CN
China
Prior art keywords
track
view
similarity
visual angle
ground
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210311724.2A
Other languages
Chinese (zh)
Inventor
冯伟
韩瑞泽
万亮
王松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN202210311724.2A priority Critical patent/CN114648557A/en
Publication of CN114648557A publication Critical patent/CN114648557A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a multi-target cooperative tracking method based on a high-altitude visual angle and a ground visual angle, which comprises the following steps of: step s1, dividing the two sections of synchronous videos of the space-ground collaborative visual angle into V small sections of videos with the same length; step s2, respectively generating a small segment track of the target under different visual angles for each small segment of video according to the target detection result; step s3, calculating similarity scores between different small segment tracks; step s4, performing track association by adopting a joint optimization function according to the similarity score to generate a short cross-view track; step s5, repeating steps s2 to s4, and connecting the short cross-view tracks, so as to obtain a cross-view long track as a final tracking result; the method has higher matching precision than the existing method, and obtains good cross-domain performance.

Description

Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle
Technical Field
The invention belongs to the field of target tracking, relates to target tracking of videos shot by a wearable camera and an unmanned aerial vehicle at two visual angles, and particularly relates to a multi-target cooperative tracking method based on a high-altitude visual angle and a ground visual angle.
Background
The background art related to the present invention is as follows:
(1) pedestrian target detector (see document [1 ]): as an important component of data preprocessing, a pedestrian target detector has been widely applied in various fields such as target tracking, pedestrian repositioning, and motion recognition. The pedestrian detectors commonly used at present can be divided into two main categories, namely methods based on background modeling and methods based on statistical learning. The former must adapt to environmental changes and is limited by picture shaking caused by camera shaking. The latter is a commonly used method for detecting pedestrians at present, and a pedestrian detection classifier is constructed by constructing a large number of samples. At present, the role of deep learning in pedestrian detection is not negligible. Wherein a YOLO detector is used in the present invention to provide a corresponding target detection box.
(2) Space-ground collaborative perspective mobile camera network: advances in motion camera technology provide new perspectives for video surveillance. The drone may provide an overhead view of a set of objects on the ground. The wearable cameras can provide a ground view of the same set of objects. The invention provides a new air-ground collaborative visual angle mobile camera network, wherein a top view can provide target global information, a ground view can provide target local details, and therefore the top view and the ground view can complement information well, and better coverage and flexibility are provided for outdoor monitoring. The method can be effectively applied to tasks such as cooperative tracking, personal/group activity identification and the like through the complementary network.
(3) Cross-view multi-target association based on spatial distribution: since the appearance and motion information cannot be utilized, the association of the top view and the ground view becomes very difficult, and cross-view association based on spatial distribution can realize the association between objects by means of spatial position distribution information between high-altitude view and ground view. By the method, the data of different visual angles can be better subjected to collaborative analysis, so that the method is applied to tasks such as target tracking, pedestrian re-recognition, action recognition and the like.
Disclosure of Invention
The invention aims to provide a multi-target tracking technology combining global information and local information for synchronous videos under a hollow-ground cooperative visual angle in the same scene, and realizes cooperative tracking of targets under the hollow-ground cooperative visual angle by means of wearing cameras and unmanned aerial vehicle equipment and utilizing a relatively simple and effective strategy; the method can be used for simultaneously tracking the targets in the ground and high-altitude visual angles under the condition that the number of the targets in the visual angles is not limited by utilizing the space and ground cooperated visual angles in the same scene, and the targets in the two visual angles are in one-to-one correspondence.
The invention is implemented by adopting the following technical scheme:
a multi-target cooperative tracking method based on a high-altitude visual angle and a ground visual angle comprises the following steps:
step s1, dividing the two sections of synchronous videos of the air-ground collaborative visual angle into V small sections of videos with the same length;
step s2, respectively generating a small segment track of the target under different visual angles for each small segment of video according to the target detection result;
step s3, calculating similarity scores between different small segment tracks;
step s4, performing track association by adopting a joint optimization function according to the similarity score C to generate a short cross-view track;
step s5, repeating steps s2 to s4, and connecting the short cross-view tracks, so as to obtain a cross-view long track as a final tracking result; wherein: the short cross-view trajectory generation process:
obtaining similarity measurement between any cross-space-time trajectory segments through a similarity score C;
solving by adopting the following formula according to the similarity measurement:
Figure BDA0003568651300000021
wherein: c. CijRepresenting the similarity between the track segments i and j, which can be obtained by the calculation in the third step; a isijA binary variable of 0-1 indicating whether two track segments belong to the same target.
Further, a step of obtaining similarity scores between two segment trajectories at the same time and different angles in step S3 by using cross-view trajectory similarity:
201. using spatial distribution uniformity
Obtaining a matching result of the high altitude and the ground view by using a method based on spatial distribution:
if the detection box BmAnd a detection frame BqIf the matching result is the same person, S (B)m,Bq) 1 is ═ 1; otherwise, S (B)m,Bq) 0; high altitude view track BmAnd ground view track BqThe similarity between them is expressed as follows:
Figure BDA0003568651300000022
where F is the number of overlapping frames, BmIs the detection result under the top view, BqIs the detection result under the ground view, | BmI is track BmNumber of, | BqI is track BqThe number of (c);
202. by appearance similarity
Measuring inter-track similarity by using a twin network;
respectively calculating average images of the two sections of tracks under the space-ground collaborative visual angle and inputting the average images into a network;
extracting features
Calculating Euclidean distance between features to obtain similarity score
Figure BDA0003568651300000023
Combining the similarity scores, calculating the weight of the edge by utilizing linear combination as follows:
Figure BDA0003568651300000024
further, a step of obtaining similarity score generation between two segments of trajectories at the same view angle at different times in step S3 by adopting the inter-time trajectory consistency:
302. appearance trajectory consistency is obtained by using a color histogram to measure appearance similarity of single-view objects:
calculating a target color histogram of a single view trajectory;
taking the median Ψ (B) of the color histogram as an appearance descriptor of the track B;
computing inter-track appearance similarity using histogram intersection
Figure BDA0003568651300000031
302. Motion continuity is obtained by predicting motion consistency by using a constant speed motion model:
calculating the forward deviation error delta by using a motion modelpAnd reverse bias error deltan；
By δ being equal to α (δ)p+δn) Measuring the difference between the tracks;
converting errors into similarities
Figure BDA0003568651300000032
Combining the similarity scores, calculating the weight of the edge by utilizing linear combination as follows:
Figure BDA0003568651300000033
advantageous effects
According to the invention, by using the wearable camera and the unmanned aerial vehicle device, a simple and effective strategy is selected, a combined air-ground cooperative visual angle multi-target tracking method is autonomously designed, the problem of shielding in a multi-pedestrian motion scene is effectively solved, the result is accurate, the algorithm is efficient, and particularly, the method has the following advantages:
a. high tracking precision
The global motion track and the local appearance details of the target can be obtained by utilizing the video shot under the space and ground cooperative view angle, the target loss caused by shielding is greatly relieved, and therefore higher tracking precision is achieved. The visualization of the results of the collaborative tracking is shown in fig. 2.
b. The algorithm has low complexity and high speed
The method is used for target tracking, and the running speed is 4.24fps through a large number of experimental statistics, so that the method is superior to the existing most tracking methods. On the premise of ensuring the accuracy, the time consumption is greatly reduced.
c. Strong migration ability
The invention does not need to train a large amount of data, does not depend on specific data characteristics, and has strong universality.
Drawings
FIG. 1: air-ground cooperative visual angle multi-target tracking problem flow chart
FIG. 2: space-ground cooperative view angle multi-target tracking result schematic diagram
Detailed Description
The invention is described in detail below with reference to the attached drawing figures:
the invention provides a multi-target cooperative tracking method based on a high-altitude visual angle and a ground visual angle. Then, the trajectories of adjacent video segments in the multiple views are used to establish spatiotemporal data associations, resulting in intersecting view trajectories. And finally, connecting the short tracks to obtain the cross-view long track as a final tracking result. The specific process shown in fig. 1:
step one, s1, dividing two sections of synchronous videos of the space-ground collaborative view angle into V small sections of videos with the same length; wherein: synchronized video segmentation
Given a video from a high-altitude perspective and multiple videos from a ground perspective, we segment both videos synchronously into video segments of the same length.
Step s2, respectively generating a small segment track of the target under different visual angles for each small segment of video according to the target detection result; step s3, calculating similarity scores between different small segment tracks;
and a step of obtaining similarity scores between two short segment tracks at the same time and different angles of view in the step S3 by using the cross-angle track similarity:
201. using spatial distribution uniformity
Obtaining a matching result of the high altitude and the ground view by using a method based on spatial distribution:
if the detection box BmAnd a detection frame BqIf the matching result is the same person, thenS(Bm,Bq) 1 is ═ 1; otherwise, S (B)m,Bq) 0; high altitude view track BmWith ground view track BqThe similarity between them is expressed as follows:
Figure BDA0003568651300000041
where F is the number of overlapping frames, BmIs the detection result under the top view, BqIs the detection result under the ground view, | BmI is track BmNumber of, | BqI is track BqThe number of (2);
202. by appearance similarity
Measuring inter-track similarity by using a twin network;
respectively calculating average images of two sections of tracks under the space-ground collaborative visual angle and inputting the average images into a network;
extracting features;
calculating Euclidean distance between features to obtain similarity score
Figure BDA0003568651300000042
Combining the similarity scores, calculating the weight of the edge by utilizing linear combination as follows:
Figure BDA0003568651300000043
and a step of generating similarity scores between two small segment tracks at the same view angle at different times in the step S3 by using the consistency of the tracks across time:
301. appearance trajectory consistency is obtained by using a color histogram to measure appearance similarity of single-view objects:
calculating a target color histogram of a single view trajectory;
taking the median Ψ (B) of the color histogram as an appearance descriptor of the track B;
computing inter-track appearance similarity using histogram intersection
Figure BDA0003568651300000051
302. Motion continuity is obtained by predicting motion consistency by using a constant speed motion model:
calculating the forward deviation error delta by using a motion modelpAnd reverse bias error deltan；
By δ being equal to α (δ)p+δn) Measuring the difference between the tracks;
converting errors into similarities
Figure BDA0003568651300000052
Combining the similarity scores, calculating the weight of the edge by utilizing linear combination as follows:
Figure BDA0003568651300000053
step s4, performing track association by adopting a joint optimization function according to the similarity score C to generate a short cross-view track;
step s5, repeating steps s2 to s4, and connecting the short cross-view tracks, so as to obtain a cross-view long track as a final tracking result; wherein: the short cross-view trajectory generation process:
obtaining similarity measurement between any cross-space-time trajectory segments through a similarity score C;
solving by the following formula according to the similarity measure
Figure BDA0003568651300000054
Wherein: c. CijRepresenting the similarity between track segments i and j; a isijA binary variable of 0-1 indicating whether two track segments belong to the same target.

Claims (3)

1. A multi-target cooperative tracking method based on a high-altitude visual angle and a ground visual angle is characterized by comprising the following steps:
step s1, dividing the two sections of synchronous videos of the space-ground collaborative visual angle into V small sections of videos with the same length;
step s2, respectively generating a small segment track of the target under different visual angles for each small segment of video according to the target detection result;
step s3, calculating similarity scores between different small segment tracks;
step s4, performing track association by adopting a joint optimization function according to the similarity score C to generate a short cross-view track;
step s5, repeating steps s2 to s4, and connecting the short cross-view tracks, so as to obtain a cross-view long track as a final tracking result; wherein: the short cross-view trajectory generation process:
obtaining similarity measurement between any cross-space-time trajectory segments through a similarity score C;
solving by the following formula according to the similarity measure
Function(s)
Figure FDA0003568651290000011
Wherein: c. CijRepresenting the similarity between the track segments i and j, which can be obtained by the calculation in the third step; a isijA binary variable of 0-1, indicating whether two track segments belong to the same target.
2. The multi-target cooperative tracking method based on the high-altitude visual angle and the ground visual angle as claimed in claim 1, characterized in that: and a step of obtaining similarity scores between two short segment tracks at the same time and different angles of view in the step S3 by using the cross-angle track similarity:
201. using spatial distribution uniformity
Obtaining a matching result of the high altitude and the ground view by using a method based on spatial distribution:
if the detection box BmAnd a detection frame BqIf the matching result is the same person, S (B)m,Bq) 1 is ═ 1; otherwise, S (B)m,Bq)＝0；
High altitude view track BmTrack for viewing groundTrace BqThe similarity between them is expressed as follows:
Figure FDA0003568651290000012
wherein F is the number of overlapping frames, BmIs the detection result under the top view, BqIs the detection result under the ground view, | BmI is track BmNumber of, | BqI is track BqThe number of (2);
202. by appearance similarity
Measuring inter-track similarity by using a twin network;
respectively calculating average images of the two sections of tracks under the space-ground collaborative visual angle and inputting the average images into a network;
extracting features
Calculating Euclidean distance between features to obtain similarity score
Figure FDA0003568651290000013
Combining the similarity scores, calculating the weight of the edge by utilizing linear combination as follows:
Figure FDA0003568651290000021
3. the multi-target cooperative tracking method based on the high-altitude visual angle and the ground visual angle as claimed in claim 1, characterized in that: and a step of generating similarity scores between two small segment tracks at the same view angle at different times in the step S3 by using the consistency of the tracks across time:
301. appearance trajectory consistency is obtained by using a color histogram to measure appearance similarity of single-view objects:
calculating a target color histogram of a single view trajectory;
taking the median Ψ (B) of the color histogram as an appearance descriptor of the track B;
computing inter-track appearance similarity using histogram intersection
Figure FDA0003568651290000022
302. Motion continuity is obtained by predicting motion consistency by using a constant speed motion model:
calculating the forward deviation error delta by using a motion modelpAnd reverse bias error deltan；
By δ being equal to α (δ)p+δn) Measuring the difference between the tracks;
converting errors into similarities
Figure FDA0003568651290000023
Combining the similarity scores, calculating the weight of the edge by utilizing linear combination as follows:
Figure FDA0003568651290000024
CN202210311724.2A 2022-03-28 2022-03-28 Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle Pending CN114648557A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210311724.2A CN114648557A (en) 2022-03-28 2022-03-28 Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210311724.2A CN114648557A (en) 2022-03-28 2022-03-28 Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle

Publications (1)

Publication Number Publication Date
CN114648557A true CN114648557A (en) 2022-06-21

Family

ID=81994694

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210311724.2A Pending CN114648557A (en) 2022-03-28 2022-03-28 Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle

Country Status (1)

Country Link
CN (1) CN114648557A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115619832A (en) * 2022-12-20 2023-01-17 浙江莲荷科技有限公司 Multi-camera collaborative multi-target track confirmation method, system and related device
CN116843721A (en) * 2023-08-28 2023-10-03 天津大学 Video multi-target detection association and track generation method and device and electronic equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115619832A (en) * 2022-12-20 2023-01-17 浙江莲荷科技有限公司 Multi-camera collaborative multi-target track confirmation method, system and related device
CN116843721A (en) * 2023-08-28 2023-10-03 天津大学 Video multi-target detection association and track generation method and device and electronic equipment
CN116843721B (en) * 2023-08-28 2024-01-19 天津大学 Video multi-target detection association and track generation method and device and electronic equipment

Similar Documents

Publication Publication Date Title
CN113034548B (en) Multi-target tracking method and system suitable for embedded terminal
Minaeian et al. Effective and efficient detection of moving targets from a UAV’s camera
CN109064484B (en) Crowd movement behavior identification method based on fusion of subgroup component division and momentum characteristics
Cao et al. Vehicle detection and motion analysis in low-altitude airborne video under urban environment
Rout A survey on object detection and tracking algorithms
CN105528794A (en) Moving object detection method based on Gaussian mixture model and superpixel segmentation
CN114648557A (en) Multi-target cooperative tracking method based on high-altitude visual angle and ground visual angle
CN112581503B (en) Multi-target detection and tracking method under multiple visual angles
Ke et al. Motion-vector clustering for traffic speed detection from UAV video
Moutakki et al. Real-time system based on feature extraction for vehicle detection and classification
Najiya et al. UAV video processing for traffic surveillence with enhanced vehicle detection
CN107315994B (en) Clustering method based on Spectral Clustering space trajectory
CN105160649A (en) Multi-target tracking method and system based on kernel function unsupervised clustering
CN102034267A (en) Three-dimensional reconstruction method of target based on attention
CN104517095A (en) Head division method based on depth image
Saif et al. Crowd density estimation from autonomous drones using deep learning: challenges and applications
Li et al. Vehicle object detection based on rgb-camera and radar sensor fusion
Wu et al. Collective density clustering for coherent motion detection
CN113190711A (en) Video dynamic object trajectory space-time retrieval method and system in geographic scene
CN111950524A (en) Orchard local sparse mapping method and system based on binocular vision and RTK
CN102123234B (en) Unmanned airplane reconnaissance video grading motion compensation method
CN116862832A (en) Three-dimensional live-action model-based operator positioning method
Zhou et al. Fast road detection and tracking in aerial videos
Cao et al. Tracking vehicles as groups in airborne videos
Brown et al. Feature-aided multiple target tracking in the image plane

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination