CN116704368A - Multi-target tracking method, system and storage medium based on satellite video data association - Google Patents
Multi-target tracking method, system and storage medium based on satellite video data association Download PDFInfo
- Publication number
- CN116704368A CN116704368A CN202310660081.7A CN202310660081A CN116704368A CN 116704368 A CN116704368 A CN 116704368A CN 202310660081 A CN202310660081 A CN 202310660081A CN 116704368 A CN116704368 A CN 116704368A
- Authority
- CN
- China
- Prior art keywords
- data
- track
- detection
- detection frame
- score
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000001514 detection method Methods 0.000 claims abstract description 275
- 238000012549 training Methods 0.000 claims description 8
- 230000002596 correlated effect Effects 0.000 claims 2
- 238000003384 imaging method Methods 0.000 abstract 1
- 238000006243 chemical reaction Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 241001239379 Calophysus macropterus Species 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application relates to the technical field of photoelectric imaging, in particular to a multi-target tracking method, a system and a storage medium based on satellite video data association. The multi-target tracking method based on satellite video data association comprises the following steps: setting a detection threshold of the detection model, and dividing a target detection frame into a high-resolution detection frame and a low-resolution detection frame; detecting multiple targets in a current frame of satellite video data by adopting a detection model to obtain high-score detection frame data and low-score detection frame data; performing primary data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; when the first data association of part of the predicted tracks in the predicted tracks is unsuccessful, carrying out the second data association on the low-resolution detection frame data and part of the predicted tracks to form second tracks; and outputting a tracking result of multiple targets in the current frame based on the first track and the second track, and improving the accuracy of satellite video multi-target tracking on the premise of not increasing the computational complexity.
Description
Technical Field
The application belongs to the technical field of remote sensing image processing, and particularly relates to a multi-target tracking method, a system and a storage medium based on satellite video data association.
Background
The multi-target tracking technology under satellite video is to estimate the positions of multiple targets simultaneously frame by frame in a satellite video sequence and acquire their motion trajectories. In recent years, the multi-target tracking technology for satellite video has urgent application requirements in the military and civil fields, and has gradually developed into a hot spot technology in the current remote sensing image processing field.
The current mainstream general multi-target tracking method is multi-oriented to a common scene, a detection model related in a tracking frame is difficult to be suitable for a small target scene under satellite video, and in addition, the related correlation mode has high computational complexity when processing large-size satellite video data, so that the tracking performance is obviously reduced when the general multi-target tracking method is applied to the satellite video scene, and the model robustness is insufficient to cope with specific tracking problems of satellite video such as high background complexity, low spatial resolution, small target size and the like. Therefore, how to improve the accuracy of satellite video multi-target tracking without increasing the computational complexity becomes a problem to be solved.
Disclosure of Invention
The object of one or more embodiments of the present disclosure is to provide a multi-target tracking method, system and storage medium based on satellite video data association, which improves the accuracy of satellite video multi-target tracking without increasing the computational complexity compared with the prior art.
To solve the above technical problems, one or more embodiments of the present specification are implemented as follows:
in a first aspect, a multi-target tracking method based on satellite video data association is provided, including the steps of: setting a detection threshold of the detection model, and dividing a target detection frame into a high-resolution detection frame and a low-resolution detection frame; detecting multiple targets in the current frame of the satellite video data by adopting the detection model to obtain high-score detection frame data and low-score detection frame data; performing first data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; when the first data association of part of the predicted tracks is unsuccessful, carrying out the second data association on the low-score detection frame data and the part of the predicted tracks to form a second track; and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track.
In a second aspect, a multi-target tracking system based on satellite video data association is provided, comprising: the detection rule setting module is used for setting a detection threshold value of the detection model and dividing the target detection frame into a high-score detection frame and a low-score detection frame; the detection module is used for detecting multiple targets in the current frame of the satellite video data by adopting the detection model to obtain high-score detection frame data and low-score detection frame data; the first data association module is used for carrying out first data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; the second data association module is used for carrying out second data association on the low-score detection frame data and the part of the predicted track to form a second track when the first data association of the part of the predicted track in the current frame is unsuccessful; and the result output module is used for outputting a tracking result of multiple targets in the current frame based on the first track and the second track.
In a third aspect, a storage medium is provided for computer readable storage, where the storage medium stores one or more programs, and the one or more programs are executable by one or more processors to implement the above multi-objective tracking method based on satellite video data association.
As can be seen from the technical solutions provided in one or more embodiments of the present disclosure, in the multi-target tracking method based on satellite video data association provided in the embodiments of the present disclosure, a detection threshold of a detection model is set first, and a target detection frame is divided into a high-resolution detection frame and a low-resolution detection frame; the detection model and the detection threshold are subjected to fine adjustment to enable the detection model to be suitable for the data characteristics of a large-size satellite video small-target scene, and then the detection model is adopted to detect multiple targets in the current frame of the satellite video to obtain high-score detection frame data and low-score detection frame data; performing primary data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; when the first data association of the partial predicted track is unsuccessful, carrying out the second data association on the low-score detection frame data and the partial predicted track to form a second track; and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track. The simultaneous positioning and tracking of multiple targets are realized through the association of the high-low detection frame data, and compared with the satellite multi-target tracking method in the prior art, the satellite multi-target tracking method has the advantages of lower calculation complexity, higher tracking accuracy, low ID conversion times and the like.
Drawings
For a clearer description of one or more embodiments of the present description or of the solutions of the prior art, reference will be made below to the accompanying drawings which are used in the description of one or more embodiments or of the prior art, it being apparent that the drawings in the description below are only some of the embodiments described in the description, from which, without inventive faculty, other drawings can also be obtained for a person skilled in the art.
FIG. 1 is a flow chart of a method for providing multi-target tracking based on satellite video data correlation according to an embodiment of the present application;
fig. 2 is a schematic diagram of software code executed by a multi-target tracking method providing correlation based on satellite video data according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the technical solutions in this specification, a clear and complete description of the technical solutions in one or more embodiments of this specification will be provided below with reference to the accompanying drawings in one or more embodiments of this specification, and it is apparent that the one or more embodiments described are only a part of embodiments of this specification, not all embodiments. All other embodiments, which can be made by one or more embodiments of the present disclosure without inventive faculty, are intended to be within the scope of the present disclosure.
The multi-target tracking method based on satellite video data association provided by the embodiment of the application can improve the accuracy of satellite video multi-target tracking. The multi-target tracking method based on the satellite video data association and the respective steps thereof provided in the present specification will be described in detail below.
Example 1
Referring to fig. 1, the multi-target tracking method based on satellite video data association provided by the embodiment of the application is a lightweight satellite video multi-target tracking method with small calculation amount and simple and feasible based on satellite video data association, and is characterized in that a detection model and a detection threshold are finely adjusted based on satellite data, so that the detection model is suitable for the data characteristics of a large-size satellite data small-target scene, and the multi-target simultaneous positioning and tracking is realized through the satellite video data association of high-score and low-score detection frames. The satellite video data association is to associate image information in a target frame in a space dimension, namely, associate a detection result to a tracking result.
The multi-target tracking method based on satellite video data association provided by the embodiment of the application comprises the following steps:
s10: setting a detection threshold of the detection model, and dividing a target detection frame into a high-resolution detection frame and a low-resolution detection frame;
before online detection is carried out on a plurality of targets in a current frame by using a YOLOX detection model, a detection threshold of the detection model is set according to the size of a target scale, and a target detection frame is divided into a high-resolution detection frame and a low-resolution detection frame. Specific operations may be referred to the following formulas.
1. A in the formula (1) represents a confidence threshold value of the detection frame, and D in the formula (2) high Representing a high score detection frame set, D low Representing a low-resolution collection of test frames, d.score representing the test frameConfidence level. Confidence refers to reliability, i.e. the likelihood that an object within a detection frame is a real target, a confidence threshold is set based on the data, when the confidence is greater than the confidence threshold, the corresponding detection frame is initialized to a trajectory, di is added to D high Or D low Is a kind of medium.
S20: detecting multiple targets in a current frame of the satellite video by adopting a detection model to obtain high-score detection frame data and low-score detection frame data;
the trained detection model is used to detect the target in the current frame online, where the detection model may be a YOLOX detection model. According to the satellite video data correlation method, a high-low frame secondary correlation satellite video data correlation method is provided according to the data characteristics of satellite video, most of low-resolution detection frames are false alarms (false alarms refer to other non-interested targets or backgrounds contained in detection frames), or the probability of shielding caused by the fact that the targets detected by a detector are right due to shielding, scale transformation and other reasons is reduced, so that the confidence of the shielded targets is low, and in the two-time correlation method, the second-time data correlation low-resolution detection frames can well track some shielded targets.
S30: and carrying out primary data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track. Optionally, the first data association is performed on the high-score detection frame data and the predicted track in the current frame to form a first track, which specifically includes: calculating the cross ratio IOU of the high-score detection frame and the multi-target prediction track in the current frame; and then carrying out similarity matching on the cross-correlation ratio by adopting a Hungary algorithm to obtain a first track.
Further, S30: the multi-target tracking method based on satellite video data association provided by the embodiment of the application further comprises the following steps of: and initializing partial high-score detection frame data as the track of the new target when the first data association of the partial high-score detection frame data in the high-score detection frame data is unsuccessful.
IOU is calculated for the high-score detection frame data and the multi-target predicted track in the current frame. The Hungary algorithm is a combined optimization algorithm for solving task allocation problems in polynomial time, and in the embodiment of the application, the optimal solutions of the corresponding tracks of a plurality of targets are found by carrying out first data association on the targets and the tracks through the Hungary algorithm. If there is high-resolution detection frame data which is not matched, initializing the part of the high-resolution detection frame data as the track of a new target. The IOU is the cross-over ratio between two target frames, and the purpose of calculating the IOU for the predicted track and the high frame data is to allocate the high frame data in the current frame to the predicted track according to the IOU. The IOU for each high frame data and each predicted track is calculated for data correlation, and the relationship between each high frame data and each predicted track is determined.
S40: when the first data association of the partial predicted track is unsuccessful, carrying out the second data association on the low-resolution detection frame data and the partial predicted track to form a second track; optionally, S40: when the first data association of the partial predicted track is unsuccessful, performing a second data association on the low-resolution detection frame data and the partial predicted track to form a second track, and then, the multi-target tracking method based on satellite video data association provided by the embodiment of the application further comprises the following steps: when the second data association of the residual predicted track in the partial predicted track is unsuccessful, reserving the residual predicted track with a set frame number; and deleting part of the low score detection frame data when the second data association of part of the low score detection frame data is unsuccessful.
The low-score detection frame mainly comprises targets such as false alarm, shielding, motion blurring, size transformation and the like, so that the low-score detection frame and part of the prediction tracks which are not matched are enabled to calculate the IOU, then data association is carried out through a Hungary algorithm, if the rest of the prediction tracks which are not associated and matched are reserved for a set number of frames, and if the rest of the prediction tracks are not associated and matched, the rest of the prediction tracks are deleted. And deleting if the low-score detection frame which is not associated with the matching appears. The IOU is the cross-over ratio between two target frames, and the purpose of calculating the IOU for the predicted track and the high frame data is to allocate the high frame data in the current frame to the predicted track according to the IOU. The hungarian algorithm correlates the detection frame data of the current frame with the predicted trajectory according to the calculated IOU.
S50: and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track.
After each frame in the satellite video is subjected to target detection, the detection results are subjected to data correlation so as to output a complete multi-target motion trail. The data association can calculate the similarity between the detection frame data of the current frame and the multi-target prediction track in the current frame through the IOU, and the association matching is carried out according to the similarity through the Hungary algorithm. Firstly, dividing the detection result of the current frame into high-resolution detection frame data and low-resolution detection frame data, in each frame, firstly, associating the predicted track of multiple targets in the current frame with the high-resolution detection frame data, then associating the low-resolution detection frame data with part of the unmatched predicted tracks, distinguishing the background from the real targets through the similarity of target tracking tracks, and deleting the tracks if the number of the rest predicted tracks exceeds the set frame number. And initializing the track of the new target by the unmatched detection frame data.
Combining the first track and the second track as a tracking result, wherein the single-target tracking and the multi-target tracking are distinguished by a data association process, the data association is used for matching a plurality of targets between frames, wherein the targets comprise the appearance of a new target and the disappearance of an old target, the ID of the previous frame is matched with the ID of the current frame, and a unique ID is allocated for each target.
Optionally, S10: before setting a detection threshold of a detection model and dividing a target detection frame into a high-score detection frame and a low-score detection frame, the multi-target tracking method based on satellite video data association provided by the embodiment of the application further comprises the following steps: training a detection model by adopting satellite video data; and adjusting the super parameters of the detection model based on the satellite video data to obtain the detection weight of the detection model on the satellite video data. The off-line training is to train all data sets directly, and the obtained detection weight has better detection effect due to large satellite video data volume, and can be on-line training but the tracking effect is reduced. The super-parameters of the detection model are adjusted by utilizing some characteristics of the satellite video to adjust some parameters in the training algorithm, such as learning rate, training round number and the like, so that the detection effect of the detection model is better.
The embodiment of the application designs a satellite video data association method for secondary association of high and low frames according to the data characteristics of satellite videos: because of the false alarms when the low-resolution detection frames are mostly, the low-resolution detection frame data associated with the second data can track some blocked targets well in the two-time association. Specifically, the detection algorithm of the YOLOX detection model is finely adjusted through the data set characteristics of the VISO satellite video, unlike the ground video, the satellite video has the characteristics of complex background, fewer available target characteristics, small targets and the like, so that the detected target confidence is very low, and the parameters of the YOLOX can be correspondingly adjusted according to experimental conditions. When the target size is smaller, the confidence coefficient threshold of the detection frame can be properly adjusted down, so that more detected targets are transferred into the tracking track; when the target is large and clear, the confidence threshold of the detection frame can be properly increased, so that false alarms are reduced, and the accuracy is improved.
Optionally, S30: the multi-target tracking method based on satellite video data association provided by the embodiment of the application further comprises the following steps of: and predicting the positions of multiple targets in the current frame based on the tracking result of the previous frame to obtain a predicted track. An embodiment of a multi-target tracking method based on satellite video data association provided by an embodiment of the application is shown in fig. 2.
And carrying out Kalman filtering on the tracking result formed by the previous frame to predict the target position in the current frame. Kalman filtering is a highly efficient autoregressive filter that can be used in the presence of a number of uncertaintiesThe state of the dynamic system is estimated from the combined information of the dynamic system. In the embodiment of the application, the Kalman filtering is adopted to predict the target state in the future frame. Wherein d is 1:t+1 Represents the set of tracks formed in frames 1 through t+1, d 1:t Representing the track set formed from the first frame through the t frame.
The multi-target tracking method based on satellite video data association provided by the embodiment of the application has lower conversion times of the ID of the target track number in the multi-target tracking process in the satellite video than other technologies. Because the moving targets in the satellite video are similar in characteristic, dense in distribution and easy to mutually block, the ID is invalid and the conversion times are high, namely the same ID tracking frame continuously flashes and the same moving target has a plurality of different IDs. The embodiment of the application can effectively reduce the ID conversion times and improve the IDF1, wherein the IDF1 is an index obtained by comprehensively considering the tracking precision and the ID conversion times, and the larger the value is, the lower the ID conversion times and the higher the precision are represented. According to the embodiment of the application, indexes of different methods are obtained by testing a plurality of methods under 5 satellite videos under a VISO data set, the specific index conditions are shown in a table 1, wherein the higher the tracking effect of IDF1 is, the better the tracking effect of IDs is, the lower the tracking effect of IDs is, and the highest IDF1 of the embodiment of the application is in the table 1, so that the multi-target tracking method provided by the embodiment of the application is better than other methods.
The multi-target tracking method based on satellite video data association provided by the embodiment of the application has higher tracking accuracy in the satellite video multi-target tracking process than other technologies. Accuracy of multi-target tracking (MOTA), which is embodied in determining the number of targets and accuracy in relation to the relevant properties of the targets, is used to count the accumulation of errors in tracking. MOTA is calculated by indexes such as GT (true trace), FP (false positive), FN (false negative) and the like. The higher MOTA and MOTP indicate the better tracking effect; lower FP, FN indicates better tracking. The experimental results in Table 1 show that the embodiment of the application can realize more representativeness under satellite videoThe target tracking technique (FairMOT, deepsort, TGraM) has a higher multi-target tracking accuracy (MOTA) and multi-target tracking accuracy (MOTP). The calculation formulas of MOTA and MOTP are shown as follows, and GT, FP and FN in the formula (4) respectively represent indexes such as real track, false positive, false negative and the like. C in formula (5) t Represents the number of successful matches in the t frame, d t Representing the distance between the detection target and the kalman filter prediction result.
TABLE 1 tracking index under the same test set
| IDF1 | FP | FN | IDs | MOTA | MOTP | |
| FairMOT | 22.3％ | 4014 | 51454 | 3036 | 8.4％ | 0.523 |
| Deepsort | 28.6％ | 2458 | 30120 | 1485 | 12.5％ | 0.214 |
| TGram | 32.6％ | 3240 | 45621 | 2548 | 13.3％ | 0.475 |
| The application is that | 78.6％ | 1405 | 17681 | 161 | 69.9％ | 0.307 |
As can be seen from the above analysis, in the multi-target tracking method based on satellite video data association provided by the embodiment of the present application, a detection threshold of a detection model is set first, and a target detection frame is divided into a high-resolution detection frame and a low-resolution detection frame; the detection model and the detection threshold are subjected to fine adjustment to enable the detection model to be suitable for the data characteristics of a large-size satellite video small-target scene, and then the detection model is adopted to detect multiple targets in the current frame of the satellite video to obtain high-score detection frame data and low-score detection frame data; performing primary data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; when the first data association of the partial predicted track is unsuccessful, carrying out the second data association on the low-score detection frame data and the partial predicted track to form a second track; and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track. The simultaneous positioning and tracking of multiple targets are realized through the association of the high-low detection frame data, and compared with the satellite multi-target tracking method in the prior art, the satellite multi-target tracking method has the advantages of lower calculation complexity, higher tracking accuracy, low ID conversion times and the like.
Example two
As shown in fig. 1-2, the present embodiment provides a multi-target tracking system based on satellite video data association, the multi-target tracking system comprising:
the detection rule setting module is used for setting a detection threshold value of the detection model and dividing the target detection frame into a high-score detection frame and a low-score detection frame. Before online detection is carried out on a plurality of targets in a current frame by using a YOLOX detection model, a detection threshold of the detection model is set according to the size of a target scale, and a target detection frame is divided into a high-resolution detection frame and a low-resolution detection frame. Specific operations may be referred to the following formulas.
A in the formula (1) represents a confidence threshold value of the detection frame, and D in the formula (2) high Representing a high score detection frame set, D low Representing a low-score collection of test frames, d.score represents the confidence of the test frames. Confidence refers to reliability, i.e., the likelihood that an object within the detection frame is a real target. The confidence threshold is set according to the data, when the confidence is larger than the confidence threshold, the detection frame is initialized to be a target tracking track, and di is added into Dhigh or Dlow.
The detection module is used for detecting multiple targets in the current frame of the satellite video by adopting a detection model to obtain high-resolution detection frame data and low-resolution detection frame data. The trained detection model is used to detect the target in the current frame online, where the detection model may be a YOLOX detection model. According to the embodiment of the application, a satellite video data association method with high and low sub-frames for secondary association is provided according to the data characteristics of the satellite video, most of low sub-detection frames are false alarms, but the confidence of the blocked targets is low, and in the two-time association method, the second sub-detection frames for secondary data association can well track the blocked targets.
And the first data association module is used for carrying out first data association on the high-resolution detection frame data and the multi-target predicted track in the current frame to form a first track. Optionally, the first data association module is specifically configured to: calculating the cross ratio IOU of the high-score detection frame data and the multi-target prediction track in the current frame; and calculating the cross-over ratio by adopting a Hungary algorithm, and performing similarity matching to obtain a first track.
IOU is calculated for the high-score detection frame data and the predicted track. The Hungary algorithm is a combined optimization algorithm for solving task allocation problems in polynomial time, and in the embodiment of the application, the optimal solutions of the corresponding tracks of a plurality of targets are found by carrying out first data association on the targets and the tracks through the Hungary algorithm. If there is high-resolution detection frame data which is not matched, initializing the track of the new target by the partial high-resolution detection frame data.
And the second data association module is used for carrying out second data association on the low-resolution detection frame data and the part of the predicted track to form a second track when the first data association of the part of the predicted track is unsuccessful. The low-score detection frame mainly comprises targets such as false alarm, shielding, motion blurring, size transformation and the like, so that the low-score detection frame and part of the prediction tracks which are not matched are enabled to calculate the IOU, then data association is carried out through a Hungary algorithm, if the rest of the prediction tracks which are not associated and matched are reserved for a set number of frames, and if the rest of the prediction tracks are not associated and matched, the rest of the prediction tracks are deleted. And deleting if the low-score detection frame which is not associated with the matching appears.
And the result output module is used for outputting a tracking result of the multiple targets in the current frame based on the first track and the second track. After each frame in the satellite video is subjected to target detection, the detection results are subjected to data correlation so as to output a complete multi-target motion trail. The data association can calculate the similarity between the detection frame data of the current frame and the multi-target prediction track in the current frame through the IOU, and the association matching is carried out according to the similarity through the Hungary algorithm. Firstly, dividing the detection result of the current frame into high-resolution detection frame data and low-resolution detection frame data, in each frame, firstly, associating the predicted track of multiple targets in the current frame with the high-resolution detection frame data, then associating the low-resolution detection frame data with part of the unmatched predicted tracks, distinguishing the background from the real targets through the similarity of target tracking tracks, and deleting the tracks if the number of the rest predicted tracks exceeds the set frame number. And initializing unmatched detection frame data into the track of the new target.
Optionally, the multi-target tracking system provided by the embodiment of the present application further includes a modeling module, configured to: training a detection model by adopting satellite video data; and adjusting the super parameters of the detection model based on the satellite video data to obtain the detection weight of the detection model on the satellite video data. And inputting satellite video data to be detected into the detection model based on the detection weight, outputting detection frame data, and dividing the detection frame data into high-resolution detection frame data and low-resolution detection frame data based on a detection threshold value.
The embodiment of the application designs a satellite video data association method for secondary association of high and low frames according to the data characteristics of satellite videos: because of the false alarms when the low-resolution detection frames are mostly, the low-resolution detection frame data associated with the second data can track some blocked targets well in the two-time association. Specifically, the detection algorithm of the YOLOX detection model is finely adjusted through the data set characteristics of the VISO satellite video, unlike the ground video, the satellite video has the characteristics of complex background, fewer available target characteristics, small targets and the like, so that the detected target confidence is very low, and the parameters of the YOLOX can be correspondingly adjusted according to experimental conditions. When the target size is smaller, the confidence coefficient threshold of the detection frame can be properly adjusted down, so that more detected targets are transferred into the tracking track; when the target is large and clear, the confidence threshold of the detection frame can be properly increased, so that false alarms are reduced, and the accuracy is improved.
As can be seen from the above analysis, in the multi-target tracking method based on satellite video data association provided by the embodiment of the present application, a detection threshold of a detection model is set first, and a target detection frame is divided into a high-resolution detection frame and a low-resolution detection frame; the detection model and the detection threshold are subjected to fine adjustment to enable the detection model to be suitable for the data characteristics of a large-size satellite video small-target scene, and then the detection model is adopted to detect multiple targets in the current frame of the satellite video to obtain high-score detection frame data and low-score detection frame data; performing primary data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; when the first data association of the partial predicted track is unsuccessful, carrying out the second data association on the low-score detection frame data and the partial predicted track to form a second track; and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track. The simultaneous positioning and tracking of multiple targets are realized through the association of the high-low detection frame data, and compared with the satellite multi-target tracking method in the prior art, the satellite multi-target tracking method has the advantages of lower calculation complexity, higher tracking accuracy, low ID conversion times and the like.
Example III
The present embodiment provides a storage medium for computer-readable storage, the storage medium storing one or more programs which, when executed by one or more processors, implement a multi-objective tracking method based on satellite video data correlation as described above.
As can be seen from the above analysis, in the multi-target tracking method based on satellite video data association provided by the embodiment of the present application, a detection threshold of a detection model is set first, and a target detection frame is divided into a high-resolution detection frame and a low-resolution detection frame; the detection model and the detection threshold are subjected to fine adjustment to enable the detection model to be suitable for the data characteristics of a large-size satellite video small-target scene, and then the detection model is adopted to detect multiple targets in the current frame of the satellite video to obtain high-score detection frame data and low-score detection frame data; performing primary data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track; when the first data association of the partial predicted track is unsuccessful, carrying out the second data association on the low-score detection frame data and the partial predicted track to form a second track; and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track. The simultaneous positioning and tracking of multiple targets are realized through the association of the high-low detection frame data, and compared with the satellite multi-target tracking method in the prior art, the satellite multi-target tracking method has the advantages of lower calculation complexity, higher tracking accuracy, low ID conversion times and the like.
In summary, the foregoing description is only a preferred embodiment of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification should be included in the protection scope of the present specification.
The systems, devices, modules, or units illustrated in one or more of the embodiments described above may be implemented in particular by a computer chip or entity, or by a product having some function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Claims (10)
1. The multi-target tracking method based on satellite video data association is characterized by comprising the following steps of:
setting a detection threshold of the detection model, and dividing a target detection frame into a high-resolution detection frame and a low-resolution detection frame;
detecting multiple targets in the current frame of the satellite video data by adopting the detection model to obtain high-score detection frame data and low-score detection frame data; performing first data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track;
when the first data association of part of the predicted tracks is unsuccessful, carrying out the second data association on the low-score detection frame data and the part of the predicted tracks to form a second track;
and outputting a tracking result of the multiple targets in the current frame based on the first track and the second track.
2. The method of claim 1, wherein before setting a detection threshold of a detection model and dividing a target detection frame into a high score detection frame and a low score detection frame, the method further comprises:
training a detection model by adopting the space data;
and adjusting the super parameters of the detection model based on the satellite video data to obtain the detection weight of the detection model on the satellite video data.
3. The satellite video data correlation-based multi-target tracking method of claim 1, wherein the high score detection frame data is first data correlated with a predicted trajectory to form a first trajectory, the method further comprising:
and predicting the positions of the multiple targets in the current frame based on the tracking result of the multiple targets in the previous frame to obtain a predicted track.
4. The multi-target tracking method based on satellite video data association according to claim 1, wherein the first data association is performed between the high-score detection frame data and a predicted track of a plurality of targets in a current frame to form a first track, and specifically comprises:
calculating an intersection ratio IOU of the high-score detection frame data and the predicted track;
and matching the similarity between the high-resolution detection frame and the predicted track based on the intersection ratio by adopting a Hungary algorithm to obtain the first track.
5. The multi-target tracking method based on satellite video data association according to any one of claims 1-4, wherein the high-score detection frame data is data-associated with the predicted trajectory of the multi-target in the current frame a first time, and after forming the first trajectory, the method further comprises:
and initializing partial high-score detection frame data as the track of a new target when the first data association of partial high-score detection frame data in the high-score detection frame data is unsuccessful.
6. The satellite video data correlation-based multi-target tracking method of claim 5, wherein when the partial predicted trajectory first data correlation is unsuccessful, the low-score detection frame data is data correlated with the partial predicted trajectory a second time to form a second trajectory, the method comprising:
when the second data association of the residual predicted track in the partial predicted track is unsuccessful, reserving the residual predicted track with a set frame number; the method comprises the steps of,
and deleting part of the low-score detection frame data when the second data association of the part of the low-score detection frame data is unsuccessful.
7. A multi-target tracking system based on satellite video data association, comprising:
the detection rule setting module is used for setting a detection threshold value of the detection model and dividing the target detection frame into a high-score detection frame and a low-score detection frame;
the detection module is used for detecting multiple targets in the current frame of the satellite video data by adopting the detection model to obtain high-score detection frame data and low-score detection frame data;
the first data association module is used for carrying out first data association on the high-score detection frame data and the predicted track of the multiple targets in the current frame to form a first track;
the second data association module is used for carrying out second data association on the low-score detection frame data and the part of the predicted tracks to form a second track when the first data association of the part of the predicted tracks in the predicted tracks is unsuccessful;
and the result output module is used for outputting a tracking result of multiple targets in the current frame based on the first track and the second track.
8. The multi-target tracking system of claim 7, further comprising a modeling module for:
training a detection model by adopting the space data;
and adjusting the super parameters of the detection model based on the satellite video data to obtain the detection weight of the detection model on the satellite video data.
9. The multi-target tracking system of claim 8, wherein the first association module is configured to:
calculating an intersection ratio IOU of the high-score detection frame data and the predicted track;
and matching the similarity between the high-resolution detection frame and the predicted track based on the intersection ratio by adopting a Hungary algorithm to obtain the first track.
10. A storage medium for computer readable storage, the storage medium storing one or more programs which, when executed by one or more processors, implement a multi-objective tracking method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310660081.7A CN116704368A (en) | 2023-06-05 | 2023-06-05 | Multi-target tracking method, system and storage medium based on satellite video data association |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310660081.7A CN116704368A (en) | 2023-06-05 | 2023-06-05 | Multi-target tracking method, system and storage medium based on satellite video data association |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116704368A true CN116704368A (en) | 2023-09-05 |
Family
ID=87827110
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310660081.7A Pending CN116704368A (en) | 2023-06-05 | 2023-06-05 | Multi-target tracking method, system and storage medium based on satellite video data association |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116704368A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117670940A (en) * | 2024-01-31 | 2024-03-08 | 中国科学院长春光学精密机械与物理研究所 | Single-stream satellite video target tracking method based on correlation peak value distance analysis |
-
2023
- 2023-06-05 CN CN202310660081.7A patent/CN116704368A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117670940A (en) * | 2024-01-31 | 2024-03-08 | 中国科学院长春光学精密机械与物理研究所 | Single-stream satellite video target tracking method based on correlation peak value distance analysis |
| CN117670940B (en) * | 2024-01-31 | 2024-04-26 | 中国科学院长春光学精密机械与物理研究所 | Single-stream satellite video target tracking method based on correlation peak value distance analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112308881B (en) | Ship multi-target tracking method based on remote sensing image | |
| CN108875548B (en) | Character track generation method and device, storage medium and electronic equipment | |
| CN106971401B (en) | Multi-target tracking device and method | |
| CN111476826A (en) | Multi-target vehicle tracking method based on SSD target detection | |
| CN108875533B (en) | Face recognition method, device, system and computer storage medium | |
| CN106570851B (en) | A kind of notable figure fusion method based on weight assignment D-S evidence theory | |
| CN116704368A (en) | Multi-target tracking method, system and storage medium based on satellite video data association | |
| CN110991397B (en) | Travel direction determining method and related equipment | |
| CN110070560B (en) | Object motion direction identification method based on target detection | |
| CN115830075A (en) | Hierarchical association matching method for pedestrian multi-target tracking | |
| CN111104925A (en) | Image processing method, image processing apparatus, storage medium, and electronic device | |
| CN112541403B (en) | Indoor personnel falling detection method by utilizing infrared camera | |
| CN116403139A (en) | Visual tracking and positioning method based on target detection | |
| CN116311063A (en) | Personnel fine granularity tracking method and system based on face recognition under monitoring video | |
| CN116129328A (en) | Method, device, equipment and storage medium for detecting carryover | |
| US20200211202A1 (en) | Fall detection method, fall detection apparatus and electronic device | |
| Nair et al. | Face recognition and tracking for security surveillance | |
| CN112070035A (en) | Target tracking method and device based on video stream and storage medium | |
| CN116958873A (en) | Pedestrian tracking method, device, electronic equipment and readable storage medium | |
| CN116563345A (en) | Multi-target tracking method based on point-surface matching association | |
| CN116523957A (en) | Multi-target tracking method, system, electronic equipment and storage medium | |
| CN106991684B (en) | Foreground extracting method and device | |
| CN114037950B (en) | Multi-row person tracking method and device based on pedestrian and head detection | |
| CN114494349A (en) | Video tracking system and method based on target feature space-time alignment | |
| CN113721240A (en) | Target association method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |