CN109934849A - Online multi-object tracking method based on track metric learning - Google Patents
Online multi-object tracking method based on track metric learning Download PDFInfo
- Publication number
- CN109934849A CN109934849A CN201910176113.XA CN201910176113A CN109934849A CN 109934849 A CN109934849 A CN 109934849A CN 201910176113 A CN201910176113 A CN 201910176113A CN 109934849 A CN109934849 A CN 109934849A
- Authority
- CN
- China
- Prior art keywords
- track
- target
- moment
- detection
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses a kind of online multi-object tracking method based on track metric learning, the technical issues of the practicability is poor for solving existing online multi-object tracking method.Technical solution is to generate detection response using existing algorithm of target detection first；Then, it is high confidence set and low confidence set by existing track set-partition, the data connection problem of high confidence set and subsequent time detection response is handled using static nature and traditional measure method, it is directed to the data connection capabilities of low confidence set using the enhancing of similarity measurements moment matrix, obtains final result.The present invention using existing trace information as training sample set, on-line study track and detection response between similarity measurements moment matrix, enhancing track differentiate resolution capability.It solves the technical issues of background technique method is difficult to accurately to carry out data connection, tracking effect is severely limited by detection effect, improves multiple target tracking effect, practicability is good.
Description
Technical field
It is the present invention relates to a kind of online multi-object tracking method, in particular to a kind of based on the online more of track metric learning
Method for tracking target.
Background technique
Vision multiple target tracking can will be examined in the case where given video sequence and object detection results in different moments
It surveys response to correspond, finally obtains target dynamic changes in time domain and airspace.As visual perception and understanding
An important ring, multiple target tracking can efficiently extract the motion information and timing variations information of target, can assist intelligence
Equipment can it is accurate in time domain and airspace, robustly understand ambient enviroment.Such as: multiple target tracking is applied to intelligent monitoring
System, it will help abnormal object is detected；Applied to intelligent vehicle, it will help avoid the traffic accidents such as knock into the back, collide
Generation；Applied to intelligent robot, it will help tracking, crawl dynamic object.In view of smart machine shooting platform not
Fixed, visual angle is indefinite and scene in target it is more crowded, often inadequate robust, practicability be not high for traditional multiple target tracking.How
Designing the multiple target tracking algorithm that a kind of stability is high, robustness is high, real-time is good is urgently open question.Therefore, complicated
Multiple target tracking has a good application prospect and has important theoretical significance under scene.
According to the difference for using video data mode, existing vision multi-target tracking algorithm is broadly divided into: offline more
Target algorithm and online multi-objective Algorithm.Off-line method mainly solves the continuous moment using the global information in video sequence
Data connection problem between middle detection response, avoids ambiguity.Wang et al. is in document " B.Wang, G.Wang, K.Chan and
L.Wang.Tracklet Association by Online Target-Specific Metric Learning and
Coherent Dynamics Estimation.IEEE Transaction on Pattern Analysis and Machine
It is proposed that a kind of level based on expense flow network solves in Intelligence, vol.39, no.3, pp.589-602,2017. "
Scheme, program target detection response first is used as fee flows network node, using Pirsiavash et al. in document
“H.Pirsiavash,D.Ramanan and C.Fowlkes.Globally-optimal Greedy Algorithms for
Tracking a Variable Number of Objects.In Proceedings of IEEE Conference on
Computer Vision and Pattern Recognition, the method that pp.1201-1208,2011. " is proposed obtain just
Then beginning target following using track as a result, learn initial results being divided into short and robust tracking segment, finally by these
Tracking segment is solved as the node in expense flow network.And mainly believed using the vision at cut-off current time in line method
It ceases to handle the data connection problem between continuous existing track and subsequent time detection response.Sadeghian et al. is in document
“A.Sadeghian,A.Alahi and S.Savarese.Tracking the Untrackable:Learning to
Track Multiple Cues with Long-Term Dependencies.In Proceedings of IEEE
It is proposed that one kind is directed in International Conference on Computer Vision, pp.4696-4704,2015. "
The end-to-end deep neural network of target detection similarity study, which pertains generally to Recognition with Recurrent Neural Network, and utilizes three
Sub-network calculates separately the similitude of relevance between appearance characterization, movement, target, obtains existing target trajectory and subsequent time is examined
The similarity degree of response is surveyed, and then realizes the data connection between target trajectory and subsequent time object detection information.
Existing method has its limitation, and off-line method can not be suitable for the practical application of high real-time, and path segment
Between data connection robustness and accuracy it is to be improved；And online multi-objective Algorithm not only critical constraints detection effect is good
It is bad, and the data connection precision needs between existing path segment and subsequent time detection response further increase.
Summary of the invention
In order to overcome the shortcomings of existing online multi-object tracking method, the practicability is poor, and the present invention provides a kind of based on track degree
Measure the online multi-object tracking method of study.This method generates detection response using existing algorithm of target detection first；Then,
It is high confidence set and low confidence set by existing track set-partition, handles height using static nature and traditional measure method
The data connection problem of confidence set and subsequent time detection response is directed to low confidence set using the enhancing of similarity measurements moment matrix
Data connection capabilities, obtain final result.The present invention is using existing trace information as training sample set, on-line study track and inspection
Survey the similarity measurements moment matrix between response, the resolution capability that enhancing track differentiates.Solve background technique method be difficult to accurately into
Row data connection, tracking effect are severely limited by detection effect.Connected by the data of the judgement of track confidence level, high confidence track
Connect, the models such as metric learning, the study of target context dynamic relationship of track and detection response cooperate, improve more mesh
Tracking effect is marked, practicability is good.
A kind of the technical solution adopted by the present invention to solve the technical problems: online multiple target based on track metric learning
Tracking, its main feature is that the following steps are included:
Step 1: considering that length, the target of track are blocked degree, the level of intimate of track and detection response, track
The factors such as smoothness, planned course confidence level function:
WhereinIndicate the path length by the end of i-th of target of t moment,Indicate i-th of target in the k moment
Level of intimate between its corresponding detection response,Indicate that i-th of target of k moment is not blocked degree, TkFor k
Moment all target trajectories, smo (Tt i) indicate by the end of the track of i-th of target of t moment smoothness.
Step 2: for the track T of i-th of target of t momentt i, using Kalman filter estimation target in the position at k moment
It setsConsidering vision, the kinematics information of target, the level of intimate detection response its corresponding between target calculates,
Formula is as follows:
WhereinWithRespectively indicate the close journey obtained using visual information and kinematics information
Degree.It is indicated using the friendship and ratio of target state estimator position and detection response position, it may be assumed that
WhereinWithIt respectively indicatesPlace rectangular area.Using Pasteur's coefficient come
It calculates, formula is as follows:
WhereinIt isI-th of component of the hsv color histogram of place rectangular area, Bs indicate that hsv color is straight
The component sum of square figure, Bs=48.
Step 3: consider that some time inscribes the hiding relation of target Yu other targets, calculating target trajectory is not blocked degreeFormula is as follows:
Wherein NtIt is target trajectory number in kth frame.
Step 4: calculating track T by comparison target trajectory and smooth trackt iSmoothness smo (Tt i).It will three times
B-spline curves are smoothly used as trajectory smoothing method, obtain track Tt iSharpening result beIt is smooth trackAt the k moment
Position, and then track Tt iSmoothness smo (Tt i), formula is as follows:
WhereinIt is off the path length to i-th of target of t frame.
Step 5: calculating track Tt iConfidence level confidence (Tt i), 0.5 is set by track confidence threshold value, when
confidence(Tt i) > 0.5, then Tt iFor high confidence level track；Otherwise, Tt iFor low confidence track.
Tt iWhen for high confidence level track:
Bipartite graph weight limit assignment problem is converted by the data connection problem of high confidence track, wherein high confidence level rail
Mark is considered as vertex set A, and the target detection response of subsequent time is considered as vertex set B, the view between track and subsequent time detection response
Feel similarity as the side right between corresponding vertex.
For the vision similarity between track and subsequent time detection response calculates, t moment track T is only considered heret i
RegionDetection with the t+1 moment responds, and uses following formula computation vision similarity:
WhereinIt isI-th of component of the hsv color histogram of place rectangular area, Bs indicate that hsv color is straight
The component sum of square figure.
Bipartite graph weight limit assignment problem is solved using Kuhn-Munkres algorithm, and converts high confidence rail for result
The tracking result of mark.
Tt iWhen for low confidence track:
Image-region involved in target trajectory is chosen in nearest 10 frame as sample.
For each sample, adjusts to same size, then extract HSV the and HOG visual signature of sample.For i-th
Sample of the target in t momentFor,WithRespectively indicate sampleHSV and HOG visual signature, depending on
Feel feature are as follows:
Two samples for belonging to same target are considered as positive sample pair, two samples for belonging to different target are considered as negative sample
This is right.For i-th of target, the absolute distance of positive sample pair isWherein t1≠t2,
The absolute distance of negative sample pair isWherein i ≠ j.Measurement is constructed using mahalanobis distance
Function f (d)i, formula is as follows:
f(d)i=dTMid
Wherein MiIt is positive semidefinite matrix.Given positive sample pair absolute distance vector sum negative sample pair absolute distance to
In the case of amount, learn a metric matrix Mi, so that the f (d) of positive sample pairiIt is worth as small as possible while negative sample pair as far as possible
Greatly.Design is directed to the risk function r (M of i-th of targeti), by minimizing risk function, solve Mi, and then obtain f (d)i；
Specific risk function is as follows:
Due to MiBe it is positive semi-definite, if MiWhen being higher dimensional matrix, higher time complexity can be brought by directly calculating, therefore
Directly utilize minimum risk function r (Mi) learn metric matrix MiIt is inappropriate.To MiCarry out Eigenvalues Decomposition, i.e. Mi
=ABAT=WiWi T, wherein Wi=AB1/2, by calculating WiM is obtained indirectlyi.Final objective function and constraint condition is as follows:
WhereinWithRespectively indicate WiJth column and kth column, using Zheng et al. in document " W.Zheng, S.Gong
and T.Xiang.Reidentification by Relative Distance Comparison.IEEE
Transactions on Pattern Analysis and Machine Intelligence,2013,35(3):653-
668. " the iterative algorithms proposed solve metric matrix Wi.To each track, learns its corresponding metric matrix, measured
Set of matrices W={ Wi, wherein i=1,2 ..., N, N are the target trajectory number of t moment.
For the sample at t+1 momentThe sample and detection for calculating i-th of target first respondAbsolute distance, meter
Calculation method is as follows:
Then track sample is calculatedIt is responded with detectionCorrelation degreeFinally integrate institute
There are sample and detection to respondAssociation situation, i.e.,
Calculate the correlation degree between all low confidence tracks and the response of non-matching detection.Then by low confidence track
Data connection problem is converted into bipartite graph weight limit assignment problem, and wherein low confidence track set is considered as vertex set A, next
The non-matching detection response sets at moment are considered as vertex set B, the correlation degree between low confidence track and the response of non-matching detection
As the side right between corresponding vertex.Bipartite graph weight limit assignment problem finally is solved using Kuhn-Munkres algorithm optimization,
And convert solving result to the data connection result of low confidence track.
The beneficial effects of the present invention are: this method generates detection response using existing algorithm of target detection first；Then,
It is high confidence set and low confidence set by existing track set-partition, handles height using static nature and traditional measure method
The data connection problem of confidence set and subsequent time detection response is directed to low confidence set using the enhancing of similarity measurements moment matrix
Data connection capabilities, obtain final result.The present invention is using existing trace information as training sample set, on-line study track and inspection
Survey the similarity measurements moment matrix between response, the resolution capability that enhancing track differentiates.Solve background technique method be difficult to accurately into
Row data connection, tracking effect are severely limited by detection effect.Connected by the data of the judgement of track confidence level, high confidence track
Connect, the models such as metric learning, the study of target context dynamic relationship of track and detection response cooperate, improve more mesh
Tracking effect is marked, practicability is good.
It elaborates with reference to the accompanying drawings and detailed description to the present invention.
Detailed description of the invention
Fig. 1 is the flow chart of the online multi-object tracking method the present invention is based on track metric learning.
Fig. 2 is tracking result sample of the method for the present invention in MOT2015 data in PETS09-S2L1 sequence.
Fig. 3 is tracking result sample of the method for the present invention in MOT2015 data in KITTI-17 sequence.
Specific embodiment
Referring to Fig.1-3.The present invention is based on the online multi-object tracking method of track metric learning, specific step is as follows:
Step 1: considering that length, the target of track are blocked degree, the level of intimate of track and detection response, track
The factors such as smoothness, planned course confidence level function:
WhereinIndicate the path length by the end of i-th of target of t moment,Indicate i-th of target in the k moment
Level of intimate between its corresponding detection response,Indicate that i-th of target of k moment is not blocked degree, TkFor k
Moment all target trajectories, smo (Tt i) indicate by the end of the track of i-th of target of t moment smoothness.
Step 2: for the track T of i-th of target of t momentt i, using Kalman filter estimation target in the position at k moment
It setsConsidering vision, the kinematics information of target, the level of intimate detection response its corresponding between target calculates,
Its formula is as follows:
WhereinWithRespectively indicate the close journey obtained using visual information and kinematics information
Degree.It is indicated using the friendship and ratio of target state estimator position and detection response position, it may be assumed that
WhereinWithIt respectively indicatesPlace rectangular area.Using Pasteur's coefficient come
It calculates, the specific method is as follows:
WhereinIt isI-th of component of the hsv color histogram of place rectangular area, Bs indicate that hsv color is straight
The component sum of square figure, Bs=48.
Step 3: consider that some time inscribes the hiding relation of target Yu other targets, calculating target trajectory is not blocked degreeIts method is as follows:
Wherein NtIt is target trajectory number in kth frame.
Step 4: calculating track T by comparison target trajectory and smooth trackt iSmoothness smo (Tt i).It will three times
B-spline curves are smoothly used as trajectory smoothing method, obtain track Tt iSharpening result be It is smooth trackAt the k moment
Position, and then track Tt iSmoothness smo (Tt i), the method is as follows:
WhereinIt is off the path length to i-th of target of t frame.
Step 5: calculating track Tt iConfidence level confidence (Tt i), 0.5 is set by track confidence threshold value, when
confidence(Tt i) > 0.5, then Tt iFor high confidence level track；Otherwise, Tt iFor low confidence track.
Tt iSteps are as follows for realization when for high confidence level track:
Bipartite graph weight limit assignment problem is converted by the data connection problem of high confidence track, wherein high confidence level rail
Mark is considered as vertex set A, and the target detection response of subsequent time is considered as vertex set B, the view between track and subsequent time detection response
Feel similarity as the side right between corresponding vertex.
For the vision similarity between track and subsequent time detection response calculates, t moment track T is only considered heret i
RegionDetection with the t+1 moment responds, and uses following formula computation vision similarity:
WhereinIt isI-th of component of the hsv color histogram of place rectangular area, Bs indicate that hsv color is straight
The component sum of square figure.
Bipartite graph weight limit assignment problem is solved using Kuhn-Munkres algorithm, and converts high confidence rail for result
The tracking result of mark.
Tt iSteps are as follows for realization when for low confidence track:
Image-region involved in target trajectory is chosen in nearest 10 frame as sample.
For each sample, adjusts to same size (a height of 128 pixel, width are 64 pixels), then extract the HSV of sample
With HOG visual signature.With regard to the calculating of HSV feature, need each channel 64 components are arranged；For HOG feature, connected region
The size of cell is 8*8, and the connected region cell of 2*2 merges into a connection block block.For i-th of target in t moment
SampleFor,WithRespectively indicate sampleHSV and HOG visual signature, visual signature are as follows:
Two samples for belonging to same target are considered as positive sample pair, two samples for belonging to different target are considered as negative sample
This is right.For i-th of target, the absolute distance of positive sample pair isWherein t1≠t2,
The absolute distance of negative sample pair isWherein i ≠ j.Measurement is constructed using mahalanobis distance
Function f (d)i, method is as follows:
f(d)i=dTMid
Wherein MiIt is positive semidefinite matrix.Given positive sample pair absolute distance vector sum negative sample pair absolute distance to
In the case of amount, learn a metric matrix Mi, so that the f (d) of positive sample pairiIt is worth as small as possible while negative sample pair as far as possible
Greatly.Design is directed to the risk function r (M of i-th of targeti), by minimizing risk function, solve Mi, and then obtain f (d)i；
Specific risk function is as follows:
Due to MiBe it is positive semi-definite, if MiWhen being higher dimensional matrix, higher time complexity can be brought by directly calculating, therefore
Directly utilize minimum risk function r (Mi) learn metric matrix MiIt is inappropriate.To MiCarry out Eigenvalues Decomposition, i.e. Mi
=ABAT=WiWi T, wherein Wi=AB1/2, by calculating WiM is obtained indirectlyi.Final objective function and constraint condition is as follows:
WhereinWithRespectively indicate WiJth column and kth column, using Zheng et al. in document " W.Zheng, S.Gong and
T.Xiang.Reidentification by Relative Distance Comparison.IEEE Transactions on
Pattern Analysis and Machine Intelligence, 2013,35 (3): the iteration that 653-668. " is proposed is calculated
Method solves metric matrix Wi.To each track, learn its corresponding metric matrix, obtains metric matrix set W={ Wi, wherein i
=1,2 ..., N, N are the target trajectory number of t moment.
For the sample at t+1 momentThe sample and detection for calculating i-th of target first respondAbsolute distance, meter
Calculation method is as follows:
Then track sample is calculatedIt is responded with detectionCorrelation degreeFinally integrate institute
There are sample and detection to respondAssociation situation, i.e.,
Calculate the correlation degree between all low confidence tracks and the response of non-matching detection.Then by low confidence track
Data connection problem is converted into bipartite graph weight limit assignment problem, and wherein low confidence track set is considered as vertex set A, next
The non-matching detection response sets at moment are considered as vertex set B, the correlation degree between low confidence track and the response of non-matching detection
As the side right between corresponding vertex.Bipartite graph weight limit assignment problem finally is solved using Kuhn-Munkres algorithm optimization,
And convert solving result to the data connection result of low confidence track.
Emulation experiment:
It is in central processing unitOn i5-3470 3.2GHz CPU, memory 4G, Centos system, with MATLAB
Software carries out emulation experiment.
The experimental data used are as follows: MOT2015 data set.
For the performance of evaluation algorithms, with reference to Bernardin et al., Ristani et al. and Lee et al. respectively in document
“K.Bernardin and R.Stiefelhagen.Evaluating Multiple Object Tracking
Performance:the CLEAR MOT Metrics.Journal on Image and Video Processing,vol
1,pp.1-10,2008.”、“E.Ristani,F.Solera,R.Zou,R.Cucchiara and
C.Tomasi.Performance Measures and a Data Set for Multi-target,Multi-camera
Tracking.In Proceedings of European Conference on Computer Vision,pp.17-35,
2016. " and " Y.Li, C.Huang and R.Nevatia.Learning to Associate:Hybridboosted
The method that Multi-target Tracker for Crowded Scene, pp.2953-2960,2009. " are proposed is selected
FAF (English name False Alarms Per Frame), MT (English name Mostly Tracked Targets), ML (English
Literary fame claims Mostly Lost Targets), FP (English name False Positives), FN (English name False
Negatives), IDs (English name Identity Switches), FM (English name Fragmentations), MOTA (English
Literary fame claims Multiple Object Tracking Accuracy), MOTP (English name Multiple Object
Tracking Precision) etc. indexs never ipsilateral reflect tracking effect.Meanwhile the validity in order to prove algorithm, choosing
Selected the methods of OMT_DFH, JPDA_m, TBX algorithm as a comparison, OMT_DFH algorithm document " J.Ju, D.Kim, B.Ku,
D.Han and H.Ko.Online Multi-object Tracking with Efficient Track Drift and
There is detailed introduction in Fragmentation Handling.JOSA A, vol.34, no.2, pp.280-293,2017. "；
JPDA_m algorithm is document " S.Hamid Rezatofighi, A.Milan, Z.Zhang, Q.Shi, A.Dick and
I.Reid.Joint Probabilistic Data Association Revisited,In Proceedings of IEEE
It is proposed in International Conference on Computer Vision, pp.3047-3055,2015. "；TBX is calculated
Method is document " R.Henschel, L.Leal-Taix é, B.Rosenhahn and K.Schindler.Tracking with
Multi-level Features, what arXiv:1607.07304,2016. " was proposed.
Comparing result is as shown in table 1:
Evaluation result of 1 algorithms of different of table on 2015 test set of MOT
As seen from Table 1, the multiple target tracking effect of the method for the present invention is substantially better than existing multiple target tracking effect, from
This three indexs of MOTA, MT, IDs can be seen that the method for the present invention when easily generating long, stable target trajectory, and avoid
The data connection of mistake, the accuracy and robustness of algorithm greatly improve.
Fig. 2 and Fig. 3 shows the method for the present invention in MOT2015 data in PETS09-S2L1 and KITTI-17 sequence respectively
Tracking result sample.The method of the present invention target trajectory generated is relatively stable as can be seen from Figures 2 and 3, and accuracy
It is higher.
Claims (1)
1. a kind of online multi-object tracking method based on track metric learning, it is characterised in that the following steps are included:
Step 1: consider track length, target be blocked degree, track and detection respond level of intimate, track it is smooth
The factors such as degree, planned course confidence level function:
WhereinIndicate the path length by the end of i-th of target of t moment,Indicate the k moment in i-th of target and its
Level of intimate between corresponding detection response,Indicate that i-th of target of k moment is not blocked degree, TkFor the k moment
All target trajectories, smo (Tt i) indicate by the end of the track of i-th of target of t moment smoothness；
Step 2: for the track T of i-th of target of t momentt i, using Kalman filter estimation target in the position at k momentConsider vision, the kinematics information of target, the level of intimate detection response its corresponding between target calculates, public
Formula is as follows:
WhereinWithRespectively indicate the level of intimate obtained using visual information and kinematics information；It is indicated using the friendship and ratio of target state estimator position and detection response position, it may be assumed that
WhereinWithIt respectively indicatesPlace rectangular area；It is calculated using Pasteur's coefficient,
Formula is as follows:
WhereinIt isI-th of component of the hsv color histogram of place rectangular area, Bs indicate hsv color histogram
Component sum, Bs=48；
Step 3: consider that some time inscribes the hiding relation of target Yu other targets, calculating target trajectory is not blocked degreeFormula is as follows:
Wherein NtIt is target trajectory number in kth frame；
Step 4: calculating track T by comparison target trajectory and smooth trackt iSmoothness smo (Tt i)；By cubic B-spline
Curve smoothing obtains track T as trajectory smoothing methodt iSharpening result be It is smooth trackIn the position at k moment
It sets, and then track Tt iSmoothness smo (Tt i), formula is as follows:
WhereinIt is off the path length to i-th of target of t frame；
Step 5: calculating track Tt iConfidence level confidence (Tt i), 0.5 is set by track confidence threshold value, when
confidence(Tt i) > 0.5, then Tt iFor high confidence level track；Otherwise, Tt iFor low confidence track；
Tt iWhen for high confidence level track:
Bipartite graph weight limit assignment problem is converted by the data connection problem of high confidence track, wherein high confidence level track regards
For vertex set A, the target detection response of subsequent time is considered as vertex set B, the vision phase between track and subsequent time detection response
Like degree as the side right between corresponding vertex；
For the vision similarity between track and subsequent time detection response calculates, t moment track T is only considered heret iPlace
RegionDetection with the t+1 moment responds, and uses following formula computation vision similarity:
WhereinIt isI-th of component of the hsv color histogram of place rectangular area, Bs indicate hsv color histogram
Component sum；
Bipartite graph weight limit assignment problem is solved using Kuhn-Munkres algorithm, and converts high confidence track for result
Tracking result；
Tt iWhen for low confidence track:
Image-region involved in target trajectory is chosen in nearest 10 frame as sample；
For each sample, adjusts to same size, then extract HSV the and HOG visual signature of sample；For i-th of target
In the sample of t momentFor,WithRespectively indicate sampleHSV and HOG visual signature, vision is special
Sign are as follows:
Two samples for belonging to same target are considered as positive sample pair, two samples for belonging to different target are considered as negative sample
It is right；For i-th of target, the absolute distance of positive sample pair isWherein t1≠t2, bear
The absolute distance of sample pair isWherein i ≠ j；Measurement letter is constructed using mahalanobis distance
Number f (d)i, formula is as follows:
f(d)i=dTMid
Wherein MiIt is positive semidefinite matrix；In the absolute distance vector feelings of the absolute distance vector sum negative sample pair of given positive sample pair
Under condition, learn a metric matrix Mi, so that the f (d) of positive sample pairiIt is worth as small as possible while negative sample pair as big as possible；
Design is directed to the risk function r (M of i-th of targeti), by minimizing risk function, solve Mi, and then obtain f (d)i；Tool
The risk function of body is as follows:
Due to MiBe it is positive semi-definite, if MiWhen being higher dimensional matrix, higher time complexity can be brought by directly calculating, therefore directly
Utilize minimum risk function r (Mi) learn metric matrix MiIt is inappropriate；To MiCarry out Eigenvalues Decomposition, i.e. Mi=ABAT
=WiWi T, wherein Wi=AB1/2, by calculating WiM is obtained indirectlyi；Final objective function and constraint condition is as follows:
WhereinWithRespectively indicate WiJth column and kth column, use iterative algorithm solve metric matrix Wi；To each track,
Learn its corresponding metric matrix, obtains metric matrix set W={ Wi, wherein i=1,2 ..., N, N are the target track of t moment
Mark number；
For the sample at t+1 momentThe sample and detection for calculating i-th of target first respondAbsolute distance, calculating side
Method is as follows:
Then track sample is calculatedIt is responded with detectionCorrelation degreeFinally integrate all samples
It is responded with detectionAssociation situation, i.e.,
Calculate the correlation degree between all low confidence tracks and the response of non-matching detection；Then by the data of low confidence track
Connectivity problem is converted into bipartite graph weight limit assignment problem, and wherein low confidence track set is considered as vertex set A, subsequent time
Non- matching detection response sets be considered as vertex set B, the correlation degree conduct between low confidence track and the response of non-matching detection
Side right between corresponding vertex；Bipartite graph weight limit assignment problem finally is solved using Kuhn-Munkres algorithm optimization, and will
Solving result is converted into the data connection result of low confidence track.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910176113.XA CN109934849B (en) | 2019-03-08 | 2019-03-08 | Online multi-target tracking method based on trajectory metric learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910176113.XA CN109934849B (en) | 2019-03-08 | 2019-03-08 | Online multi-target tracking method based on trajectory metric learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109934849A true CN109934849A (en) | 2019-06-25 |
| CN109934849B CN109934849B (en) | 2022-05-31 |
Family
ID=66986530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910176113.XA Active CN109934849B (en) | 2019-03-08 | 2019-03-08 | Online multi-target tracking method based on trajectory metric learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109934849B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110415339A (en) * | 2019-07-19 | 2019-11-05 | 清华大学 | The method and apparatus for calculating the matching relationship between input three-dimensional body |
| CN110516717A (en) * | 2019-08-09 | 2019-11-29 | 南京人工智能高等研究院有限公司 | Method and apparatus for generating image recognition model |
| CN110751096A (en) * | 2019-10-21 | 2020-02-04 | 陕西师范大学 | Multi-target tracking method based on KCF track confidence |
| CN113158813A (en) * | 2021-03-26 | 2021-07-23 | 精英数智科技股份有限公司 | Real-time statistical method and device for flow target |
Citations (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783020A (en) * | 2010-03-04 | 2010-07-21 | 湖南大学 | Video multi-target fast tracking method based on joint probability data association |
| CN103955947A (en) * | 2014-03-21 | 2014-07-30 | 南京邮电大学 | Multi-target association tracking method based on continuous maximum energy and apparent model |
| CN104008392A (en) * | 2014-05-09 | 2014-08-27 | 南京邮电大学 | Multi-objective tracking method based on continuous minimum-energy appearance model |
| CN104200488A (en) * | 2014-08-04 | 2014-12-10 | 合肥工业大学 | Multi-target tracking method based on graph representation and matching |
| CN104616324A (en) * | 2015-03-06 | 2015-05-13 | 厦门大学 | Target tracking method based on adaptive appearance model and point-set distance metric learning |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN105678804A (en) * | 2016-01-06 | 2016-06-15 | 北京理工大学 | Real-time on-line multi-target tracking method by coupling target detection and data association |
| CN105809714A (en) * | 2016-03-07 | 2016-07-27 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Track confidence coefficient based multi-object tracking method |
| CN106203260A (en) * | 2016-06-27 | 2016-12-07 | 南京邮电大学 | Pedestrian's recognition and tracking method based on multiple-camera monitoring network |
| CN106373145A (en) * | 2016-08-30 | 2017-02-01 | 上海交通大学 | Multi-target tracking method based on tracking fragment confidence and discrimination appearance learning |
| CN106875421A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of multi-object tracking method and device |
| CN107145827A (en) * | 2017-04-01 | 2017-09-08 | 浙江大学 | Across the video camera pedestrian recognition methods again learnt based on adaptive distance metric |
| CN107292911A (en) * | 2017-05-23 | 2017-10-24 | 南京邮电大学 | A kind of multi-object tracking method merged based on multi-model with data correlation |
| CN107341817A (en) * | 2017-06-16 | 2017-11-10 | 哈尔滨工业大学（威海） | Self-adaptive visual track algorithm based on online metric learning |
| CN107516303A (en) * | 2017-09-01 | 2017-12-26 | 成都通甲优博科技有限责任公司 | Multi-object tracking method and system |
| CN105261040B (en) * | 2015-10-19 | 2018-01-05 | 北京邮电大学 | A kind of multi-object tracking method and device |
| CN107590821A (en) * | 2017-09-25 | 2018-01-16 | 武汉大学 | A kind of method for tracking target and system based on track optimizing |
| CN108470332A (en) * | 2018-01-24 | 2018-08-31 | 博云视觉（北京）科技有限公司 | A kind of multi-object tracking method and device |
| CN108875666A (en) * | 2018-06-27 | 2018-11-23 | 腾讯科技（深圳）有限公司 | Acquisition methods, device, computer equipment and the storage medium of motion profile |
| CN109035290A (en) * | 2018-07-16 | 2018-12-18 | 南京信息工程大学 | A kind of track algorithm updating accretion learning based on high confidence level |
| US20190005657A1 (en) * | 2017-06-30 | 2019-01-03 | Baidu Online Network Technology (Beijing) Co., Ltd . | Multiple targets-tracking method and apparatus, device and storage medium |
| CN109191497A (en) * | 2018-08-15 | 2019-01-11 | 南京理工大学 | A kind of real-time online multi-object tracking method based on much information fusion |
| US20190035088A1 (en) * | 2017-07-31 | 2019-01-31 | National Technology & Engineering Solutions Of Sandia, Llc | Data-driven delta-generalized labeled multi-bernoulli tracker |
-
2019
- 2019-03-08 CN CN201910176113.XA patent/CN109934849B/en active Active
Patent Citations (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101783020A (en) * | 2010-03-04 | 2010-07-21 | 湖南大学 | Video multi-target fast tracking method based on joint probability data association |
| CN103955947A (en) * | 2014-03-21 | 2014-07-30 | 南京邮电大学 | Multi-target association tracking method based on continuous maximum energy and apparent model |
| CN104008392A (en) * | 2014-05-09 | 2014-08-27 | 南京邮电大学 | Multi-objective tracking method based on continuous minimum-energy appearance model |
| CN104200488A (en) * | 2014-08-04 | 2014-12-10 | 合肥工业大学 | Multi-target tracking method based on graph representation and matching |
| CN104616324A (en) * | 2015-03-06 | 2015-05-13 | 厦门大学 | Target tracking method based on adaptive appearance model and point-set distance metric learning |
| CN104915970A (en) * | 2015-06-12 | 2015-09-16 | 南京邮电大学 | Multi-target tracking method based on track association |
| CN105261040B (en) * | 2015-10-19 | 2018-01-05 | 北京邮电大学 | A kind of multi-object tracking method and device |
| CN105678804A (en) * | 2016-01-06 | 2016-06-15 | 北京理工大学 | Real-time on-line multi-target tracking method by coupling target detection and data association |
| CN105809714A (en) * | 2016-03-07 | 2016-07-27 | 广东顺德中山大学卡内基梅隆大学国际联合研究院 | Track confidence coefficient based multi-object tracking method |
| CN106203260A (en) * | 2016-06-27 | 2016-12-07 | 南京邮电大学 | Pedestrian's recognition and tracking method based on multiple-camera monitoring network |
| CN106373145A (en) * | 2016-08-30 | 2017-02-01 | 上海交通大学 | Multi-target tracking method based on tracking fragment confidence and discrimination appearance learning |
| CN106875421A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of multi-object tracking method and device |
| CN107145827A (en) * | 2017-04-01 | 2017-09-08 | 浙江大学 | Across the video camera pedestrian recognition methods again learnt based on adaptive distance metric |
| CN107292911A (en) * | 2017-05-23 | 2017-10-24 | 南京邮电大学 | A kind of multi-object tracking method merged based on multi-model with data correlation |
| CN107341817A (en) * | 2017-06-16 | 2017-11-10 | 哈尔滨工业大学（威海） | Self-adaptive visual track algorithm based on online metric learning |
| US20190005657A1 (en) * | 2017-06-30 | 2019-01-03 | Baidu Online Network Technology (Beijing) Co., Ltd . | Multiple targets-tracking method and apparatus, device and storage medium |
| US20190035088A1 (en) * | 2017-07-31 | 2019-01-31 | National Technology & Engineering Solutions Of Sandia, Llc | Data-driven delta-generalized labeled multi-bernoulli tracker |
| CN107516303A (en) * | 2017-09-01 | 2017-12-26 | 成都通甲优博科技有限责任公司 | Multi-object tracking method and system |
| CN107590821A (en) * | 2017-09-25 | 2018-01-16 | 武汉大学 | A kind of method for tracking target and system based on track optimizing |
| CN108470332A (en) * | 2018-01-24 | 2018-08-31 | 博云视觉（北京）科技有限公司 | A kind of multi-object tracking method and device |
| CN108875666A (en) * | 2018-06-27 | 2018-11-23 | 腾讯科技（深圳）有限公司 | Acquisition methods, device, computer equipment and the storage medium of motion profile |
| CN109035290A (en) * | 2018-07-16 | 2018-12-18 | 南京信息工程大学 | A kind of track algorithm updating accretion learning based on high confidence level |
| CN109191497A (en) * | 2018-08-15 | 2019-01-11 | 南京理工大学 | A kind of real-time online multi-object tracking method based on much information fusion |
Non-Patent Citations (6)
| Title |
|---|
| BING WANG 等: "Tracklet Association by Online Target-Specific Metric Learning and Coherent Dynamics Estimation", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| ERGYS RISTANI 等: "Tracking Multiple People Online and in Real Time", 《ACCV 2014》 * |
| THARINDU RATHNAYAKE 等: "Labeled Multi-Bernoulli Track-Before-Detect for Multi-Target Tracking in Video", 《18TH INTERNATIONAL CONFERENCE ON INFORMATION FUSION》 * |
| YUAN YUAN 等: "Tracking as a Whole: Multi-Target Tracking by Modeling Group Behavior With Sequential Detection", 《IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS》 * |
| 滑维鑫 等: "基于目标性权值度量的多示例学习目标跟踪", 《北京航空航天大学学报》 * |
| 程艳云 等: "基于数据关联与轨迹评估的多目标跟踪", 《江苏通信》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110415339A (en) * | 2019-07-19 | 2019-11-05 | 清华大学 | The method and apparatus for calculating the matching relationship between input three-dimensional body |
| CN110415339B (en) * | 2019-07-19 | 2021-07-13 | 清华大学 | Method and device for calculating matching relation between input three-dimensional shapes |
| CN110516717A (en) * | 2019-08-09 | 2019-11-29 | 南京人工智能高等研究院有限公司 | Method and apparatus for generating image recognition model |
| CN110516717B (en) * | 2019-08-09 | 2022-02-01 | 南京人工智能高等研究院有限公司 | Method and apparatus for generating image recognition model |
| CN110751096A (en) * | 2019-10-21 | 2020-02-04 | 陕西师范大学 | Multi-target tracking method based on KCF track confidence |
| CN110751096B (en) * | 2019-10-21 | 2022-02-22 | 陕西师范大学 | Multi-target tracking method based on KCF track confidence |
| CN113158813A (en) * | 2021-03-26 | 2021-07-23 | 精英数智科技股份有限公司 | Real-time statistical method and device for flow target |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109934849B (en) | 2022-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109934849A (en) | Online multi-object tracking method based on track metric learning | |
| CN107545582B (en) | Video multi-target tracking method and device based on fuzzy logic | |
| CN106443622B (en) | A kind of distributed object tracking based on improvement joint probability data association | |
| Del-Blanco et al. | An efficient multiple object detection and tracking framework for automatic counting and video surveillance applications | |
| CN107907124B (en) | Positioning method based on scene recognition, electronic equipment, storage medium and system | |
| CN103735269B (en) | A kind of height measurement method followed the tracks of based on video multi-target | |
| CN113012203A (en) | High-precision multi-target tracking method under complex background | |
| CN107066806B (en) | Data Association and device | |
| CN103617636A (en) | Automatic video-target detecting and tracking method based on motion information and sparse projection | |
| Xiong et al. | LXL: LiDAR excluded lean 3D object detection with 4D imaging radar and camera fusion | |
| Liu et al. | Dynamic RGB-D SLAM based on static probability and observation number | |
| Yao et al. | Robust RGB-D visual odometry based on edges and points | |
| CN112634368A (en) | Method and device for generating space and OR graph model of scene target and electronic equipment | |
| Efrat et al. | Semi-local 3d lane detection and uncertainty estimation | |
| Fakhfakh et al. | Bayesian curved lane estimation for autonomous driving | |
| Zhuoins et al. | 4drvo-net: Deep 4d radar–visual odometry using multi-modal and multi-scale adaptive fusion | |
| Yi et al. | Multi-Person tracking algorithm based on data association | |
| Song et al. | DGM-VINS: Visual-Inertial SLAM for Complex Dynamic Environments with Joint Geometry Feature Extraction and Multiple Object Tracking | |
| US20210295559A1 (en) | Extrinsic calibration of multi-camera system | |
| Mohedano et al. | Camera localization usingtrajectories and maps | |
| Jiang et al. | Tracking multiple targets based on min-cost network flows with detection in RGB-D data | |
| Xi et al. | A* algorithm with dynamic weights for multiple object tracking in video sequence | |
| Rozsa et al. | Virtually increasing the measurement frequency of LIDAR sensor utilizing a single RGB camera | |
| Correa et al. | Real time people tracking in crowded environments with range measurements | |
| Meng et al. | An ellipse feature tracking method based on the kalman filter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |