CN110288627A - One kind being based on deep learning and the associated online multi-object tracking method of data - Google Patents

One kind being based on deep learning and the associated online multi-object tracking method of data Download PDF

Info

Publication number
CN110288627A
CN110288627A CN201910429444.XA CN201910429444A CN110288627A CN 110288627 A CN110288627 A CN 110288627A CN 201910429444 A CN201910429444 A CN 201910429444A CN 110288627 A CN110288627 A CN 110288627A
Authority
CN
China
Prior art keywords
target
detection
state
frame
detection response
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910429444.XA
Other languages
Chinese (zh)
Other versions
CN110288627B (en
Inventor
陈小波
冀建宇
王彦钧
蔡英凤
王海
陈龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu University
Original Assignee
Jiangsu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu University filed Critical Jiangsu University
Priority to CN201910429444.XA priority Critical patent/CN110288627B/en
Publication of CN110288627A publication Critical patent/CN110288627A/en
Application granted granted Critical
Publication of CN110288627B publication Critical patent/CN110288627B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses one kind to be based on deep learning and the associated online multi-object tracking method of data, includes the following steps: the image of 1, input video present frame；2, application target detector obtains detection response all in image；3, the external appearance characteristic of depth cosine metric learning model extraction detection response is utilized；4, initialized target state；5, the position and scale using Kalman filtering algorithm prediction target in next frame；6, target is associated with based on two stages data correlation with the matching of detection response, obtains optimal association results；7, according to the state and feature of the optimal association results more fresh target in step 6；8, the image of next video frame is inputted, step 2,3,4,5,6,7 are repeated, until video terminates.Compared with prior art, the present invention can realize the correct association between target, complete robust and lasting multiple target tracking in the case where target interaction and blocking, having the complex situations such as similar appearance between target.

Description

One kind being based on deep learning and the associated online multi-object tracking method of data
Technical field
It is the present invention relates to a kind of method for tracking target, in particular to a kind of associated online more based on deep learning and data Method for tracking target belongs to computer vision field.
Background technique
Multitarget Tracking is an especially important branch in computer vision field, is widely used in various Video analysis scene, such as autonomous driving vehicle, robot navigation, Intelligent traffic video monitoring and motion analysis etc..
The task of online multiple target tracking is reliably to estimate position and the same target of across frame tracking of target frame by frame Estimate the track of multiple targets.In recent years, due to the development of deep learning, the performance of algorithm of target detection is constantly promoted, detection Respond relatively reliable, tracking (Tracking-by-detection) frame based on detection receives significant attention, and achieves aobvious The effect of work becomes the mainstream of current multiple target tracking.Under this tracking frame, target inspection good using off-line training first It surveys device independently to detect the target in every frame image, obtains the number and location of target, then, according to the outer of target The information such as sight, movement, the target detected in consecutive frame is associated, realizes the matching and tracking of target.Based on detection Track algorithm can be divided into two classes: off-line tracking and online tracking.
Currently, the track algorithm based on detection is also faced with lot of challenges, tracking effect depends critically upon the property of detector Can, in complicated scene, when serious block occurs between target and barrier or target, multiple target tracking algorithm It is easy to that entanglement occurs with losing target or target designation.Secondly, object detector detection noise and target scale it is violent Variation also results in multiple target tracking algorithm and tracking drift occurs.
Summary of the invention
Goal of the invention: when mutually being blocked for the target in complex scene with similar appearance, existing multiple target tracking There is the problems such as serious number switching, tracking drift in technology, it is associated based on deep learning and data that the invention proposes one kind Online multi-object tracking method.
The invention proposes a kind of new multi-object tracking methods, solve the problems, such as multiple target tracking from multiple angles.1) it adopts With the depth cosine metric learning modelling display model of target, spy is extracted from target image using multilayer convolutional network Sign realizes effective identification of different target appearance using the cosine between feature vector as the similitude between target appearance；2) In view of the continuity of target appearance dynamic change, a kind of target appearance similarity measurements for merging multiframe history external appearance characteristic are constructed Amount method, defect or target that detector can be effectively relieved mutually block the influence to object matching precision；3) it proposes to be based on mesh The two stages data correlation method of mark state separately designs corresponding associating policy for the reliability of target, and uses breast tooth Sharp algorithm carries out data correlation.It is crowded, frequently block the vehicles in complex traffic scene of generation under, which is able to achieve is accurate, stablizes Multiple target tracking.
Technical solution: one kind being based on deep learning and the associated online multi-object tracking method of data, which is characterized in that institute The method of stating includes the following steps:
Step 1: the image of input video present frame；
Step 2: application target detector obtains the set D of all detection responses in imaget={ D1, D2..., DM, t is Current frame number, DjFor j-th of detection response, it is expressed asWhereinD is responded for detectionj's Center point coordinate,D is responded for detectionjWidth and height, M be detection response sum；
Step 3: utilizing depth cosine metric learning model from detection response sets DtIn all detections response extract it is outer Feature vector is seen, { Z is expressed as1, Z2..., ZM, wherein Zj∈RpD is responded for detectionjExternal appearance characteristic；
Step 4: dbjective state is divided into 4 classes by initialized target state: original state, tracking mode, lost condition and being deleted Except state；If t=1, i.e. the first frame of input video generates target collection Tt={ T1, T2..., TN, N=M, target TjWith Detection response DjIt is corresponding, and by target TjState be set to original state, go to step 1；Otherwise, step 5 is gone to；
Step 5: applying Kalman filtering algorithm, predict target collection Tt-1In each target TiPosition in the current frame It sets and scale, is expressed asWhereinFor the center point coordinate of prediction,For prediction Width and height；
Step 6: target being associated with detection response matching based on two stages data correlation, obtains optimal association results；
Step 7: according to the state and feature of the optimal association results more fresh target in step 6；
Step 8: inputting the image of next video frame, repeat step 2,3,4,5,6,7 until video terminates.
Preference, dbjective state of the step 6 based on two stages data correlation are associated with the matching of detection response, are wrapped It includes:
(a) state based on targets all in former frame, by target collection Tt-1={ T1, T2..., TNIt is divided into two classes Ω1And Ω2, Ω1∪Ω2=Tt-1, Ω1It is made of the target in original state and tracking mode, Ω2By being in lost condition Target composition, N be target sum；
(b) Ω is calculated1In each target and DtIn each detection response matching similarity, obtain similarity matrix A1；With-A1To be associated with cost matrix, by Ω1In target and DtIn detection response be associated, asked using Hungary Algorithm Solve optimal association；According to association results by Ω1With DtIt is divided:WhereinIn Target and DAIn detection respond successful association,For not associated target collection, DBFor the detection that the first stage is not associated Response sets；
(c) Ω is calculated2In each target and DBIn each detection response matching similarity, obtain similarity matrix A2；With-A2To be associated with cost matrix, by Ω2In target and DBIn detection response be associated, asked using Hungary Algorithm Solve optimal association.According to association results by Ω2With DBIt is divided: Wherein In target with For not associated target collection,It is not associated for second stage Detection response sets.
Preference, the method calculate Ω1In each target and DtIn each detection response matching similarity, packet It includes:
(a) Ω is calculated1In target TiWith DtIn detection respond DjAppearance similarity degree
And
Wherein<*, *>be vector inner product, Xi(t-K) target T is indicatediExternal appearance characteristic vector in t-k frame, ZjTable Show detection response DjExternal appearance characteristic vector, ωkIndicate external appearance characteristic vector Xi(t-k) weight, CiIt (t-k) is target Ti? The matching cost of t-k frame and detection response；
(b) Ω is calculated1In target TiWith DtIn detection respond DjShape similarity
(c) Ω is calculated1In target TiWith DtIn detection respond DjKinematic similitude degree
For target TiEstimation rangeD is responded with detectionjCorresponding regionFriendship and than (IOU), wherein area (*) indicates area；
(d) Ω is calculated1In target TiWith DtIn detection respond DjMatching similarity A1(i, j):
Preference, the method calculate Ω2In each target and DBIn each detection response matching similarity, packet It includes:
(a) Ω is calculated using above-mentioned formula (1), (2), (3)2In target TiWith DBIn detection respond DjAppearance phase Like degreeAnd shape similarity
(b) target T is calculatediSearch radius ri:
WhereinFor current frame number and target TiThe difference of maximum frame number when in tracking mode, α are constant.With mesh Mark TiPredicted position in the current frameCentered on, riFor radius, target T is definediRegion of search Ri；
(c) Ω is calculated2In target TiWith detection response sets DBIn detection respond DjMatching similarity A2(i, j):
Wherein I (Ri∩Dj> 0) it is indicator function, as region of search RiD is responded with detectionjWhen in the presence of overlapping, I (Ri∩Dj > 0)=1, otherwise I (Ri∩Dj> 0)=0.
Preference, the step 7: according to the state and feature of the optimal association results more fresh target in step 6, comprising:
(a) forIn not associated detection response, indicate to be likely to occur fresh target in video, initialize fresh target, And state is set to original state.When f continuously occurs in the target of original stateinitFrame is then Target Assignment ID, and state is arranged Then target is converted to tracking mode by parameter；
(b) forIn target, due to keeping dbjective state constant, using Kalman there are associated detection response The state of filtering algorithm more fresh target, and target is saved in the external appearance characteristic vector of present frame；
(c) forIn target dbjective state is converted to by tracking mode since no associated detection responds Lost condition, and target is saved in the external appearance characteristic vector of present frame；
(d) forIn target, since there are associated detection response, dbjective state being converted to by lost condition Tracking mode using the state of Kalman filtering algorithm more fresh target, and saves target in the external appearance characteristic vector of present frame；
(e) forIn target keep dbjective state constant since no associated detection responds；
(f) as the continuous f of targetdelFrame is in lost condition, then is converted to deletion state, and destroys the target.
The utility model has the advantages that the 1, present invention is utilized by using the display model of depth cosine metric learning model learning target Multilayer convolutional network extracts feature from target image, using the cosine between feature vector as similar between target appearance Property, it realizes effective identification of different target appearance, effectively overcomes the target in complex scene with similar appearance in interaction When caused ID switching problem；2, the present invention considers the continuity of target appearance dynamic change, more by constructing a kind of fusion The target appearance method for measuring similarity of frame history external appearance characteristic effectively alleviates detector defect or target and mutually blocks pair The influence of matching precision；3, the present invention is by using the two stages data correlation method based on dbjective state, not for target Corresponding associating policy is separately designed with state, and data correlation is carried out using Hungary Algorithm, is effectively alleviated due to number (Fragment) problem is broken according to track caused by association failure.
Detailed description of the invention
Fig. 1 is flow chart of the invention；
Fig. 2 is the frame of depth cosine metric learning model of the invention；
Fig. 3 is that dbjective state of the invention shifts figure.
Specific embodiment
Technical solution of the present invention is further explained in detail below in conjunction with attached drawing and specific embodiment, with For line pedestrian's multiple target tracking, but the scope of the present invention is not limited to following embodiments.
Off-line training step:
Off-line training depth cosine metric learning model:
Given training sample set { (xi, yi), i=1,2,3 ..., L }, wherein xi∈R128×64For the pedestrian after standardization Image, yi∈ { 1,2,3 ..., K } is corresponding pedestrian's class label, and L is training sample number.Depth cosine metric learning mould Type learns a feature extraction function f (x) from training sample, and input pedestrian image is mapped in insertion feature space, Then cosine softmax classifier is applied in insertion feature space, maximizes the posterior probability of classification.Cosine softmax points Class device is defined as follows:
WhereinFor normalized weight vectors, ωkFor the weight vectors of kth class, τ is calibrating parameters, f It (x) is the feature vector extracted from image, f (x) has unit length.Due toUnit length is all had with f (x), in formula 'sIt is expressed as pressing from both sides cosine of an angle between two vectors, by maximizing posterior probability P (y=k | f (x)), can reduce often Angle between the corresponding weight vectors of class target.
For training the cross entropy loss function of depth cosine metric learning model are as follows:
Wherein I (yi=k) it is indicator function, work as yiWhen=k, I (yi=k)=1, otherwise I (yi=k)=0.
In the present embodiment, feature extraction function f (x) is realized using convolutional neural networks CNN, the structure of CNN such as Fig. 2 institute Show, input image size is 128 × 64, and output feature vector length is 128, and every layer of activation primitive is index linear unit (ELU).Using the above-mentioned network of pedestrian image training in Market-1501 database, and net is updated using Adam optimization method Network parameter.
Online pedestrian's multiple target tracking stage:
Specifically, as shown in Figure 1, the invention proposes one kind based on deep learning and the associated online multiple target of data with Track method, steps are as follows for the key technology of this method:
Step 1: the image of input video present frame；
Step 2: obtaining the set D of all detection responses in image using detectort={ D1, D2..., DM, t is current Frame number, DjFor j-th of detection response, it is expressed asWhereinD is responded for detectionjCentral point Coordinate,D is responded for detectionjWidth and height, M be detection response sum；
In the present embodiment, the pedestrian detector used is DPM (Deformable Parts Model).
Step 3: using the good depth cosine metric learning model of above-mentioned off-line training from detection response sets DtIn institute There is detection response to extract external appearance characteristic vector, is expressed as { Z1, Z2..., ZM, wherein Zj∈RpD is responded for detectionjThe appearance of extraction Feature；
Step 4: initialized target state.Dbjective state is divided into 4 classes: original state, tracking mode and is deleted lost condition Except state.If t=1, i.e. the first frame of input video generates target collection Tt={ T1, T2..., TN, N=M, target TjWith Detection response DjIt is corresponding, and by target TjState be set to original state, go to step 1.Otherwise, step 5 is gone to.
Step 5: applying Kalman filtering algorithm, predict target collection Tt-1In each target TjPosition in the current frame It sets and scale, is expressed asWhereinFor the center point coordinate of prediction,For prediction Width and height；
Step 6: target being associated with detection response matching based on two stages data correlation, obtains optimal association results；
6.1: the state based on targets all in former frame, by target collection Tt-1={ T1, T2..., TNIt is divided into two classes Ω1And Ω2, Ω1∪Ω2=Tt-1, Ω1It is made of the target in original state and tracking mode, Ω2By being in lost condition Target composition, N be target sum；
6.2: calculating Ω1In each target and DtIn each detection response matching similarity, obtain similarity matrix A1, with-A1To be associated with cost matrix, by Ω1In target and DtIn detection response be associated, asked using Hungary Algorithm Solve optimal association；According to association results by Ω1With DtIt is divided:Dt=DA∪DB, whereinIn mesh Mark and DAIn detection respond successful association,For not associated target collection, DBFor the detection response that the first stage is not associated Set.Calculate similarity matrix A1Specific step is as follows:
(a) Ω is calculated1In target TiWith DtIn detection respond DjAppearance similarity degree
And
Wherein<*, *>be vector inner product, Xi(t-K) external appearance characteristic vector of the target Ti in t-k frame, Z are indicatedjTable Show detection response DjExternal appearance characteristic vector, ωkIndicate external appearance characteristic vector Xi(t-k) weight, CiIt (t-k) is target Ti? The matching cost of t-k frame and detection response.
In the present embodiment, history external appearance characteristic of the target in nearest 6 frame, i.e. K=6 are saved.
(b) Ω is calculated1In target TiWith DtIn detection respond DjShape similarity
(c) Ω is calculated1In target TiWith DtIn detection respond DjKinematic similitude degree
For target TiEstimation rangeD is responded with detectionjCorresponding regionFriendship and than (IOU), wherein area (*) indicates area.
(d) Ω is calculated1In target TiWith DtIn detection respond DjMatching similarity A1(i, j):
6.3: calculating Ω2In each target and DBIn each detection response matching similarity, obtain similarity matrix A2；With-A2To be associated with cost matrix, by Ω2In target and DBIn detection response be associated, asked using Hungary Algorithm Solve optimal association.According to association results by Ω2With DBIt is divided: Wherein In target withIn detection respond successful association,For not associated target collection,It is not associated for second stage Detection response sets.Calculate similarity matrix A2Specific step is as follows:
(a) Ω is calculated using above-mentioned formula (3), (4), (5)2In target TiWith DBIn detection respond DjAppearance phase Like degreeShape similarity
(b) target T is calculatediSearch radius ri:
In the present embodiment, α takes 0.15.
WhereinThe difference of maximum frame number when being in tracking mode for current frame number and target Ti, α is constant.
(c) with target TiPredicted position in the current frameCentered on, riFor radius, target T is definediSearch Rope region Ri。
(d) Ω is calculated2In target TiWith DBIn detection respond DjMatching similarity A2(i, j):
Wherein I (Ri∩Dj> 0) it is indicator function, when detection responds DjWith region of search RiWhen in the presence of overlapping, I (Ri∩Dj > 0)=1, otherwise I (Ri∩Dj> 0)=0.
Step 7: as shown in figure 3, according to the state and feature of the optimal association results more fresh target in step 6, it is specific to walk It is rapid as follows:
(a) forIn not associated detection response, indicate to be likely to occur fresh target in video, initialize fresh target, And state is set to original state.When f continuously occurs in the target of original stateinitFrame is then Target Assignment ID, and state is arranged Then target is converted to tracking mode by parameter.
(b) forIn target, due to keeping dbjective state constant, using Kalman there are associated detection response The state of filtering algorithm more fresh target, and target is saved in the external appearance characteristic vector of present frame.
(c) forIn target dbjective state is converted to by tracking mode since no associated detection responds Lost condition, and target is saved in the external appearance characteristic vector of present frame.
(d) forIn target, since there are associated detection response, dbjective state being converted to by lost condition Tracking mode using the state of Kalman filtering algorithm more fresh target, and saves target in the external appearance characteristic vector of present frame.
(e) forIn target keep dbjective state constant since no associated detection responds.
(f) as the continuous f of targetdelFrame is in lost condition, then is converted to deletion state, and destroys the target.
In the present embodiment, f is takeninit=3, fdel=20.
Step 8: inputting the image of next video frame, repeat step 2,3,4,5,6,7 until video terminates.
Implementation result:
According to above-mentioned steps, we have carried out reality on the MOT16 data set of multiple target tracking challenge MOT Challenge It tests.All experiments all realize in PC machine, the major parameter of the PC machine are as follows: central processing unit Intel Core i7 2.3GHz, 16G memory.Algorithm is realized with Python.
The results show that the technical program can effectively track the pedestrian being detected in video, block as pedestrian or There are lasting tracking is also able to achieve when detection noise, the correct track of target is exported.Moreover, program operational efficiency is higher, about 1 Second it can handle 10 frame input pictures.This experiment shows that the multiple target tracking algorithm of the present embodiment can be realized accurately and rapidly Online pedestrian tracking.
To sum up, the invention proposes one kind to be based on deep learning and the associated online multi-object tracking method of data. This method is widely used in the target following under various video scenes, such as the pedestrian tracking under video monitoring scene, pacifies for wisdom Anti- system provides the vehicle tracking under technical support and vehicles in complex traffic scene, provides technical support for automatic Pilot technology.This Invention follows the tracking frame based on detection, data correlation problem is converted by online multiple target tracking problem, first with instruction The object detector perfected extracts all detections response in image；Then utilize depth cosine metric learning model from each inspection It surveys response and extracts external appearance characteristic vector；Different target and detection are calculated in conjunction with clues such as the appearance of target, movement and shapes Association cost between response；The Optimum Matching of target and detection is realized using Hungary Algorithm in two stages data correlation, Finally dbjective state is updated according to association results.
Particular embodiments described above has carried out further background of the invention, technical scheme and beneficial effects It is described in detail.As it will be easily appreciated by one skilled in the art that the foregoing is merely a specific embodiments of the invention, not For limiting the scope of protection of the present invention completely.Note that those skilled in the art, it is all in spirit and original of the invention Any modification, equivalent substitution, improvement and etc. done within then, should all be included in the protection scope of the present invention.

Claims (5)

1. one kind is based on deep learning and the associated online multi-object tracking method of data, which is characterized in that the method includes Following steps:
Step 1: the image of input video present frame；
Step 2: application target detector obtains the set D of all detection responses in imaget={ D1, D2..., DM, t is present frame Number, DjFor j-th of detection response, it is expressed asWhereinD is responded for detectionjCentral point sit Mark,D is responded for detectionjWidth and height, M be detection response sum；
Step 3: utilizing depth cosine metric learning model from detection response sets DtIn all detections response extract external appearance characteristic Vector is expressed as { Z1, Z2..., ZM, wherein Zj∈RpD is responded for detectionjExternal appearance characteristic；
Step 4: dbjective state is divided into 4 classes: original state, tracking mode, lost condition and deletion shape by initialized target state State；If t=1, i.e. the first frame of input video generates target collection Tt={ T1, T2..., TN, N=M, target TjWith detection Respond DjIt is corresponding, and by target TjState be set to original state, go to step 1；Otherwise, step 5 is gone to；
Step 5: applying Kalman filtering algorithm, predict target collection Tt-1In each target TiPosition and ruler in the current frame Degree, is expressed asWhereinFor the center point coordinate of prediction,For the width and height of prediction；
Step 6: target being associated with detection response matching based on two stages data correlation, obtains optimal association results；
Step 7: according to the state and feature of the optimal association results more fresh target in step 6；
Step 8: inputting the image of next video frame, repeat step 2,3,4,5,6,7 until video terminates.
2. it is according to claim 1 a kind of based on deep learning and the associated online multi-object tracking method of data, it is special Sign is that dbjective state of the step 6 based on two stages data correlation is associated with the matching of detection response, comprising:
(a) state based on targets all in former frame, by target collection Tt-1={ T1, T2..., TNIt is divided into two class Ω1With Ω2, Ω1∪Ω2=Tt-1, Ω1It is made of the target in original state and tracking mode, Ω2By the target for being in lost condition Composition, N are target sum；
(b) Ω is calculated1In each target and DtIn each detection response matching similarity, obtain similarity matrix A1； With-A1To be associated with cost matrix, by Ω1In target and DtIn detection response be associated, most using Hungarian Method Excellent association；According to association results by Ω1With DtIt is divided:Dt=DA∪DB, whereinIn target with DAIn detection respond successful association,For not associated target collection, DBCollection is responded for first stage not associated detection It closes；
(c) Ω is calculated2In each target and DBIn each detection response matching similarity, obtain similarity matrix A2； With-A2To be associated with cost matrix, by Ω2In target and DBIn detection response be associated, most using Hungarian Method Excellent association.According to association results by Ω2With DBIt is divided:WhereinIn Target withIn detection respond successful association,For not associated target collection,For the not associated inspection of second stage Survey response sets.
3. it is according to claim 2 a kind of based on deep learning and the associated online multi-object tracking method of data, it is special Sign is that the method calculates Ω1In each target and DtIn each detection response matching similarity, comprising:
(a) Ω is calculated1In target TiWith DtIn detection respond DjAppearance similarity degree
And
Wherein<*, *>be vector inner product, Xi(t-K) target T is indicatediExternal appearance characteristic vector in t-k frame, ZjIndicate inspection Survey response DjExternal appearance characteristic vector, ωkIndicate external appearance characteristic vector Xi(t-k) weight, CiIt (t-k) is target TiIn t-k The matching cost of frame and detection response；
(b) Ω is calculated1In target TiWith DtIn detection respond DjShape similarity
(c) Ω is calculated1In target TiWith DtIn detection respond DjKinematic similitude degree
For target TiEstimation rangeD is responded with detectionjCorresponding region Friendship and than (IOU), wherein area (*) indicates area；
(d) Ω is calculated1In target TiWith DtIn detection respond DjMatching similarity A1(i, j):
4. it is according to claim 2 a kind of based on deep learning and the associated online multi-object tracking method of data, it is special Sign is that the method calculates Ω2In each target and DBIn each detection response matching similarity, comprising:
(a) Ω is calculated using above-mentioned formula (1), (2), (3)2In target TiWith DBIn detection respond DjAppearance similarity degreeAnd shape similarity
(b) target T is calculatediSearch radius ri:
WhereinFor current frame number and target TiThe difference of maximum frame number when in tracking mode, α are constant.With target Ti Predicted position in the current frameCentered on, riFor radius, target T is definediRegion of search Ri；
(c) Ω is calculated2In target TiWith detection response sets DBIn detection respond DjMatching similarity A2(i, j):
Wherein I (Ri∩Dj> 0) it is indicator function, as region of search RiD is responded with detectionjWhen in the presence of overlapping, I (Ri∩Dj> 0) =1, otherwise I (Ri∩Dj> 0)=0.
5. it is according to claim 1 a kind of based on deep learning and the associated online multi-object tracking method of data, it is special Sign is, the step 7: according to the state and feature of the optimal association results more fresh target in step 6, comprising:
(a) forIn not associated detection response, indicate to be likely to occur fresh target in video, initialize fresh target, and by shape State is set to original state.When f continuously occurs in the target of original stateinitFrame is then Target Assignment ID, and state parameter is arranged, Then target is converted into tracking mode；
(b) forIn target, due to keeping dbjective state constant, using Kalman filtering there are associated detection response The state of algorithm more fresh target, and target is saved in the external appearance characteristic vector of present frame；
(c) forIn target dbjective state is converted into loss by tracking mode since no associated detection responds State, and target is saved in the external appearance characteristic vector of present frame；
(d) forIn target, since there are associated detection response, dbjective state is converted to tracking by lost condition State using the state of Kalman filtering algorithm more fresh target, and saves target in the external appearance characteristic vector of present frame；
(e) forIn target keep dbjective state constant since no associated detection responds；
(f) as the continuous f of targetdelFrame is in lost condition, then is converted to deletion state, and destroys the target.
CN201910429444.XA 2019-05-22 2019-05-22 Online multi-target tracking method based on deep learning and data association Active CN110288627B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910429444.XA CN110288627B (en) 2019-05-22 2019-05-22 Online multi-target tracking method based on deep learning and data association

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910429444.XA CN110288627B (en) 2019-05-22 2019-05-22 Online multi-target tracking method based on deep learning and data association

Publications (2)

Publication Number Publication Date
CN110288627A true CN110288627A (en) 2019-09-27
CN110288627B CN110288627B (en) 2023-03-31

Family

ID=68002271

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910429444.XA Active CN110288627B (en) 2019-05-22 2019-05-22 Online multi-target tracking method based on deep learning and data association

Country Status (1)

Country Link
CN (1) CN110288627B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110796687A (en) * 2019-10-30 2020-02-14 电子科技大学 Sky background infrared imaging multi-target tracking method
CN111476826A (en) * 2020-04-10 2020-07-31 电子科技大学 Multi-target vehicle tracking method based on SSD target detection
CN111932588A (en) * 2020-08-07 2020-11-13 浙江大学 Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning
CN112149762A (en) * 2020-11-24 2020-12-29 北京沃东天骏信息技术有限公司 Target tracking method, target tracking apparatus, and computer-readable storage medium
CN112163473A (en) * 2020-09-15 2021-01-01 郑州金惠计算机系统工程有限公司 Multi-target tracking method and device, electronic equipment and computer storage medium
CN112581496A (en) * 2019-09-29 2021-03-30 四川大学 Multi-target pedestrian trajectory tracking method based on reinforcement learning
CN113077495A (en) * 2020-01-06 2021-07-06 广州汽车集团股份有限公司 Online multi-target tracking method, system, computer equipment and readable storage medium
CN117292327A (en) * 2023-11-23 2023-12-26 安徽启新明智科技有限公司 Method, device, equipment and medium for associating targets
CN117495917A (en) * 2024-01-03 2024-02-02 山东科技大学 Multi-target tracking method based on JDE multi-task network model

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835178A (en) * 2015-02-02 2015-08-12 郑州轻工业学院 Low SNR(Signal to Noise Ratio) motion small target tracking and identification method
CN109360226A (en) * 2018-10-17 2019-02-19 武汉大学 A kind of multi-object tracking method based on time series multiple features fusion

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835178A (en) * 2015-02-02 2015-08-12 郑州轻工业学院 Low SNR(Signal to Noise Ratio) motion small target tracking and identification method
CN109360226A (en) * 2018-10-17 2019-02-19 武汉大学 A kind of multi-object tracking method based on time series multiple features fusion

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112581496A (en) * 2019-09-29 2021-03-30 四川大学 Multi-target pedestrian trajectory tracking method based on reinforcement learning
CN110796687A (en) * 2019-10-30 2020-02-14 电子科技大学 Sky background infrared imaging multi-target tracking method
CN113077495A (en) * 2020-01-06 2021-07-06 广州汽车集团股份有限公司 Online multi-target tracking method, system, computer equipment and readable storage medium
CN111476826A (en) * 2020-04-10 2020-07-31 电子科技大学 Multi-target vehicle tracking method based on SSD target detection
CN111932588A (en) * 2020-08-07 2020-11-13 浙江大学 Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning
CN111932588B (en) * 2020-08-07 2024-01-30 浙江大学 Tracking method of airborne unmanned aerial vehicle multi-target tracking system based on deep learning
CN112163473A (en) * 2020-09-15 2021-01-01 郑州金惠计算机系统工程有限公司 Multi-target tracking method and device, electronic equipment and computer storage medium
CN112149762A (en) * 2020-11-24 2020-12-29 北京沃东天骏信息技术有限公司 Target tracking method, target tracking apparatus, and computer-readable storage medium
CN117292327A (en) * 2023-11-23 2023-12-26 安徽启新明智科技有限公司 Method, device, equipment and medium for associating targets
CN117495917A (en) * 2024-01-03 2024-02-02 山东科技大学 Multi-target tracking method based on JDE multi-task network model
CN117495917B (en) * 2024-01-03 2024-03-26 山东科技大学 Multi-target tracking method based on JDE multi-task network model

Also Published As

Publication number Publication date
CN110288627B (en) 2023-03-31

Similar Documents

Publication Publication Date Title
CN110288627A (en) One kind being based on deep learning and the associated online multi-object tracking method of data
Yu et al. Harry potter's marauder's map: Localizing and tracking multiple persons-of-interest by nonnegative discretization
Choi et al. A general framework for tracking multiple people from a moving camera
Liu et al. Improved human action recognition approach based on two-stream convolutional neural network model
Rout A survey on object detection and tracking algorithms
CN104268598B (en) Human leg detection method based on two-dimensional scanning lasers
CN104680559B (en) The indoor pedestrian tracting method of various visual angles based on motor behavior pattern
CN109191497A (en) A kind of real-time online multi-object tracking method based on much information fusion
Ji et al. RGB-D SLAM using vanishing point and door plate information in corridor environment
CN106127125A (en) Distributed DTW human body behavior intension recognizing method based on human body behavior characteristics
CN103150552B (en) A kind of driving training management method based on number of people counting
CN112861808B (en) Dynamic gesture recognition method, device, computer equipment and readable storage medium
Lalos et al. Efficient tracking using a robust motion estimation technique
Prokaj et al. Using 3d scene structure to improve tracking
Hu et al. Human interaction recognition using spatial-temporal salient feature
Ding et al. Simultaneous body part and motion identification for human-following robots
Mbelwa et al. Visual tracking tracker via object proposals and co-trained kernelized correlation filters
Bruckschen et al. Detection of generic human-object interactions in video streams
Xu et al. Deep learning technique based surveillance video analysis for the store
Jun et al. Online visual multiple target tracking by intuitionistic fuzzy data association
Wei et al. Object clustering with Dirichlet process mixture model for data association in monocular SLAM
CN105809719A (en) Object tracking method based on pixel multi-coding-table matching
Rougier et al. 3D head trajectory using a single camera
JP2019174910A (en) Information acquisition device and information aggregation system and information aggregation device
Chen et al. Fast compressive tracking combined with Kalman filter

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant